The loss ended New York's eight-game winning streak.
"It was a hard game," Rangers coach Alain Vigneault said. "Both teams battled hard. Both teams had some great looks. I think what probably made the difference a little bit is our inability to get through the neutral zone or their ability to stop us from getting through. They played a good game. They made us pay for a couple of our mistakes and they won the game."
Kari Lehtonen made 28 saves for Dallas (16-14-5), which has won two straight and six of seven.
The Stars are 11-1-2 when leading after two periods, a source of pride for coach Lindy Ruff.
"I thought we played good (in the third period)," Ruff said. "We played smart. We were diligent in not giving the puck away. I thought we forechecked hard when we had some opportunities. I thought with their goalie out we stayed aggressive. We really didn't give them any set-up time. There was a lot of good in that third period."
Roussel scored his eighth goal 54 seconds into the second period when he collected a carom off the end boards and tapped it into an empty net.
Rangers goaltender Henrik Lundqvist, who made 24 saves, had stopped the initial shot from Ryan Garbutt, sending the puck wide of the net with a pad save. The puck took a weird bounce off the boards behind the net and landed right at Roussel's skates, and he capitalized.
"I'm not really sure (what happened on that goal)," Rangers defenseman Dan Girardi said. "[Garbutt] kind of threw it across. [Lundqvist] made a good save. It gets in his stick and comes right back out front. That was kind of a bit unlucky there."
Vigneault chalked up the loss to one simple yet inescapable fact.
"I just think that we didn't manage the puck as well as we needed to," Vigneault said.
The Rangers (19-11-4) struck first when Miller scored a power-play goal 5:57 into the first period. Miller received the puck from Mats Zuccarello at the Dallas blue line, then beat Lehtonen with a wrist shot over the goalie's left shoulder from the slot for his fifth goal.
Dallas answered at 9:02 with a power-play goal from Klingberg from the high slot. Klingberg's initial shot was blocked by New York defenseman John Moore. However, Klingberg's second attempt sailed under Lundqvist's blocker for his fourth goal, first in 13 games.
"It's a big thing for me," Klingberg said. "But it's just a goal, and it feels good to score again. And it's obviously on Henrik Lundqvist, so that's an extra thing, so it's kind of fun."
The Stars took their first lead with 4:14 left in the first when a Hemsky centering attempt deflected in off the stick of Rangers defenseman Marc Staal. Hemsky was behind the New York net when his backhand pass struck the shaft of Staal's stick and beat Lundqvist on his short side.
The Rangers wasted little time tying the score. Kevin Hayes redirected a Staal shot for his fifth goal one minute after Hemsky had scored.
Early in the second, with the Stars ahead 3-2, Dallas rookie defenseman Jyrki Jokipakka made a goal-saving play with a block on Moore's shot as Lehtonen had left the net unprotected.
"That was huge," Stars defenseman Alex Goligoski said. "I was already out of it and looking at the other corner and I was just listening to what was happening next and I didn't know he got it. But I owe him a big thank you, maybe a light beer."
Lehtonen preserved the lead at 11:44 of the third period when he denied New York defenseman Dan Boyle at close range. Boyle's shot struck Lehtonen near his glove hand after the Dallas goalie moved out to cut down Boyle's angle.
"[Lehtonen] had some huge [saves]," Goligoski said. "He had huge ones in the first and second too, especially that one in the third. I think we fed off of that one really well, and we tightened things up pretty good. He was excellent all night."
Lundqvist headed to the bench with 1:29 remaining in the third period. Dallas missed the empty net several times, but New York couldn't find the equalizer and took its first loss since Dec. 6 at the Detroit Red Wings.
"It's disappointing," Lundqvist said. "It doesn't matter how many games you win, when you lose, you're disappointed. Talked about it the other day, we have so much hockey in front of us that you need to go after it every single night. You use this as motivation for next game, last game of the year try to end on a high note."
And even though New York lost for the first time in 23 days, Rangers defenseman Ryan McDonagh saw one big positive to take away from the loss.
"I don't think we took the foot off the gas pedal. We tried to create and had some push there," McDonagh said.