Rick Nash scored the game-winning goal with 1:21 to go in the third period to give the Rangers its only lead of the game to cap off a night that saw five different Blueshirts find the back of the net.
Kevin Hayes scored a goal and two assists while Michael Grabner and J.T. Miller each had a goal and an assist. Jesper Fast picked up his first goal of the season and Brady Skjei had two assists.
Nash was able to knock down a Nick Holden shot from the point, control the puck and slide it around Cam Talbot's pad to put the Blueshirts up 4-3. Miller would ice the game with an empty-net goal with 1:04 left in the game.
The two teams were tied, 2-2, heading into the third period after Ryan Nugent-Hopkins' tally was answered by Hayes' fourth goal of the season in the first period. In the middle frame, Patrick Maroon scored for the Oilers but that was answered by Grabner, who as of press time leads the NHL in even strength goals with six.
Edmonton would regain the lead at 8:49 of the third period, but Fast's deflection of Brandon Pirri's shot evened things up for the third time.
Nash's sixth goal of the season, which extended his point streak to three games.
Henrik Lundqvist made made 21 saves for his fifth win of the season.
The Hayes line combined for seven points on the evening and now has 18 points in its last three games.
"We just bring a different element to the line, every one," Grabner said. "It's been going good so far. At the same time, when we're not scoring we try to be responsible in our own end and don't make too many mistakes to feed their energy. I think it's been going pretty good and hopefully we can keep it up."
Head coach Alain Vigneault said although the win wasn't the best of the season, it showed what his team is made of.
"I really thought for us it wasn't our best game, but I thought it was a character win," Vigneault said. "We were down three times, we came back, found a way to win it with less than two minutes to go. It's about winning games and that's what we did tonight."
In addition to the comebacks, Lundqvist said he was impressed with the team's ability to respond as quickly as it did throughout the night.
"I think the key is just that we responded right away. When a team gets a lead and they get to play with a lead they get confident," Lundqvist said. "But I don't think they had a chance to really think about having the lead because we responded right away with a big play. I think that was the key for us to come back, how we responded in the first shift."
The Rangers head to Boston for a matchup with the Bruins Saturday night.
