Goalie Cam Talbot made 26 saves for the Rangers (32-16-5), who have won the first two games of a four-game road trip. They moved into third place in the Metropolitan Division, one point ahead of the Washington Capitals.
"I thought Cam played extremely well," Rangers coach Alain Vigneault said of Talbot, who is 3-1-1 while Henrik Lundqvist recovers from a neck injury. "Not a tremendous amount of work in the first period, a little more in the second and they blitzed us in the third. He gave us a chance to battle back and win the game."
Rick Nash and Ryan McDonagh each had a goal and an assist for New York. Hayes and linemates J.T. Miller and Carl Hagelin combined for two goals, three assists, eight of the Rangers' 31 shots and a plus-9 rating.
"Without a doubt, that line tonight was huge," Vigneault said. "They played an unbelievable game. Every time they were on the ice things were happening offensively. They helped us win this game."
The Avalanche (22-22-11) have lost four games in a row, all in regulation, and are eight points out of the second Western Conference wild-card position for the Stanley Cup Playoffs with 27 games remaining.
Down 3-1 heading to the third period, the Avalanche tied the game on goals by Jarome Iginla at 6:43 and Jan Hejda at 10:44. Iginla knocked in the rebound of Maxime Talbot's shot for his team-leading 17th goal and 577th of his NHL career, tying him with Mark Recchi for 19th place on the all-time list.
"We could have just rolled over, but as group in the third period I thought we came out right from the drop of the puck," Iginla said. "We put on a ton of pressure, took a ton of shots and tied it up. It hurts because we battled in a tough game, a rollercoaster game, and we didn't come out with a win."
But the Rangers responded with goals from Hagelin, who cashed in Hayes' pass at 12:02, and Nash, whose wraparound bounced into the net off Avalanche defenseman Nick Holden's stick at 15:42. Nash leads the NHL with 34 goals.
McDonagh scored a power-play goal into an empty net with 1:01 remaining.
The Avalanche took the first 13 shots of the third period before Hagelin scored to give the Rangers the lead for good.
"Obviously it gave us momentum," Hagelin said of his goal. "They were dominating for 10 minutes in the third. When you give a team like that a lot of room to skate, they're going to make plays."
Hayes and Mats Zuccarello scored second-period goals for the Rangers, giving them a 3-1 lead after 40 minutes.
Hayes, who set up Dan Boyle's first-period goal, wasn't touched as he skated from his own end into the Avalanche zone. He cut inside defenseman Tyson Barrie, moved through the slot and swept the puck past goalie Semyon Varlamov at 1:47.
"I was thinking 'get it deep,' but there was no one in front of me," said Hayes, who has two goals and four assists during a four-game scoring streak. "They gave me some space and I cut it back. Being able to score, get a couple of assists and help your team win, that's what you go for every night."
The Avalanche closed to 2-1 at 12:46 on a goal by Gabriel Landeskog. Colorado applied plenty of pressure in the Rangers' end and it paid off when Nathan MacKinnon took a shot that hit Landeskog in the chest as he was skating to the net. The puck bounced into the net over Talbot's left pad.
The Avalanche had a chance to tie the game less than a minute later during a two-man advantage that lasted 1:53 with Rangers J.T. Miller (hooking) and Dan Girardi (tripping) in the penalty box. They were held to one shot, finished the night 0-for-4 on the power play and are 0-for-31 in their past 10 games.
"It doesn't hurt as much as when they score right after it," Avalanche coach Patrick Roy said. "It was a moment where we had an opportunity to tie the game despite the fact that we were not playing our best hockey. I think we're maybe squeezing the stick a little too tight and our confidence is not there. It's my job to remain patient, continue to practice it and hopefully eventually it will turn and help us to win some hockey games."
Zuccarello, who has four goals and five assists during a seven-game points streak, then gave the Rangers a 3-1 lead at 16:02. Nash was above the right circle when he passed to Zuccarello in the left circle.
The Rangers grabbed a 1-0 lead at 16:30 of the first period on Boyle's goal. Taking advantage of a poor line change by the Avalanche, McDonagh passed to Hayes, who slipped the puck to Boyle with Hejda the lone defender back. Boyle skated in on a breakaway and shot the puck past Varlamov's right leg.