Brad Boyes whistled a shot behind goalie Reto Berra at 2:23 of the period 63 seconds after Avalanche defenseman Brad Stuart was penalized for tripping Aleksander Barkov in the corner in Colorado's end.
The Panthers were in a 2-for-24 drought with the man advantage before Boyes found the back of the net off a pass from Jonathan Huberdeau.
"It was a play we drew up," Boyes said. "We worked on it a little bit in practice. Everyone executed properly and it was good play off good motion. It's good to win games like this. You lose the lead and then battle back."
The Panthers (2-2-2) took a 2-0 lead in the first period on goals by Brandon Pirri and Barkov, matching their season high for goals in a game. They didn't score a goal in the first period in their previous five games.
The Avalanche (1-4-2) scored three times in the second period, two on power plays 31 seconds apart, to grab a 3-2 lead before the Panthers tied the game at 17:41 on a goal by Sean Bergenheim.
"You saw it. You get pretty frustrated," Panthers coach Gerard Gallant said. "The good thing is we battled back after getting pretty frustrated, taking some penalties and they scored some power-play goals. The guys held their composure. We knew we played a good first half of the game, battled back to tie it up."
Berra, who made 33 saves, was screened on the winning goal.
"On four-against-three they had a lot of room," he said. "It was a little screen and I have no idea where it went in, somewhere on the stick side."
Both teams came in struggling to score. The Panthers had scored five goals in their first five games; the Avalanche had nine goals in six games.
Panthers goalie Roberto Luongo made 30 saves. He has not lost to the Avalanche in regulation in 15 games (13-0-2) and collected his 375th career win to move past John Vanbiesbrouck into 13th place on the all-time list.
"I'm fortunate," Luongo said. "It seems you have success like this against one or two teams."
Turnovers continue to plague the Avalanche, a problem that has frustrated coach Patrick Roy. A giveaway led to Bergenheim's tying goal. He corralled the puck in front of the Avalanche goal despite being checked by Nathan MacKinnon after a point shot by Dylan Olsen and poked it past Berra.
"We're not sharp mentally, we're making bad decisions," Roy said. "We make too many giveaways and we're losing easy one-on-one battles, then we're going to give 35 shots per night. This is a game of one-on-one battles. Yeah, we battled back and I'm very happy about that, but we need to play 60 minutes."
After going 1-for-22 on the power play in their first six games, the Avalanche got a goal from Tyson Barrie on a two-man advantage at 10:51 of the second period and one from Gabriel Landeskog at 11:22 on a 5-on-4 situation.
Barrie became the first Colorado defenseman to score a goal this season when he beat Luongo with a one-timer from the left point with Boyes (high sticking) and Erik Gudbranson (delay of game) sitting side-by-side in the penalty box.
Landeskog scored when he drove to the front of the net and chipped the puck over Luongo's right shoulder.
The Avalanche took a brief 3-2 lead at 14:41 on Alex Tanguay's team-leading third goal. Tanguay stripped the puck from Florida defenseman Aaron Ekblad in the neutral zone and took a shot that went in off Bergenheim's skate.
Pirri opened the scoring at 4:43 with his first goal. Jussi Jokinen was in the left circle when he intercepted Avalanche defenseman Ryan Wilson's clearing attempt and fed Pirri in the high slot for a quick blast past Berra.
Barkov scored his first goal of the season at 13:03 to complete a pretty passing play with Boyes and Ekblad. Ekblad passed to Barkov cutting to the net for a tap inside the left post.
Wilson sustained a shoulder injury in the second period and didn't return. He sat out one game last week because of a head injury.