With Florida's Dmitry Kulikov off for holding in overtime, Grant Clitsome fired a slap shot from the right circle that Umberger deflected past goalie Tomas Vokoun.
"I'm standing in front trying to stand in traffic and it went off my shaft and up into the corner," Umberger said. "We've been fighting and clawing the last few games, and we've had a lot of overtime games that we didn't get the second point and tonight we found a way. We needed that big-time."
Umberger left soon after the game to be with his wife, who is expecting to give birth at any time.
On the tying goal, Vermette scored on a backhander after Vokoun blocked Clitsome's shot.
"It's always exciting. A lot of emotion when you win at the end like that," Vermette said.
Cory Stillman and David Booth had first-period goals for Florida.
Vokoun made 25 saves in his first start after sitting out the last three games.
"It wasn't about giving up a goal in the last minute, it's the whole game and how we play," Vokoun said. "We were asking for it the whole third period. We had chances but we didn't score. We just do it to ourselves."
The Panthers blew a lead late for the second straight game. On Monday, they lost to Atlanta by allowing two goals in the final 2 minutes of regulation and lost in a shootout.
"This is a repeat of a lot of other nights," Panthers coach Peter De Boer said. "We get the lead, have some opportunities to extend it, we don't, and the other team pushes back. We lose our composure in the last few minutes of the game. That's the story, it's blown chances."
The Blue Jackets thought they had tied it with 2:38 left in the second on a shot by Derick Brassard, but a video review showed that the puck skimmed the crossbar and stayed out of the net.
The Blue Jackets closed to 2-1 with 1:19 left in the first period. The puck came around the end boards to Tyutin just inside the blue line at the left point and his slap shot got past Vokoun.
"We definitely knew what had happened to them the last game," Mason said. "We weren't happy with how we played early, but we were only down 2-1 after one period. I think the guys had a lot of confidence going into the second and third periods and we were rewarded for it."
Florida made it 2-0 at 8:21 on a fluke play. Radek Dvorak took a shot from the right circle that bounced off Mason. It then hit Stillman in the head and bounced into the net.
Booth opened the scoring at 2:43 with his 14th goal of the season. Kulikov took a shot from the point that was blocked by Mason. Mike Santorelli got a stick on it before the puck came out to the high point. Booth grabbed it and his wrist shot beat Mason. The Panthers have scored first in 10 of their past 13 games.
Material from wire services and team media was used in this report.