Grabovski took a nice pass from Phil Kessel, who tied the game in the third period, and stuffed a shot behind Marc-Andre Fleury to bring the capacity crowd to its feet.
"It's good for our confidence," Toronto coach Ron Wilson said. "We were a little tentative, a little tight for most of the game. We found a way to hang in there."
The Leafs improved to 3-24-3 this season in games where they trail after two periods. They also moved back within four points of eighth-place Carolina heading into Thursday's road game in Philadelphia.
It was far from the best effort from a tired-looking Toronto team that still managed to improve to 9-2-4 since the all-star break.
The Penguins played their second consecutive game at Air Canada Centre. They returned to Pittsburgh for a couple days of practice after Saturday's 6-5 shootout victory over the Maple Leafs before coming back to Toronto.
The strain of playing seven straight one-goal games appeared to be catching up with the Leafs, who managed just 10 shots through 40 minutes.
But they were only trailing 2-1 heading to the final period and Kessel tied the score just over five minutes in. His eighth goal in as many games came off a nifty tip of a point shot from Carl Gunnarsson.
The Leafs were given a fantastic opportunity to avoid another overtime when Ben Lovejoy was penalized for tripping MacArthur with just over three minutes left in regulation. However, they were unable to generate many chances with the man advantage.
Toronto finished 0-for-5 on the power play.
Niskanen opened the scoring for Pittsburgh at 6:52 of the first period with his first goal of the season. Shortly after the Penguins won a faceoff in the offensive zone, he beat Leafs goaltender James Reimer with a shot from the point.
"I would have liked to see us get to 3-1," said Penguins coach Dan Bylsma.
"There was a couple rebound chances that looked to be pretty good opportunities from the bench. But we didn't get there and really, going into the third period with a 2-1 lead, we need to find ways to be better in those situations and close out that game regardless of whether we got that third goal or not."
The Leafs tied it up a little over seven minutes later. Kulemin scored his 23rd of the season with a well-placed wrist shot that eluded Fleury.
Conner's goal at 6:40 of the second period put Pittsburgh back in the lead. The rookie protected the puck while skating around defenseman Dion Phaneuf and slid it between Reimer's legs.
Grabovski missed a golden opportunity to tie the game before the end of the period when he had Fleury beat on a clear breakaway and shot wide.
"We came out storming in the third (period)," said Reimer, who made 27 saves.
"The guys deserved it. ... It was a real sweet victory for us, it was a good one to watch."
Material from wire services and team media was used in this report.
1 - 0 PIT
1 - 1 Tie
2 - 1 PIT
2 - 2 Tie
3 - 2 TOR