But afterwards, both forwards were quick to point to the save made by Jonathan Bernier with 3:18 remaining and the Leafs clinging to a one-goal lead as the game changer.
Bernier went post-to-post to rob Lightning captain Martin St. Louis of a sure goal as he one-timed a 2-on-1 feed from Victor Hedman. St. Louis, who had 14 goals and eight assists in 10 games coming in, was held pointless for the first time since he was left off the Canadian Olympic roster announced on Jan. 7.
"Great save. Game-saving save," said Kadri. "Bernie's been great for us all year and we almost expect him to make those kinds of saves since he's done it so often. He's a great goaltender, right up with the best and he defiantly showed it there."
Added van Riemsdyk: "He was incredible the whole night making big post-to-post saves playing really confidently. Again we're really lucky to have him."
Bernier, who made 40 saves in the win, improved to 7-1-2 when facing 40 or more shots this season.
"He had his blade to the inside so I just read it good and that one worked out for me," said Bernier of the save. "I don't think we played our best game tonight. They had a lot of 2-on-1's, but we stuck with it and James scored a big goal and Naz had a great game obviously."
Van Riemsdyk's 21st goal of the season at 15:57 of the third stood to be the game winner. With the goal, van Riemsdyk matched his single-season career best set during the 2010-11 season while he was with the Philadelphia Flyers.
Kadri scored twice in less than eight minutes to give the Maple Leafs (28-21-6) a 2-0 lead before the midway mark of the second period, but Matthew Carle and Mark Barberio responded for Tampa Bay (31-17-5) to tie it 2-2 after 40 minutes.
"We talked about that tonight, having a good stick and forcing their D to make a play under pressure," said van Riemsdyk of the goal. "(Tyler Bozak) was able to steal the puck from their guy, gave it to Phil and Phil slid it over to me."
The loss halted the Lightning's three-game win streak, while Bishop made 23 saves in the loss.
Kadri opened the scoring at 1:49 of the second period as his shot hit a Lightning player in front and fooled Bishop.
Then at 9:48, Kadri picked up his second of the night with a highlight-reel deke to step around Hedman and beat Bishop with a backhand for his 14th of the season. It was Kadri's second multi-goal game of the season.
"I should've had a few more. I had some great chances," said Kadri. "(Joffrey Lupul) and (Nikolai Kulemin) just seemed to find me no problem out there today, wish I could've had a couple more, but we'll save them for later."
Ondrej Palat fed Carle with a no-look feed at 13:13 of the second before the Lighting blue-liner beat Bernier stick-side.
Barberio tied it four minutes later, putting home the rebound off a scramble in front of Bernier for his fourth of the season and fourth in four games.
"I thought we deserved better tonight," St. Louis said. "But we've been on the other side of the coin, too, where you don't deserve a win but somehow you can get one. Tonight was one of those game that probably evened things out a little bit."
Toronto had three straight first-period power plays, but had just three shots to show for it. The Lightning outshot the Leafs 11-6 in the opening 20 minutes.
Leafs defenceman Tim Gleason left the game late in the third period after being hit awkwardly into the boards by Lightning forward Teddy Purcell. Randy Carlyle called it a shoulder issue, adding he didn't believe it was serious enough for the blue-liner to miss game time.
The Leafs continue their three-game home stand Thursday against the Florida Panthers.
Notes: Canadian Olympic hopeful and Lightning star Steven Stamkos missed his 36th game recovering from a broken right leg. The Markham, Ont., native returned to practice Tuesday after missing Monday with discomfort. ... Maple Leafs forward David Bolland (ankle) participated in his first game day skate since suffering the injury Nov. 2. Toronto defenceman Mark Fraser was a healthy scratch for the 10th straight game while Paul Ranger was scratched for his 11th consecutive game. ... Six-time Canadian Olympic medallist Clara Hughes dropped the ceremonial puck prior to the game.