"You want to be playing strong especially during this time of year," Atkinson said. "These points are obviously crucial. It's crunch time, really. There's almost 10 games left -- 12 games for us -- but everyone is fighting and trying to gain momentum ... and I think for the most part that is what we have been doing."
WATCH: All Blue Jackets at Islanders highlights
Atkinson scored his 33rd goal of the season and seventh game-winner after a pass from Brandon Dubinsky from behind the net.
"I probably could have shot that at first as well," Atkinson said. "Dubinsky chose the harder route to go behind the net and find me to make a great play. [The Islanders] are playing for their lives, so obviously we expected them to come out hard."
The Islanders (33-26-12) lost for the fourth time in five games (1-3-1) and trail the Toronto Maple Leafs by one point for the second wild card into the Stanley Cup Playoffs from the Eastern Conference. The Maple Leafs lost 2-1 in overtime to the Chicago Blackhawks.
"It stings because it's important as it is and we are really starting to get down to the wire here," said New York captain John Tavares, who tied the game 2-2 at 8:48 of the third period. "Obviously we haven't been as consistent as we'd like, but a point is a point."
Columbus goaltender Joonas Korpisalo made 20 saves for his first win in three starts since a 7-0 victory against the Islanders in Columbus on Feb. 25.
"I'm very happy in how we played and found a way to win a close-checking, one-goal game," Blue Jackets coach John Tortorella said.
The Islanders took a 1-0 lead on a goal by defenseman Travis Hamonic at 14:18 of the first period. It was his third goal of the season, first in 35 games.
"You want to help out, you are trying to play your best," Hamonic said. "I'm just trying to get back in the groove of things. I thought tonight was nice to see that one go in and get some confidence back."
Oliver Bjorkstrand tied the game at 1-1 at 1:00 of the second period with his sixth goal of the season. After a turnover by New York defenseman Dennis Seidenberg at the blue line, Bjorkstrand skated into the zone and beat Thomas Greiss from the right circle with a wrist shot.