Travis Zajac scored, and Cory Schneider made 17 saves for the Devils (25-33-9), who have lost four games in a row.
Zajac tied the game 1-1 with a snap shot from the right face-off circle 17:00 into the second period after taking a pass from Coleman.
"With the amount of firepower they have, I thought we did a tremendous job defensively with making them work," Schneider said. "I thought we had a couple of chances to get the go-ahead goal and just missed or Bobrovsky made a big save. It wasn't good enough to get a win, but you know, that's kind of how it goes."
Atkinson gave the Blue Jackets a 1-0 lead when he controlled a shot attempt by David Savard off the end boards and scored with a snap shot at the right post 52 seconds into the game.
"I know you guys are going to keep asking me why we're struggling, but I don't have an answer," Tortorella said. "Every game in the National Hockey League is a hard game to play. I think we knew this was going to be a hard game to play."
Columbus has won seven straight games against New Jersey, including all four games this season.
Bobrovsky made the save against Kenny Agostino off a 3-on-1 breakout 9:12 into the third period to keep the game tied.
Atkinson said Tortorella didn't say a word during the second intermission, instead allowing the players to speak among each other.
"We pretty much said, 'This is our season right here,'" Atkinson said. "We can go and tiptoe our way and keep tiptoeing and expect someone to pick everyone up, but at the end of the day, we need to look in the mirror, dig deep, and elevate our play. Especially going forward because these games aren't going to get easier."
Said Blue Jackets captain Nick Foligno: "There has to be a whole other gear and just a whole other understanding of when you play a team like that, that doesn't probably have much to lose or is working that hard, then you just simplify more … and then your skill will take over at the certain points you need it to. We got a big win, but a lot of it was because of our goalie."
