Jakub Voracek and Adam Boqvist each had two assists, and Joonas Korpisalo made 27 saves for the Blue Jackets (16-16-1), who had lost three in a row. Columbus lost 3-1 at New Jersey on Thursday to begin the home-and-home.
Jimmy Vesey had a goal and an assist, and Mackenzie Blackwood made 30 saves for the Devils (14-17-5), who had won four of their past five.
"We're playing good. It was a hard-fought game," Vesey said. "They probably didn't like the way they played the other night, so they came out a lot harder and then didn't give us as much."
Bjorkstrand gave Columbus a 1-0 lead on the power play at 2:01 of the first period. Patrik Laine sent a pass a cross-ice pass from the left circle to Voracek, who quickly got the puck to Bjorkstrand for a one-timer in the slot.
"The way we are going right now, it was a huge win for us," Voracek said. "We made it a lot tougher than we had to, but we found a way. We need to get a win streak going. It's a crucial time of the year. We have to get some wins."
Boone Jenner scored off the skate of New Jersey center Jack Hughes to make it 2-0 at 4:03. He was behind the net when he attempted a centering pass to Zach Werenski.
Nico Hischier cut it to 2-1 at 14:41 with a wrist shot from the high slot.
Vesey tied it 2-2 with 20 seconds left in the period. The Devils were on the forecheck when Dawson Mercer fed Vesey trailing in the right circle for his sixth goal.
"They got a quick one on us and just a lucky goal, the second goal, and sometimes that happens, but again, we fought back," Devils coach Lindy Ruff said. "We in fact had them even by the end of the period and we played a real strong 40 after that."