Columbus called a timeout after Perreault's goal, but the Rangers kept the pressure on and made it 4-3 on Borgen's goal at 12:52 off a netfront scramble with Shesterkin pulled for an extra skater because of a delayed penalty on Columbus.
"I thought we kind of fell back on our heels a little bit and got back into some bad habits," Blue Jackets forward Mathieu Olivier said.
Perreault then tied it 4-4 at 15:14, scoring off a pass from Borgen.
"We could have easily mailed it in and gone home and felt sorry for ourselves," Rangers defenseman Adam Fox said. "We know the position we're in. We know where we're at in the standings and everything. Definitely proud of the group's fight that period to battle back. Obviously, we'd love to win, but the resolve was definitely there and that's big for this group."
Adam Fantilli gave Columbus a 1-0 lead at 5:50 of the first period, scoring from below the right hash marks on a redirection of Marchenko's pass from the top of the left face-off circle.
Marchenko extended the lead to 2-0 with a power-play goal at 15:32.
The Rangers thought they made it 2-1 at 3:04 of the second period, when J.T. Miller appeared to score a power-play goal with a one-timer from the right circle.
The Blue Jackets challenged for goalie interference and video review determined Alexis Lafreniere made contact with Merzlikins' head prior to Miller scoring, so the goal came off the board.
"I'm relying on Aron (Augustitus) and Cam (Briere), they're video coaches, they're watching it, and if they say challenge it, I'm challenging it no matter what," Bowness said. "I trust them 100 percent. … That was huge."
It became even bigger 23 seconds later, when Sean Monahan scored a short-handed goal at 3:27 to give Columbus a 3-0 lead.
"That turned things in our favor at that moment," Bowness said.