Markus Hannikainen and Pierre-Luc Dubois scored for Columbus (15-9-2). Sergei Bobrovsky made 26 saves.
"I don't [care] about the home team and how they feel," Blue Jackets coach John Tortorella said. "I'm a little concerned about the mistakes we made."
Hannikainen gave the Blue Jackets a 1-0 lead at 6:46 of the second period when he scored on a rebound in front after Greiss made an initial save on Seth Jones' slap shot from the right point.
Dubois made it 2-0 at 8:33 when Cam Atkinson stripped Tom Kuhnhackl behind the Islanders net and fed Dubois at the left face-off circle, where he sent a wrist shot through traffic past Greiss for his 13th goal of the season.
"Even though we were up 2-0, they were playing really good hockey," Dubois said. "That's just how it goes sometimes. You have a couple of good shifts and you're up 2-0, but it came down to the third period."
Lee made it 2-1 at 12:00 of the second period when he redirected Eberle's shot over the goal line. The goal was initially waved off but was awarded after video review showed Lee did not bat the puck into the net with his hand.
Beauvillier tied it 2-2 at 15:24 of the second when he skated down the right wing on a 2-on-1 with Mathew Barzal and put his own rebound past Bobrovsky for his seventh goal of the season.
"That was a lot of fun," Beauvillier said. "It felt like home. A lot of history in this building, and everyone is excited.
"We answered back after they scored their second goal and it gave us a little bit of juice. We just played north and we played behind their D and we found the back of the net. It was a big team win for us tonight."
Cizikas gave the Islanders their first lead of the game during a 4-on-4 situation when Ryan Pulock fed him from behind the net for a one-timer to make it 3-2.
"I had a lot of emotions running through my body," Cizikas said. "I think I was more tired from screaming than I was for the actual shift."
Greiss preserved the lead with 4:21 remaining when he made a save against Dubois and then stretched across his crease to deny a backhand attempt from Atkinson on the doorstep.