Nate Schmidt, Jon Merrill and Cody Eakin scored for Vegas (31-22-4), which lost for the first time in three games and for the fifth time in its past seven. Marc-Andre Fleury made 29 saves.
The Golden Knights have lost three in a row at home after starting the season 16-4-3.
"You hear those fans out there, they deserve better than that," Merrill said. "We've had a couple tough outings the last few in this building, and we've got to do a better job to get some wins in here."
Eakin scored at 9:21 of the third to give Vegas a 3-2 lead, but after Atkinson tied it with his 30th goal, Golden Knights forward Paul Stastny was called for slashing Panarin at 17:04, and Atkinson put the Blue Jackets ahead with a goal against the fifth-ranked penalty kill in the NHL (83.4 percent).
"It's been something that I think we've been good at," Schmidt said of closing games. "It feels like we scored, and then we stopped playing."