Boone Jenner also scored, and Jet Greaves made 26 saves for the Blue Jackets (32-21-8), who have won three straight games and are 14-2-1 in their past 17.
Niko Mikkola and Sam Bennett scored, Aaron Ekblad had two assists, and Daniil Tarasov made 23 saves for the Panthers (30-29-3), who have lost four in a row.
Provorov made it 1-0 at 5:08 of the first period with a power-play goal. He scored with a wrist shot from the point that deflected off the stick of Panthers defenseman Dmitry Kulikov.
Olivier increased the lead to 2-0 at 10:50 of the second period with a redirection of an Egor Zamula shot from above the left circle.
Jenner made it 3-0 at 1:41 of the third period. Kent Johnson fed him out of the right corner for a wrist shot from the right hash marks.
Mikkola cut it to 3-1 at 9:08 for the defenseman’s first goal in 26 games. He tipped a shot from the left point by Ekblad.
Bennett made it 3-2 at 14:44 with a power-play goal on a shot from the high slot.
Gustav Forsling then had what would have been the tying goal overturned at 17:05 after the Blue Jackets challenged for goaltender interference.
Olivier scored into an empty net at 18:18 for the 4-2 final.
Blue Jackets defenseman Zach Werenski missed a third straight game due to illness, and Columbus forward Mason Marchment exited at 15:52 of the second after taking a spill.
