Panarin beat goalie Philipp Grubauer off the rush after Pierre-Luc Dubois won a puck battle following a Washington turnover in the Columbus zone.
[WATCH: All Blue Jackets vs. Capitals highlights | Complete Capitals vs. Blue Jackets series coverage]
"Big goal for our team," Panarin said. "I'm happy because it was a hard game and a big win."
Blue Jackets defenseman Seth Jones tied the game 3-3 with a power-play goal at 15:34 of the third period. It was the third power play of the period for Columbus.
Alexander Wennberg and Thomas Vanek scored, and Sergei Bobrovsky made 27 saves for the Blue Jackets.
Evgeny Kuznetsov had two goals, and Devante Smith-Pelly scored for the Capitals.
Video: CBJ@WSH, Gm1: Kuznetsov scores twice in 29 seconds
Game 2 of the best-of-7 series is here Sunday (7:30 p.m. ET; NBCSN, SN360, TVAS2, NBCSWA, FS-O).
Panarin, whose 82 points in the regular season set a Columbus single-season record, had a goal and two assists. He was acquired in a trade from the Chicago Blackhawks on June 23, 2017.
"We traded away a really good player in [Brandon Saad] to get him," Columbus coach John Tortorella said. "A different type of player, a guy that makes something out of nothing. Makes a great play on the tying goal and scores just a goal that a lot of people can't score in this league. That's sometimes the difference between winning and losing."
Kuznetsov scored twice during a five-minute power play after Josh Anderson received a major penalty for boarding and a game misconduct at 17:23 of the first period for his hit on Capitals defenseman Michal Kempny, who left the game with an upper-body injury.
Kuznetsov gave Washington a 1-0 lead at 17:52. Tortorella challenged for goaltender interference by T.J. Oshie, but the goal was upheld after video review.
Kuznetsov scored again 29 seconds later, beating Bobrovsky from the left face-off circle at 18:21 to make it 2-0.
"You've got a lead there, you've got to be smart," Grubauer said. "It doesn't matter what it is. Whatever you do, protect the middle of the ice. Don't make it hard on yourself, basically. We kind of shot ourselves in the foot a little bit again."
Wennberg cut the Capitals lead to 2-1 when he scored at 4:48 of the second period off a cross-ice pass from Boone Jenner.
Video: CBJ@WSH, Gm1: Wennberg chips home Jenner's dish
Vanek scored on the power play to tie it 2-2 at 1:31 of the third period. The goal came 13 seconds after Washington forward Tom Wilson was called for charging Wennberg, who left the game with an upper-body injury.
"I'm obviously not trying to take a penalty," Wilson said. "That cost us the game. That's a critical moment. I've got to be better and maybe pass up on that hit. We've got the lead there, so maybe a big hit is not needed."
Smith-Pelly scored off a pass from Jakub Vrana to give Washington a 3-2 lead at 5:12.
Video: CBJ@WSH, Gm1: Smith-Pelly pots Vrana's dish for lead
Grubauer, in his second career Stanley Cup Playoff start, made 23 saves.
Bobrovsky made a tough save on an Alex Chiasson shot from the top of the crease at 15:46 of the second period that would have put Washington back ahead by two goals. Instead, the Blue Jackets stayed within one until Vanek scored early in the third.
"We talked about it: Don't get too high, don't get too low," Vanek said. "We play our game and whatever happens, we'll be happy with it. And I thought we did that, so it's nice. Nice to get rewarded."
Goal of the game
Panarin's goal at 6:02 of overtime.
Video: CBJ@WSH, Gm1: Panarin seals it in OT with superb goal
Save of the game
Bobrovsky stopping Chiasson at 15:46 of the second period.
Video: CBJ@WSH, Gm1: Bobrovsky makes save as Foligno clears
Highlight of the game
Jones' goal at 15:34 of the third period.
Video: CBJ@WSH, Gm1: Jones buries PPG to tie the game late
They said it
"We have been this situation. We have been playing for our lives for a while now. Obviously, there is an increased intensity there, but at the start of overtime, we weren't overwhelmed, but we were trying to feel our way and make some plays. We found a way." -- Blue Jackets forward Mark Letestu
"Right now, I'm going to re-evaluate that, your question. If we go back, right now, Philipp's body of work has been good. I thought he was fine. We'll sit down and re-evaluate all the goals, evaluate our team and where we're at, and go from there." -- Capitals coach Barry Trotz when asked if goalie Philipp Grubauer would start Game 2
Need to know
Blue Jackets forward Nick Foligno was hit in the visor and face by a Jakub Jerabek slap shot at 1:40 of the third period. The captain returned at 5:06 with a cut under his left eye. … It was the first Game 1 win in four Stanley Cup Playoff series for Columbus. … The Blue Jackets are 3-1 in overtime in the postseason. The Capitals are 28-35.
Game 2 of Eastern Conference First Round at Capital One Arena on Sunday (7:30 p.m. ET; NBCSN, SN360, TVAS2, NBCSWA, FS-O)
Video: Panarin lifts Blue Jackets to Game 1 overtime win