"It was definitely not our best game," van Riemsdyk said. "We just didn't execute at a good enough level early on. I think we weren't as dialed in as we probably should've been. We were able to find a way to get the two points, which is all that counts, but certainly not one of our better games."
Sean Couturier scored the only goal of the shootout in the top of the third round, and Brian Elliott made the save on Vladislav Namestnikov to end the game.
[WATCH: All Flyers vs. Senators highlights]
"It's a huge win. To get that extra point is huge," Couturier said. "I don't think we played that well. We grinded it out. We found a way to come out with the two points, so that's huge."
Jakub Voracek had a goal and an assist for the Flyers (20-11-5), who have scored 14 goals during a three-game winning streak. Elliott made 30 saves.
"There was a certain point where we were kind of giving them everything that they got," Elliott said. "A couple breakdowns, a couple miscommunications. Thankfully, we sorted that out a little bit and we went from there. It was a back-and-forth game, and luckily we came out on top."
Video: PHI@OTT: van Riemsdyk banks home puck off defender
Tyler Ennis had two goals and an assist for the Senators (15-18-4), who have lost three of four (1-1-2).
"I thought we played real hard," Ottawa coach D.J. Smith said. "For the guys we had on the back end, some of the young guys that haven't had a lot of experience in the League, against some elite players, I thought some guys played really hard. I thought, being down 2-0, we just kept playing. And then being down in the third, we found a way."
Marcus Hogberg made 34 saves, including seven in overtime.
"He played great," Ennis said of Hogberg. "All year, whoever's been in the pipes has been great for us, and he's no different. Our goalies have been awesome and are giving us a chance to win every time."
Kevin Hayes took a stretch pass from Voracek, beat defenseman Erik Brannstrom wide, and scored on the power play to give the Flyers a 1-0 lead at 3:16 of the first.
Video: PHI@OTT: Hayes slips puck past Hogberg for PPG
Van Riemsdyk scored to make it 2-0 at 1:03 of the second when his centering pass went in off defenseman Cody Goloubef's skate.
"One thing we've talked about, and that we've stressed, is throwing more pucks at the net," Philadelphia coach Alain Vigneault said. "I think [van Riemsdyk's] goal was exactly that. Threw it in the blue paint, and it found a way into the back of the net."
Ennis scored on the rebound of an Andreas Englund shot to cut it to 2-1 at 6:00.
Mark Borowiecki scored 25 seconds later to tie the game 2-2 when his point shot through a screen beat Elliott at 6:25.
Anthony Duclair scored with a wrist shot on the power play to give Ottawa a 3-2 lead at 18:21 and set an NHL career-high with his 21st goal of the season.
Video: PHI@OTT: Duclair rips home PPG from the circle
Voracek scored on the rebound of a Tyler Pitlick shot with 48 seconds remaining in the period to tie it 3-3.
Van Riemsdyk scored his second of the night when he beat Hogberg five-hole on a backhand deke on a breakaway to give the Flyers a 4-3 lead at 1:10 of the third.
Ennis scored his second into an open net off the rebound of a Duclair shot to tie it 4-4 at 10:17.
"I thought we battled back hard a couple times," said Borowiecki, who had an assist. "I thought we showed a lot of heart and character. We know we're under-manned a little bit, but for the most part, we played our game. We played hard."
Video: PHI@OTT: Ennis picks up loose puck tucks it into net
The Senators were called for too many men on the ice for the second time in the game with 2:50 remaining but killed off the penalty.
"I'll take the heat for that," Smith said of Ottawa's 10th too many men penalty of the season. "For sure, it's an area we have to clean up, but in saying that, the best part of our game tonight was probably our penalty kill. I think we had more chances on our penalty kill than we had 5-on-5."
Ottawa was 4-for-5 on the penalty kill and 1-for-4 on the power play.
They said it
"I would say, in the last month, he's been skating much better. That skating is permitting him to get real good opportunities. That fourth goal there was him using his speed, getting behind the [defensemen], and obviously he's got good hands, and he was able to score a big goal for us." -- Flyers coach Alain Vigneault on forward James van Riemsdyk
"He shoots the puck. He doesn't play around, he doesn't get it and make a bunch of moves; he just puts everything on net. And when you shoot pucks, pucks sometimes go in. He doesn't give you an inch, he plays hard, he gave us a great game tonight." -- Senators coach D.J. Smith on defenseman Mark Borowiecki
Video: PHI@OTT: Couturier beats Hogberg in shootout
Need to know
Flyers forward Claude Giroux had a game-high five shots on goal. ... Philadelphia defenseman Matt Niskanen did not play because of the flu. ... Duclair has 15 points (11 goals, four assists) in the past 10 games. ... Englund's assist on Ennis' first goal was the rookie defenseman's first NHL point in his 13th game.
Flyers: Host the New York Rangers on Monday (7 p.m. ET; NBCSN, NBCSP, MSG 2, NHL.TV)
Senators: Host the Buffalo Sabres on Monday (7:30 p.m. ET; RDS2, TSN5, MSG-B, NHL.TV)
Video: Flyers top Senators in shootout