Neilsen moved in on Senators goaltender Robin Lehner and put a backhand in on the Islanders' second shot of the shootout, and last they would need.
Erik Condra, Bobby Ryan, Mika Zibanejad and Clarke MacArthur scored for the Senators (4-6-3), but it was Lehner making the biggest contribution with 53 saves, including one off Thomas Vanek on a breakaway in overtime.
Ottawa went into the intermission clinging to a 4-3 lead after a frantic finish to the second period that had two Senators in the penalty box for the final 55 seconds.
When the third period resumed, the Islanders still had 57 seconds of a 5-on-3 advantage to work with on fresh ice. But they were unable to score, producing just two shots and double-digit passes.
They did manage to tie the game at 3:14 of the third on Nielsen's goal.
There were a total of five goals in the second period as the Islanders scored twice in 28 seconds to tie the game before the Senators rallied to grab a 4-2 lead.
Tavares had an empty net to deposit the puck in after a blind pass from Vanek found him alone in front. At 9:35 of the period Martin tied the game as he tipped a point shot from former Senator Matt Carkner past Lehner.
The Senators responded with goals from Ryan and MacArthur at 12:34 and 14:47 respectively.
With the teams playing four on four following a pair of minors, Bouchard put a rebound past Lehner from a sharp angle to pull New York back to within a goal.
Zack Smith was the Ottawa player in the box at the time. Thirty seconds after the Bouchard goal Smith emerged from the penalty box but barely made it 10 feet before being penalized for interference just three seconds after his first infraction ended.
Seven seconds after that Jared Cowen was called for holding, setting up the Islanders' five-on-three that carried into the third period.
Ottawa took a 2-0 lead in the first on goals by Condra, with his first of the season, and Zibanejad. Condra left the game in the second period with a muscle strain in his leg and didn't return.
Notes: Derek Grant's first-period assist Friday was his first NHL point in 11 career games, all with the Senators ... Vanek played in the 600th game of his NHL career Friday night, and second with the New York Islanders, recording his first point with his new team. In 598 career games with the Buffalo Sabres he had 497 points ... Tavares extended his point streak to 11-games Friday with a second-period goal. It's the longest such streak in the NHL this season ... Matt Kassian and Joe Corvo were scratches for the Senators while Colin McDonald, Eric Boulton and Brian Straight didn't dress for the Islanders.