Ottawa opened a two-goal advantage at 12:07 of the first on a goal by Johnny Oduya before Grabner's first of the afternoon got the Rangers back within one.
The Senators, though, regained the two-goal lead on Matt Duchene's 15th of the season. But like in the first, New York answered with a power play goal by Mika Zibanejad, his team-leading ninth of the year with the man advantage.
Former Ranger Derick Brassard again put the home squad ahead by two with 1:07 remaining in the second, and Thomas Chabot made it 5-2 7:27 into the third, thus ending Henrik Lundqvist's afternoon. Brandon Halverson entered in relief for his NHL debut.
Grabner's second of the day made it 5-3 and give the Blueshirts some life with 4:49 left in the game, but Magnus Paajarvi's first goal as a Senator iced the game in the host's advantage.
New York returns home to take on the Flyers Sunday at noon.