Brad Richards, Rick Nash and Mats Zuccarello also scored for the Rangers (26-21-3), who have now won five of their last six. Cam Talbot was solid in goal stopping 27 shots for his 10th win of the season.
The loss put an end to the Senators' eight-game (6-0-2) point streak.
For whatever reason the Senators struggle with afternoon games. Ottawa is 1-3-2 on home ice when having to play midday. This one was no different.
Giveaways proved to be extremely costly for the Senators as the Rangers scored twice on two terrible ones.
Zuccarello took advantage of a giveaway by Chris Phillips near centre ice and broke in alone on Anderson. He caught Anderson moving to make it 4-1 New York late in the third.
While the Senators were managing to hold on during the game despite their sluggish play a late goal in the second proved to be the back breaker.
With 11.5 seconds remaining Erik Karlsson went to pass the puck back behind his own net and resulted in putting it directly on Stepan's stick. Stepan beat Anderson shortside to give the Rangers a 3-1 lead and disappoint the 19,978 on hand at Canadian Tire Centre.
Earlier in the period Nash picked up his 12th of the season as he broke in alone on Anderson and roofed the puck high.
Ottawa had gotten off to a decent start with Methot scoring his fourth, tying a career high, to take a 1-0 lead, but midway through the period the Rangers tied it with a power-play goal.
With a two-man advantage Stepan made a cross-crease pass to Richards who went high on a kneeling Anderson.
Saturday's game marked the first of three meetings between the two teams this season. Ottawa had claimed five straight overall against New York in the regular-season series, including the last three in Ottawa.
Notes: D Patrick Wiercioch, D Joe Corvo and LW Matt Kassian were a healthy scratch for the Senators. RW Chris Neil returned to the lineup after missing the last six games with a lower-body injury. ... D Justin Falk was a healthy scratch for the Rangers.