Washington defenceman Nate Schmidt added a goal in the third for the Capitals, who beat Nashville at home on Saturday. Schmidt, a 22-year-old rookie, scored his first NHL goal in that game.
Grabovski's goal came on a penalty shot.
In his fourth NHL game, Philipp Grubauer made 30 saves for Washington (16-12-2) to earn his first career win. The Capitals have won four of five.
New York (15-15-1), beaten at home in overtime by New Jersey on Saturday, has lost three of five. Henrik Lundqvist made 32 saves, and Benoit Pouliot scored the Rangers' lone goal with 1:53 left to spoil Grubauer's shutout bid.