Ovechkin scored his third goal of the season at 14:49 to give Washington a 2-0 lead.
Pirri made it 2-1 at 2:21 of the second period. He scored his second goal of the season on the rush from the high slot.
"They dug in there, found a way to keep it out of the net, and they got a little momentum, got it tied up, and then they came at us a little bit in the second period," Capitals coach Barry Trotz said. "Wasn't a lot of shots, wasn't a lot of chances, but all their chances went in."
Vesey's power-play goal at 11:22 of the second tied it 2-2. Nash corralled Derek Stepan's shot attempt, and then passed to Vesey, who had a wide-open net on Holtby's glove side for the tap-in.
Vesey scored his third goal of the season at 14:22 to give the Rangers a 3-2 lead. He took a pass from Mats Zuccarello, deked around Holtby's poke-check attempt, and scored with a backhand.
"The last couple games, I think I've had a lot of chances and our line's had a lot of chances, but they just haven't gone in, and it's a tough league to play in and an even tougher league to score in, so this gives me a lot of confidence going forward," Vesey said.
Nash scored an empty-net goal with 5.9 seconds remaining.