The Rangers trailed 1-0 after the first period but scored three goals to start the second.
Rust gave Pittsburgh a 1-0 lead with his first goal of the season at 10:29 of the first period before Chytil tied it 1-1 at 2:41 of the second. After Phillip Di Giuseppe forced Brian Dumoulin into a turnover in the corner, Chytil skated in alone on Jarry before going to his backhand to score in a second straight game.
New York scored two goals 24 seconds apart to take a 3-1 lead. Fox put the Rangers ahead 2-1 with a one-timer on a power play 5:18, and Kakko scored on a wrist shot off a rebound at 5:42.
"Going into the third with the lead, you want to hold that," Fox said. "But we're a young team and that just comes with experience and learning. You don't want to have too many learning experiences where you're losing games that you could've won, but I think early in the year it's something we can learn and not let it happen again."
Jared McCann cut it to 3-2 on a power play after he shot into the crease and Rangers defenseman Ryan Lindgren batted it in with his hand at 16:56. It was McCann's first goal since Jan. 14, 2020.
"We can't keep chasing the game," McCann said. "There's going to be teams out there that keep pressing and play a full 60 minutes. We can't be sitting back. We've got to keep pushing forward and it starts with the puck drop."
NOTES:Joseph and Rangers defenseman K'Andre Miller each got his first NHL point. Joseph, who had an assist on Blueger's goal, had three shots and three hits in 13:02. Miller, in his fourth NHL game, had an assist on Kakko's goal. … Panarin (34 goals, 66 assists) set a Rangers record for fewest games to 100 points (73). Mark Messier reached 100 in 74 games in 1991-92.