The Penguins played the final 49:45 without defenseman Kris Letang, who was being evaluated for a lower-body injury, coach Mike Sullivan said. Letang appeared to injure himself defending Adam Fox in the corner with 10:37 remaining in the first period.
The Penguins were already playing without defensemen Brian Dumoulin (lower body), Mike Matheson (upper body), Marcus Pettersson (upper body) and Juuso Riikola (upper body).
"It's unfortunate but we did a pretty good job back at 'D' tonight," said Joseph, who played 25:58. "You just try and keep the game simple and good things happen."
Jake Guentzel tied it 4-4 at 9:18 of the third period, scoring off his rebound after the puck hit the right post.
"I thought the third period was great, the way we played on our toes, creating chances off the rush, in the zone," Crosby said. "That's hard to play against. That's when we're at our best. We've got to build on that."
Artemi Panarin's power-play goal at 16:01 of the second period gave the Rangers a 4-3 lead. New York tied it 3-3 at 10:25 on Kreider's goal off a rebound, which came 43 seconds after Kapanen gave the Penguins a 3-2 lead on a breakaway.
The Rangers tied it 2-2 when Kevin Rooney scored on a 2-on-0 shorthanded breakaway off a pass from Miller at 7:20 of the second.
The Penguins were 0-for-4 on the power play and are 0-for-13 in the past three games.
"I think it's just execution [on the power play]," Crosby said. "We're well-prepared. We understand what we need to do. We have to find a way to execute and make sure we have the same level of compete that the penalty kill we're playing against has."