Yannick Weber tied it 1-1 on a shot from the right circle that beat Kinkaid glove side at 14:40.
Hischier gave the Devils a 2-1 lead after redirecting a pass from Hall on the power play 19:01. New Jersey has scored a power-play goal in each of its first seven games of the season for the first time in its history (9-for-31).
"It was a step in the right direction from what we had talked about prior to our last few games," Devils coach John Hynes said. "It was more the identity we want to play with, more of the compete we want to play with. That's what we talked to our guys about, getting back to our game."
Arvidsson tied it 2-2 at 13:06 of the second period, but Hischier scored 40 seconds later on a redirection in the slot to give the Devils a 3-2 lead. It was the third career two-goal game for Hischier and first since Dec. 27, 2017.
"You could see [Hischier] had a little more jump in his step," Hynes said. "He was really tenacious. He was winning a lot of his puck battles and he's finally getting rewarded for going to the net. It was nice to see him play the way he did."