Shane Pinto scored, and Cam Talbot made 33 saves for the Senators (8-13-1), who have lost seven of their past eight home games (1-7-0).
"Give the Rangers credit, they're a good hockey team too," Talbot said. "They were just able to find ways with some of those point shots to squeeze them through on redirects. Other than that, I thought it was a pretty even game."
The second game of the home-and-home series will be played at Madison Square Garden in New York on Friday.
Vesey gave the Rangers a 1-0 lead at 10:34 of the first period when he slid in Lindgren's rebound at the edge of the crease.
"You want your players there," Gallant said. "You talk about it, you watch games every night and that's where 90 percent of the goals are scored from, the blue paint area. We had guys going there, we had a couple deflections and a couple of rebounds, so hopefully that will carry over."
Goodrow tipped a Lindgren point shot to make it 2-0 at 10:52 of the second period.
"It's something that we've been stressing probably for the last month, just getting to the net," Goodrow said. "Obviously we've been struggling to score some goals, and when that's happening it's usually because we're trying to get too pretty, too cute. We're making the extra pass without getting pucks to the net. All three goals are a result of getting pucks to the net."
Pinto scored blocker side on Halak with a wrist shot on the rush to cut it to 2-1 at 13:47.
"I just didn't think we were hard enough in our zone," Pinto said. "Just a couple soft goals in front of [Talbot] and we kind of left him out to dry there. We've got to turn the page."
Kreider tipped a Lindgren wrist shot from the top of the left face-off circle to increase the lead to 3-1 at 12:59 of the third period. It was his fourth goal in five games.
"Offensively, we had chances to shoot where we didn't shoot," Ottawa coach D.J Smith said. "Maybe we were too cute. In a 2-1 game, you're passing up some Grade A looks, and now it's 3-1 and you're squeezing it."
