Bailey, who scored his eighth of the season at 17:09 of the first period, helped New York restore its one-goal lead with an assist on Casey Cizikas' sixth at 9:31 of the third.
Milan Michalek scored his fourth goal in four games to draw Ottawa even at 1-1 on a power play at 6:41 of the third period.
"You could see it on that tying goal, they had like five guys in the crease banging away. They're obviously a desperate team," said Bailey, who has 66 NHL goals. "We knew it was going to be a tough game and they came at us pretty hard, but we had great goaltending and it was a good win."
Craig Anderson made 25 saves for the Senators, whose three-game winning streak ended. Ottawa was on a 4-0-1 run that revived its playoff hopes.
"We played a really solid game 5-on-5 and the difference was special teams," Anderson said. "That's the way this game is, and some nights you're going to generate lots of opportunities on the power play and some nights you're not. That was the difference-maker tonight."
The Senators have 78 points, five out of an Eastern Conference playoff spot with six games remaining.
"We still have a chance," Michalek said. "We still believe, and we're going to have to work hard. We've got some big games coming up, so we'll see; you never know what happens."
Ottawa next hosts the Montreal Canadiens to close a five-game homestand.
"We've got another game to play coming up on Friday against a really good opponent, and we need to come here [Thursday] and get ready to play it," MacLean said.
The Islanders, who extended their winning streak to three, have 11 of a possible 12 points during a 5-0-1 run that includes a 4-2 win against the Florida Panthers on Tuesday.
"It's always fun winning," Nilsson said. "We have a very young team right now, and obviously it's very good for the confidence, but the reason that we keep on winning right now is because we are working very hard. We're working harder than our opponent, and everybody's sticking up for each other, and that's why we're able to walk out with two points and win those one-goal games."
Michalek found a loose puck during a goalmouth scramble and shot into the right side of the net to spoil Nilsson's bid for his first shutout in more than two years.
"Ottawa did a great job of crashing the net, and that's how a lot of the goals are scored nowadays," said Nilsson, whose only NHL shutout was March 4, 2012, a 1-0 win against the New Jersey Devils.
Cizikas restored the Islanders' lead when he drove the net to put home Bailey's pass into the crease from behind the goal line. Senators defenseman Eric Gryba was penalized for slashing 1:17 earlier.
"It hurts, obviously," MacLean said. "It takes any momentum away that we got from the goal by taking a penalty, and again, we put ourselves in a bad spot by taking another penalty. We took four tonight and they got two goals on it, so it's still something that we continue to address and we have to find a better way to get better results."
New York's Cal Clutterbuck failed to convert two shorthanded breakaways. He put a backhand over the net in the dying seconds of the second period after he was stopped by Anderson on a chance early in the first.
"I should have scored on both of them," Clutterbuck said. "He made a good save on the first one. The second one, I had him beat and the puck rolled up on me and I just put it over the net. The second one was a little more rushed. I had a lot of time on the first one, but that's the way it goes."