The Blues had a chance, according to coach Ken Hitchcock to clear the puck out of their zone three times, but Schwartz's turnover proved costly.
"The fourth goal was a killer; it was a killer," Hitchcock said. "Had puck support there, left it, didn't clear it, had it on our stick three times. Forwards left the winger, turn it over and that's the game."
Mike Hoffman scored twice, Jean-Gabriel Pageau had a goal and an assist and Bobby Ryan scored for the Senators (23-15-4), who opened a three-game road trip. Mike Condon made 19 saves.
"We gave them less than 25 shots in their building, so I think anytime you do that as a team, you're doing something good," Senators coach Guy Boucher said. "I think we've asked our players to improve since Christmas at driving the net and paying the price around the net, and we certainly did that with the goals that we had."
Steen and Kevin Shattenkirk each scored a goal and assist, and Paul Stastny and Patrik Berglund scored for St. Louis (23-17-5). Hutton made 18 saves.
"We have the momentum and it's just a tough game because I think when we look back on it and we wake up tomorrow, we're going to feel like we had it," Shattenkirk said. "We played a pretty good game, and again just some mistakes that bite us."