Coyotes prospect Chris Brown scored twice and goalie Jack Campbell, the Stars' 2010 first-round pick, stopped 12 of 14 shots in playing just over half the game. His replacement, Andy Iles, stopped four of shots he faced.
Emerson Etem also had a goal for Team USA.
"I don't know if I'd call it a crazy game, but it certainly was an interesting finish," U.S. coach Keith Allain told USA Hockey's website. "We'll continue to evaluate our players as we get closer to the tournament."
Team USA took a 1-0 lead less than two minutes into the first period. Brock Nelson, who the Islanders took in the first round of the 2010 Draft, won an offensive-zone faceoff cleanly to Brown, who fired a shot high past the blocker of Rensselaer goalie Allen York, a Columbus Blue Jackets prospect. York, incidentally, made 34 saves, including 17 of 18 in the opening period when the U.S. was consistently buzzing in the RPI end.
Rensselaer took a 2-1 lead early in the second period, but Team USA responded to tie it when Panthers 2009 draft pick Drew Shore slid a pass through the slot to Emerson Etem, taken by the Ducks with the 29th pick of the 2010 Draft, who fired a quick shot over York's glove.
The U.S. regained its one-goal lead with less than 10 seconds remaining in the second off a three-on-two rush when Brown connected for his second of the night off a shot high to the far post. RPI answered back late in the third to square the game again and send it into overtime. After a scoreless extra frame, RPI needed 12 shootout rounds to defeat Team USA, as Maple Leafs prospect Jerry D'Amigo failed to convert on the final shot.
The contest was the first of three pre-tournament games slated to prepare the U.S. They next play the Czech Republic on Tuesday at The Sports Centre at MCC in Rochester, N.Y.
The defending gold medalists will announce its 22-man roster on Wednesday morning.
Canada named defenseman Ryan Ellis as captain for its 2011 World Junior Championship team.
This will be Ellis' third WJC, making him the seventh player in Hockey Canada history to participate in three tournaments. With a gold in 2009 and a silver last year, he's hoping to become the fourth Canadian player to win three WJC medals, joining Jason Botterill, Jay Bouwmeester and Jason Spezza.
Ellis, taken by the Nashville Predators with the 11th pick of the 2009 Entry Draft, also is captain of the OHL's Windsor Spitfires.
Named as alternate captains were the three other players who won silver alongside Ellis that are returning for this year's WJC -- defensemen Jared Cowen and Calvin de Haan, and forward Brayden Schenn.