Davidson, the Columbus Blue Jackets president of hockey operations, replaces Gregory as of April 1.
Gregory served as chairman for 17 years.
"John has been a dedicated member of the selection committee since 1999 and the board's appointment to the chairman's role recognizes his expertise and long-time contribution to the game as a player, media representative and hockey executive along with the fact that he is still very active in the game," Hall of Fame chairman of the board Pat Quinn said in a statement.
Clarke, senior vice-president and former general manager of the Philadelphia Flyers, Poile, GM of the Nashville Predators, and Robitaille, Los Angeles Kings president of business operations, succeed Quinn, Gregory and Serge Savard.
Gregory and Savard reached the end of their term while Quinn stepped down last year to become chairman of the board.
After being on the committee for 21 years, Gregory will stay on in a non-voting capacity. He and the other 18 members will meet June 23 and 24 to determine the Hockey Hall of Fame class of 2014.