A snap poll of builders at a housing summit found that 61% were not confident in whether the government’s target of 300,000 homes by 2030 can be achieved.