Treasurer Jim Chalmers claims Barnaby Joyce is “all at sea” on questions on migration as farmers raise fears about worker shortages.