The Sports Direct boss has slammed the Prime Minister in a letter, warning of further damage to the British high street.