Harry and Meghan were reportedly 'surprised' to have not been given more notice