The Prime Minister's own colleagues can't understand why he made such a massive mistake