The former Labour shadow chancellor said he questioned Mr Miliband's decision to sit down with the now-disgraced comedian as he tried to become prime minister.