The Prime Minister has been told his plans for a holiday tax will force "hard-working families" to "prop up local government finances".