One well-heeled borough has seen almost £500,000 wiped off the value of the average home in just four years.