More than half of small business owners worry customers are going to spend less than in the past - a bigger worry than recent tax and rates hikes.