I hate clients like that. Why employ a designer if you think you know better yourself? Unfortunately tho', when it comes to design, opinions are like arseholes.
Anyway, you are right. If the guy isn't willing to take your professional advice on board how can you be expected to work with him?
If you can afford to Fuck him right off then by all means do so.