Best thing to do if you want to sell him the domain, is explain that you have paid costs developing a website, promotional material, etc bullshit , ask him to make an offer that reflects the costs you have paid.
If he is the genuine trademark holder then I would think if it went to court he would win, but as you registered the name in good faith he might not. Sorry thats not much help is it.
So whats the domain then ?
It amazes me , the amount of people that 'forget' to renew their domains, especially if its a trade mark or critical to their site.