I'd have to know you both better, but since we don't...
Last thing I would do in your situation would be a breakup, although I understand why that might sound tempting to you. Talk to your man about this. Tell him what you just told us. My personal experience is that two people can see the situation vastly differently, while having good points backing their view. So all in all, because this is not a situation where one clearly did something wrong, but there's a lack of passion and you've had problems you mentioned, do not give up easily. If you give up too soon, you will probably regret for not turning over all stones, and you'll leave a man wondering what the hell went wrong.