That's actually not true. There are way more reasons, like he could've had a panic attack and freaked out (happens to me a lot), he could've been inexperienced, could've had something happen, could've been in the middle of a very heated competition to fuck as many women as possible, etc. The most logical thing you can take away if he won't tell you what happened, is that he's selfish and inconsiderate, bad in bed, and that he's not worth the time anyway.

You might be able to give it a week and text him something like "hey, it's fine we're not a match but would you feel up to letting me know what happened in case it was something on my end I can avoid in the future"? He may come clean without having to feel like you're mad or expecting something.