"Baby, I'm feeling kind of insecure about my intelligence."

"Hey who cares? At least you make a really good pie!"

"..Ok, thanks, I know that though. What I'm really worried about is that I'm not smart enough."

"Pssssh don't sweat it, you have a great sense of humor!"

"....."

He knows you love him. He knows you find his personality attractive. He's not worried about that, so you telling him as if it's some sort of consolation prize is pointless and is only going to make it worse. He's not upset about any of that, he's clearly feeling insecure about his physical appearance specifically. Don't try to make this about yourself, you lied, you said something without thinking, now it's on you to fix it instead of saying "ur hurting meeee!" and expecting him to solve it for you. Find a way to articulate properly to him how attractive you find him PHYSICALLY, and why you might not have been that physically attracted to this ex, as you've said here.