How big is your bed?
And on a blanket / fabric that moves / touches you as well? and probably the cat also warming you up on top of the annoying quilt :)
Does it also happens if you would sleep in one big bed with low "partner disturbance" where you have your own quilt?
I hate certain fabrics with passion and how the texture triggers with my nerves - like polyester causes sweating up as it doesn't breathe is one strong annoyances in my life as well as foam based mattresses/sheets/quilt thus I got nice latex core/wool futon and wool quilt - merino wool and cashmere <3 the temperature/moisture regulation.
I am quite temperature / humidity sensitive and I also like weighted blankets when it is evenly applied pressure points compared to say very small part of fabric touching me - I hate clothes tags! :O