I wonder if abstracting might actually be harder. Blind people who've had surgery to restore sight often initially have trouble visually recognizing shapes (e.g. seeing a tennis ball, they can't identify it's a ball, they need to feel it to conclude that). This is called Molyneux's problem.
If that is indeed correct, though, I imagine definitions like manifolds and stuff, which are "visually" motivated (e.g. "locally flat"), might be hard to understand. A sighted person knows that something diffeomorphic to R2 means that up close it "looks like" a piece of paper, but to a blind person a Euclidean space might not seem geometrically natural at all.