F1 figures very high on the list of reasons for it

That's where I disagree, that is how Ferrari got so big but their brand is no longer reliant on motorsport. How many people on this list do you think bought Ferraris because they race in F1? It's like saying Rolex is popular because they are large sponsors in tennis and F1 - no they're popular because they are the most iconic and famous watch brand today. It doesn't matter how either Rolex or Ferrari rose to popularity, my point is that in modern times it is no longer connected to F1 as much as F1 fans would like to think.

But it will diminish their unique brand of prestige in the long run.

I don't think so, what about Lambourghini? Aren't they doing just fine without F1? Wealthy people buy expensive cars that are fast, noisy, and sexy - not because that particular car or brand has strong roots in motorsport. I do not think Ferraris will decline in popularity or brand power significantly if they pull out of F1. There is simply no alternative for such a car.