That's why you run separate devices. I'm not really into crypto anymore, I just follow the pace. But I have a lot of "shit" I only need access from home that I need a phone for still and is in some cases a pain to setup each time as you say. Everything from home automation, to bank software for where I have my savings and other authentication software, which I don't need access to all the time.
I I just pick some cheap(ish) phone for home use that will get security updates for another 3-5 years. Then I just run all that shit on that phone, plus I can use it as a backup for the stuff I have on my daily phone as well.
There really isn't much you need access to on a phone for daily use when you are not at home.