I played Byzantium all the time in CK2 (I estimate my 80% of my games were as the Byzantines) and don't ever really remember having issues with the Umayyad. They only bothered me if I bothered them first. But, to be fair, I bothered them most games because I was trying (and failing) to reform Rome. I probably spent about 300 games hours trying to form Rome across multiple starts and never could. It's the one thing I wanted to do in CK2 but never even came close. I don't think I ever even got all of the Middle East provinces required for reformation.

Anyway, so far in CK3 I've stuck to Western Europe and Scandinavia only, but the Byzantines are there, calling to me. I know I'll cave eventually.