No real issue so far, I have hundreds if not thousands of .chd files. I convert every .iso to .chd if the format is supported by the core/emulator. CHD files do not take longer to load I think, in fact smaller files are faster read from disk, but internally would need some time to decompress in memory. All in all, the differences are negligible to me. I think I never had an issue where this format was the cause.

Sometimes .chd files are compressed as good or better than .7z files, sometimes less. So saving space compared to .iso files is huge (hopefully ppsspp will support it someday too). It is my preferred CD format.

I do not use aethersx2, only speaking from experience on RetroArch. I convert all files with chdman commandline tool from MAME directly, without altering an settings. So far every core in RetroArch that supports .chd has loaded all files I tried without issue or without taking much longer. But honestly I don't know how much longer it takes or can take to load .chd vs .iso, because I never made a comparison.

Note to myself: Actually a good idea to compare: SSD, oldschool HDD and micro SD card loading speed for .chd files sounds interesting.