The higher up you go in ABRSM, the longer the pieces, so learning by rote becomes more difficult. Reading music is a combination of memory and recognition, just like reading a language. There are only 7 letter names to learn A-G. Each one can be raised or lowered a half step - the shortest distance between notes on a piano.

Rhythm is another story, but it is generally based on simple mathmatic divisions - whole, half, quarter etc.

Granted, there are infinite combinations of the above, but the sooner you start to learn music notation, the easier it will be to learn new pieces.

Recently, someone posted on here that the were learning to read, but having trouble playing Chopin and Debussy. That is not a good first choice for developing reading.

If you really want to progress, you will benefit greatly from investing time in learning how to read. But start with the most basic stuff you can find. It won't be fun at first, it's a bit of a grind tbh. But regular, careful practice will get you there.