Homosexual is also not a sexual orientation, since the actual orientations are "gay", "lesbian"... and actually not even that, they are a whole spectrum and they can happen both at the same time (bisexual) or none at all (asexual). Yet we are ok with bundling them both as "homosexual" based on the sex being the same, but we are not ok with bundling the different young-age variants into "pedophilia" based on the common "young age" element? I don't see it.
There is no reason to treat "sexual attraction towards young age" as a parameter in any sense dissimilar from "sexual attraction towards men", or "sexual attraction towards ferris wheels". Most people have it in a very limited form, for very good evolutive reasons (choosing a strong and healthy mate, but not so young as to be one of their children or unable to procreate), while some have it expressed in a higher degree than normal, pretty much like some men have a higher than normal level of attraction to other men, and some women have a higher than normal level of attraction to other women. Then others have that parameter all the way down or turned the other way, and we call it gerontophilia. Seems to me like a parameter with a full spectrum totally fit for the name "sexual orientation".
And you know the anti-gay argument of "the pooper isn't designed for sex"? That's the "intrinsically involves harm" equivalent. Yet sexual attraction/orientation does not imply a sexual interaction. I think the interaction can still be illegal without having to find arguments to call stuff names that don't belong. Attraction is attraction, rape is rape, they are two clearly defined and different things.
Right now, that name-calling is quite a problem, since pedophiles are hiding on the Dark Web instead of openly seeking help and counsel to avoid raping children, which should be illegal as much as any other kind of rape. It's pretty much like banning Alcoholics Anonymous, criminalizing being an alcoholic, and then using alcoholism as a reason to send people to prison because drunk people have more car accidents. I call it highly WTF.