It's literally not possible for telemetry to not do something that impacts performance.

That's an exaggeration. It entirely depends on process priority, I/O blocking and scheduling, buffer sizes, etc. Also, it greatly depends on what data is being logged; logging application start events won't affect performance because those operations are often heavily I/O dependent (they're blocked anyways waiting for data from disk), for example, but if it were logging mouse movements (as an example; it's not logging this), it obviously would affect performance given the amount of information that would have to be logged.

It is very possible that the logging isn't affecting performance; you're assuming that your system is 100% utilized at all times and that the telemetry service is prioritized the same as every other application you're attempting to run.

Now, in all fairness, it's likely it's affecting performance (at a negligible amount) like you say, but you can't make definitive statements that it 100% most certainly exists without information to back it up.