none of these details matter. it can be wrong or fine to look at the bestiary. your purpose is a fine purpose, looking to cheat is wrong. it's not about rules that govern your actions in a crazy, micro-managing way. it's about this DM is an asshole and shouldn't be playing.

calling them out could be a waste of time or a big help. it's never "wrong," but it could be a mistake, if the guy handles it badly. you definitely show the math to the other player. you definitely find a better DM and have nothing to do with the DM you have now. he's just terrible.

in other words, sticking to your format:

• changing dice rolls is cheating, yes.
• it's absolutely OK to call out your DM on absolutely anything.
• if my players accused me of cheating, I'd find out why they thought that and then consider changing my style. but I don't cheat, so my answer is meaningless. the real question is, how does an abusive person handle being called out as an abuser? most of them respond with abuse. that's why you should have nothing to do with them under any circumstances.
• using your knowledge of probability is not metagaming. and if anybody tried to micro-manage me in this extreme way, telling me what knowledge I could use or not use, I would absolutely not have anything to do with them under any circumstances. I'm guessing you must be under 20 to even be asking questions that constrain your behavior with such totalitarian specificity. that's only stuff you could tolerate from a parent.
• of course it is not wrong for a person to look at a book!
• railroading is garbage.

TLDR: your DM is a jerk. tell your friend that you can prove for a fact that he got screwed over. then bail, and bail hard. do not play any game with that guy ever again.