Honestly, the best thing you can do, is try bring it up with him. If any relationship is going to work, you need to be able to communicate your concerns with each other and work together to solve them or come up with a compromise. You should ONLY be getting physical if you want to, and not just for his sake. If you don't communicate it with him he will probably eventually want to go for that home run.

Now if we run with the baseball analogy here, him running to home base right now would get him out. The ball is waiting there at home base and the catcher is there to tag him out. The ball in this case is you, and him getting out, is potentially you stopping when y'all are just about to have sex because you will be uncomfortable. There is nothing wrong with that, but might come as a surprise to him since you have seemed to be going along with the rest. Telling him now is like telling him to stop at 2nd or 3rd base to keeping the team in the game and not get out. He might not have the home run, but waiting to get to home base safely will secure the run, and go on to win the game.

Communication and honesty is everything in a relationship, you need to be able to have these conversations if you want to be with anyone long term. Through these conversations, you'll get so much information about the person and will give you a good idea if they are worth your time. Sometime you'll find out that maybe you and someone else have different morals or goals in life and might not be fit for each other long term, which is okay, it's better to know now than later on.

Finally I would like to mention that I too over-think everything. Bringing this up will be scary, but I promise you, it will be 100% worth it. If he doesn't take it well, then that gives you all the information you need to leave him. If sex is the most important thing to him (which probably won't be the case since he seems to be a nerd like us) and he isn't willing help you build up to it and wait like a gentleman, then he probably isn't worth your time or effort.

Wish you all the best, stay safe :)