I mean, people are going to talk and I think you're just going to have to tolerate a bit of it. In terms of questions, like other posters have said, you just need to commit to your "party line". Inevitably someone is going to ask who dad is, personally I'd say "Dad's not in the picture anymore, just us 2!" in a lighthearted/happy way. Should shut down further questions, if they ask further I think it's totally appropriate to say "I don't really want to talk about it".

I think getting pissy every time someone asks about it will be a little exhausting, and to be honest even the most well-meaning, kind people could ask that question. It's of course not their business but that's how people can be sometimes. I would just gear yourself up for the few weeks of annoyances, and once everyone settles into that it is what it is, it should get better.

Ultimately, as long as you are confident in your decision, I see no reason for this to be a major deal. People talk, but they also move on and forget and stop caring. Don't worry about your work culture being more gossipy than others--what's the worst case scenario? They think you slept with someone you don't know that well (no idea if that's the scenario, just making one up), so? Shit happens, if they are a bunch of evangelical pearl grabbers then even easier to roll your eyes at how old school they are. Its 2018 and casual sex is a thing. Own it, girl.