They absolutely can and do find out. They will interview your friends, family, past employers, and they will ask those people to provide them more names of people who they can interview. It's not cool to ask people to lie on your behalf as it exposes them to liability, and they may name people who can't or won't lie on your behalf.

And many (but not all) Top Secret clearances require a full scope/lifestyle polygraph test, where they will put you on a lie detector and ask you all the background check questions including regarding past use. My understanding is that infrequent recreational use of soft drugs (or use in a legal state) is forgivable outside of the past 12 months for those clearances, but if they detect that you're lying they can fail you. Not to mention they will interview quite a lot of people, and if any of those people know that you've smoked weed they're obligated by oath to disclose it.

I got my Secret clearance with an honest disclosure of past marijuana use that was outside of the past 6 months but within a year. They want you to be honest, and I was surprised to find that this particular agency requires it be outside of a year. Nothing regarding that turned up on Google, it's what my friend says he was told directly by the agency contact.