I have taken adderall consistently for over 6 years dude. One of the reasons I gave it up is the absolute loss of sex drive after a while of taking it. Literally I would avoid sexual situations while I was on it because of that plus the fact that I literally didn't care about sex while all pumped up on amphetamins. Once I stopped I was back to being a sex crazed animal.
Maybe your just implying your erectile dysfunction to everyone else. Heres some articles that back up my claim and prove that while your dick might no work on adderall most other people's dick works just fine if not better.