STEM courses that are required starting out are commonly referred to as "weed out" courses because lots of the general population find these topics interesting and want the job security of working in the STEM field. If you find that you can't understand the material, don't have the patience, or are otherwise not interested in these basic courses then it's my opinion that its an indicator that you shouldn't major in Neuroscience or STEM field.
I'm passionate about lots of things and when i was a freshman and sophomore, I was obsessed with programming. But I didn't understand my beginner programming class (but somehow passed), failed Data structures & Algorithms (which is a basic class), and also barely passed Calculus I. I'm not an idiot, but i do have a lot of reasons why I didn't pass so I read the writing on the wall and majored in something that was a better fit for me. To be honest there's a part of me that still wants to do programming but if i'm not good at it then how am i going to get a job doing it?
I think you should consider switching degrees while time is on your side but that's just based on my experience.