Yes, I hear you. I know the “I don’t know”s are so frustrating, but often times they really don’t know why they did it. If the visual schedule cards you bought aren’t specific enough to what you need, you can always quickly create one on Google Slides or PowerPoint. For one I made I actually took pictures of his specific things like his toothbrush and his shower, so that it was incredibly clear what he needed to do. He still struggles with remembering all the steps of basic routines, but it has gotten better.
We went the therapy plus medication route, and the medication is what has truly made a difference for him. A good therapist will help you to be able to understand how his mind works as well. I have ADHD myself, so I “get” him better than my husband does, and he has to be explicitly taught that our son wasn’t trying to be purposely defiant. Testing also caught an anxiety disorder, which we had also suspected. He now has a 504 plan, which has put helpful accommodations into place for him at school.
Good luck and do your best to figure out how to understand how his brain works. It’s good that you’re starting early—my son’s self esteem was starting to tank because he felt like he was a “bad” kid, and that broke my heart. We’re on a much better path now. Good luck!