It is normal. Moreover, if you do not use linear algebra in practice, I hardly imagine you can EVER understand what is happening.
I personally took linear algebra literally 3 times: 1st time I got a book by Klein, Coding the Matrix: Linear Algebra through Applications to Computer Science. It is all hands-on. I immediately started using it for linear regression/ML stuff. At that point I did not understand what I was doing very much except matrix multiplication being convenient to store numbers.
Then I encountered more complex issues, in particular diagonalizations in statistics. When you do a lot of ML stuff, you encounter whitening, or normal distribution transformations. These all are based off of more advanced linear algebra. So since I felt lost again, I picked up Gilbert Strang book and online courses. I started learning, and in some time I was able to understand matrix multiplications as column/row operations, to understand more about projections, what diagonalizations were, etc. Strang's book helped me understand more advanced linear algebra in a relatively short time.
However, I was still at a loss sometimes, reading internet materials/research papers/etc. In particular, I was lost at the point when people were saying that matrices are somehow connected to linear operators... And what was an operator anyways? I did not understand the importance of eigenvalues/vectors, did not see why spectral theorem is important, could not figure what similar matrices were, and could not grasp the meaning behind special (but ubiquitous) cases, like symmetric matrices. At that point I read Apostol Calculs volume 2 (first 5 chapters are dedicated to linear algebra), and finally all things went in place somehow...
Now, I can USE linear algebra, I understand MOST of it, and I seem to have a good intuition about it. I do not know it on the level of 500 pages of Axler, but I definitely know enough to feel confident with it.
So to summarize, in order to achieve good understanding of linear algebra you also need to practice a LOT. And always try to connect theoretical things you are learning to some practical tasks. One of the great things about linear algebra is that it is used literally everywhere. You can easily find examples with "real life" applications, from engineering to differential equations.
NB. On a side note, I learned differential equations like you did linear algebra. I never used them, even though i had rigorous theoretical course about ODEs. I am sure if I take a good book, I can immediately pick up what I was doing with ODEs and maybe solve some exercises. But ask me ANYTHING remotely tricky about ODEs, I would never be able to answer your question. Neither I am able to formulate problems in terms of differential equations. I blame the lack of practice since differential equations are not used much in statistics or ML. So, your particular case is not unique at all. Continue fighting.