Yes, of course you should memorize it all! In fact, you should work towards being able to recite it all without doing any calculations. "A major to B is fine but it needs a C# because A minor has C natural" is like calculating 8+4 with your fingers as an adult. It's veeery bad and indicative of a lazy mind and low work ethic. I think any person who truly loves music will find that bit of memorization incredibly easy. Trust me that what little work you put into music theory is like planting a fruit tree, and it will be returned to you manifold. So if you memorization exercises feel boring to you, remember that once the tree is planted, you'll eventually be harvesting its sweet fruit for many many years to come! So I would say, you should not feel bored but excited while doing your memorization exercises. If music is your dear friend that you tresure and love, this is definitely moving towards the next step in a very rewarding relationship!

If it feels too abstract, practice staff notation and play the scales on piano or keyboard. Sorry to say, but learning this aspect of theory is just so much easier on the piano. On guitar, every key has more or less the same shape, on piano they don't. You also won't be as confused about enharmonics. So practicing on piano will get you acquainted with your keys much more easily

Good luck on your journey!