Think of it this way: if I were to say to you, I like science, but don't know where to begin, what would your advice be? The subject of art history is vast. So you want both to the get the sort of overview of art history that you would receive in a first-year university survey of the subject, and focus on things that interest you.

You are very fortunate to live near museums. See if any have guided tours, and take them. Museum docents are usually very well-trained, informative, and engaging. It is an incredible experience to be able to learn about objects while seeing them in the flesh. Also keep an eye out for temporary, special exhibitions. They will give you an in-depth experience of a specific subject and a sense of what might interest you further.

As you start to learn more, you will find yourself drawn to certain pieces, techniques, and subjects, and by extension certain cultures and time periods. Once this happens you can start to research these specific subjects rather than feeling completely overwhelmed by the vastness of art history.

While you are doing this, you'll want to slowly build up an overview of art history. The Khan Academy provides a decent, free place to begin, or explore more specific topics within art history.

Finally, read books! For an engaging overview, something like E. H. Gombrich's The Story of Art is an old but accessible starting point. If you become interested in art history as a discipline, Donald Preziosi's The Art of Art History is a good.

Have fun!