You've posed quite a few points here, so I'll try to answer them individually.

I'm curious, if something is legal then why should it scare people when you are legally doing it.

The AR-15 from a distance is indistinguishable from a military issue M16/M4. As the general public has little to no experience with those guns, outside of what the media shows from mass shootings people are naturally going to be afraid of them. Until such time as the public is educated, you can pretty much expect to be hassled if you're trotting one in plain sight down the street.

I do agree with your point but don't you think that it's strange that even though it's completely legal to carry a weapon it instills such panic in people and the police.

In most states, open carry is only done by law enforcement. When an average Joe/Jane on the street sees someone with a gun, they've been trained by the mainstream media to either identify that person as a cop or a criminal. It's a binary reaction on their part.

I'm just saying that if it legitimately frightens people that much then maybe the law should be changed or the alternative should be that we are taught not to fear people carrying firearms legally.

No argument there. Quality education will go a long way to solving many of our societal problems.

Sure the law isn't there to baby sit us and we should be free to make our own choice but right now we're taught to fear people who are only acting within the confines of the law, it's crazy

I agree. It's just that until such time as you convince the general public to take an interest in shooting sports and to have a better understanding of firearms, walking down the street with an AR will cause a panic. And as OP learned, life isn't fair or just.