I understand what you are saying bit Ive come to understand its the whole "not here, not now" mentality that is the problem. If not now, if not here than when and where? Someplace where they aren't bothering anyone? Somewhere they wont get any attention? That's not very productive either.

The dreamers went where "dreams come true".

"First, I must confess that over the last few years I have been gravely disappointed with the white moderate. I have almost reached the regrettable conclusion that. the Negro's great stumbling block in the stride toward freedom is not the White Citizen's Council-er or the Ku Klux Klanner, but the white moderate who is more devoted to "order" than to justice; who prefers a negative peace which is the absence of tension to a positive peace which is the presence of justice; who constantly says "I agree with you in the goal you seek, but I can't agree with your methods of direct action;" who paternalistically feels he can set the timetable for another man's freedom; who lives by the myth of time and who constantly advises the Negro to wait until a "more convenient season."

MLK Jr.