It's kind of complicated and it has to do with a slightly different understanding of the word "racism" (or sexism, homophobia, or any other form of bigotry).

See, the social justice movement has kind of halfway moved past the idea of "racists". Racism isn't really a thing people are or aren't or even (to a lesser extent, more on that if you want) behaviors that people do or don't do. It's a set of social expectations, a system that emerges when you look at trends across society at large. Some groups of people will tend to have better or worse lives than others for no reason other than their membership in those groups.

So one of the ways the social justice movement is grappling with this huge, nebulous concept of injustice is by trying to dismantle the systems that create this injustice. One of the ways to do this is to create places where underprivileged/disavantaged/whatever other adjective people can get together and be a community without the systems of power imbalance and injustice interfering in what they say or do. Those systems are often used to make certain groups feel or actually be unsafe.

From that perspective, it absolutely is not racism to exclude white people from those spaces. It is a judgement made based on racial categories, but it's made in spite of institutionalized racism, or the system of power imbalance. From a more common understanding of racism, i.e. treating one group differently from another group based on race, yes. It is racist.

What the lady involved said was kind of dumb, but it was spoken by someone and intended mainly for an audience with the more nuanced understanding of racism. I don't agree completely, but I understand where she was coming from.

I hope I've made it a bit clearer.