People who say this are talking about the system of racism, not simply ignorant racial prejudice and generalizations.

The system of racism, from the beginnings of this country, were set up to help some (wealthy white people) and actively oppress others. So when something racially prejudice is said about a black person, a Native American person, an Asian person, it typically is derived from a stereotype that has deliberately been shown in society to hurt that group, cause fear and hatred of them, and keep them from progressing. So it was designed as a hierarchy, where power and influence only flow downwards.

Working from that understanding of racism, black people and other people of color don't have the ability to oppress white people as an entire group. The bottom of the pyramid doesn't have power to change the perception of the people at the top. No matter what racial stereotype a person of color says about white people, it doesn't have the context of a history designed to harm white people. Instead, our context is a society that has actively worked to harm people of color, and that is the system racism.