Holy shit, this is extremely reasonable!
I think it's more nuanced that just governments or the establishment are the problem. Lately I've come around to the idea that human nature on average tends to be shit. And while I wish that humans could self-govern, I have extreme doubts.
The people I respect don't blame the white man but rather "white supremacy." And that has a particular meaning (i.e. roughly the ideology that whiteness is better (which has a foothold across the world)...just as racism does (namely the systemic nature of the oppression).
But the American left likes to battle over words and semantics to virtue signal way more than is helpful. And so we (as a society) end up fighting about those words much more than is productive.
A friend once proposed the idea that capitalism requires a "haves" class and a "have-nots" class. From there, racism becomes one convenient vehicle for oppression. I suppose the visibility of the difference (between the haves and have-nots) could be key. Hence the yellow star for Jews in Nazi Germany. Or why women and people of different skin tones can fit. I'm just thinking out loud here. I found his theory interesting, to be sure. And perhaps the precursor to it being people who believed they could gain money, power, influence from capitalism (or whatever other economic structure).
And yeah, @Das Texan is 100% right about the political parties thing. Historians point to the King family's endorsement of JFK as the key event. That was preceded by JFK calling Coretta Scott King while Dr. King was in jail in 1960. See here.
And by the way, political parties are quite literally organizations/associations. They choose their own ideologies and activities. I vote Democrat because they are the closest to my ideas in most (definitely not all) cases.
But take Brazil for example. There, the left was disgustingly corrupt. It's what Mechanism (on Netflix) is about. If that were the case here, and a Republican with integrity was the opposition -- say, Mitt Romney -- then I'd vote Republican for the good of the country.