That the sentiment is 180° wrong. Framing things in terms of "politically correct" suggests that people should feel free to say the first thing that comes to mind. Framing things in terms of privilege suggests that people should put a lot of effort into seeing past their privilege:

Privilege is, like I said above, nasty. It is sneaky, it is quiet, it is powerful. You will have a hard as hell time seeing past that stained glass window to the horrible shit beyond. I know I do. You have a burden due to that privilege, to do everything you can to see past it.

There isn't really much middle ground here between these two viewpoints. One says you should work hard, the other says you shouldn't.

In this context, the conceit that "political correctness" constitutes a violation of free speech is particularly zany; as though society's marginalized groups wield oppressive power over the dominant mainstream.

Simply put, the great "PC" cliché, as commonly deployed in mainstream discourse, is cultural propaganda designed to befuddle and misdirect while defending the current power structure. All politics deal with power relations, and in the debate over America's alleged climate of "political correctness", there's a stark asymmetry of power between the defiant megaphone-wielders who complain of being constrained by humorless hypersensitivity from below, and the under-represented people of color, women, LGBT, handicapped, poor, and otherwise marginalized or dispossessed people who have no choice but to absorb the linguistic, cultural, and physical barbs of the ruling class. The former feel psycho-emotionally oppressed by their inability to crack puerile ethnic jokes without criticism; the latter simply are oppressed.

Political correctness is often used as a pejorative, but like many pejoratives, it can be reclaimed:

If you are tempted to begin an argument against something we’ve said here with, “God, stop being so PC!” just stop right there. We are proudly PC and have absolutely no intention of stopping. Racist, classist, homophobic, transphobic, ableist, and sexist comments are as unwelcome here as sizeist ones.