For people who appear partially masculine and partially feminine, they face the problem that either bathroom they use, they'll be challenged that they're using the "wrong bathroom". Effectively, these people are unable to use any public bathroom.
Moreover, legislators are reluctant to enact transgender non-discrimination laws because they fear that courts might interpret those laws as applying to the bathroom issue as well (even though most of these bills don't try to address the bathroom issue because it's considered an unsolved problem). Trans people are literally losing jobs because we can't solve the bathroom problem.