People who want to be androgynously stylish end up having to spend more time thinking about fashion than even gay guys. They have to learn both men's and women's fashion, and then find sensible ways to blend the two. "You have to learn the rules before you can break them."
The fact that there are very few androgynous people to glean style cues from, means that the job is even harder. (granted, runway models have been pretty androgynous for a while, but I don't think runway fashion translates well into normal practical clothing)