When it is tackled, there are various ways that people try to define it:

• genitalia
  This one is complicated.
  • According to popular media depictions of transsexuality, this is the gold standard. If you've had "the surgery", you're officially trans; if you haven't, you're not. People don't realize how wrong-headed this is, in so many ways.
  • Legislatures, when seeking a definition that reduces the likelihood that people will vacillate between male and female (which is "bad" because it requires extra paperwork and potentially can be used to game the system) (this totally ignores the fact that gender can both be 1) fluid, and 2) non-binary), will sometimes settle on genitalia (requesting that a physician certify that someone has had bottom surgery) as a way to reduce vaccilation.
  • However, this IS undoubtedly one of the main ways that is used to determine gender. "Traditionally, an attending physician or mid-wife determines a newborn's gender at birth after a visual inspection of the newborn's genitalia. If the child has a penis, scrotum, and testicles, the attendant declares the child to be male. If the child does not have a penis, scrotum, and testicles, the attendant declares the child to be female. This declaration is then memorialized by a certificate of birth, without an examination of the child's chromosomes or an inquiry about how the child feels about its sexual identity."
  • In practice, if a stranger on the street wants to challenge your gender, they're not going to ask to see your genitals to prove that your gender identity is "correct" — asking other strangers to see their genitals is obviously rude, no matter how transphobic someone is. Even in the case of a police officer who wants to challenge someone for being in the "wrong bathroom", they won't examine your genitals. (though, rarely, they may arrange for a physician to do an on-the-spot exmaination)
• chromosomes
  • Science-minded people sometimes point to this as one solution to the "problem".
  • One obvious way this fails is for intersex people. There are numerous combinations that intersex chromosomes can occur — I've heard that they number over 100 combinations.
  • In practice, very few people know what their chromosomes are — While the economics of genome mapping are getting better, they are still too expensive to be commonly done. So individuals often don't know for sure if they're intersex. Even if individuals can eventually afford to know for sure, this information may be protected by medical privacy laws (if intersex people wish to not be outed by default), and may not be generally disclosed.
• what it says on your driver's license / birth certificate / passport
  • Trans people can and do get their legal designation changed. The criteria for changing this varies greatly by state and country, ranging from "you can change it whenever you feel like it, for any reason" to "a physician must certify that you've had bottom surgery" (as well as, obviously, "you can't change it", in many middle-eastern and african countries).
  • People can have an "M" on their driver's license, appear androgynous or clearly female, and have large tits. (true story: in the state of Illinois, it's totally legal for trans folks to flash people with their size "C" tits, as long as they have "M" on their driver's license. Some of my trans friends really relish this fact.)
    • (That sounds kind of a jackhole attitude to take, but I'll explain. The reason it's humorous is that 1) the law is almost always completely out of touch with the lived realities of transgender people, and this is just one more example, but 2) in almost every other case, the law places the burden on the transgender person; in very few cases does it give them more options)
• what you most easily pass as
  • This is what most transgender groups suggest the official policy should be for "which bathroom do I use?". It recognizes that gender is non-binary, and that it's fluid.
  • However, this is somewhat subjective (passing is how someone else [person B] reads a trans person [person A], and it can be totally different for every pair of people) and is thus difficult to codify into law.