"Yeah, but what makes you think that pants are a man’s garment?"

Good question. I have a four part answer to this which demonstrates that pants always pertain to men, even today.

1. "Breeches" were an article of clothing designed by God for the priests who were all men. The word does not occur very often in scripture, but in every case it’s men’s apparel (Exodus 28:42, Leviticus 6:10, 16:4). According to my Hebrew lexicon, "breeches" means "trousers that extend to the knee, below the knee, or to the ankles." This would include pants, shorts, or culottes.
2. Until the advent of Hollywood and the movie screen, everyone (including lost people) knew that pants were men’s apparel and dresses were women’s apparel, and they dressed accordingly. Our culture’s (and sadly most churches’) acceptance of cross-dressing has resulted largely from the influence of television, the placement of women in the workforce, and the pressures of twentieth century feminism.
3. The universal symbol for designating a men’s bathroom is a stick figure wearing a pair of pants. The universal symbol for designating a woman’s bathroom is a stick figure wearing a dress. Coincidence? Hardly. Even our sinful society recognizes that there is a difference in a man’s and woman’s clothing.
4. Pants are a symbol of authority, as evidenced by the saying " I’m the one who wears the pants in the family." Sadly, most women might as well wear the pants, since they rule their homes anyway!