I wonder how you would feel about, for example, Slavic languages. There, you leak information about the gender even about speaking about yourself in first person (at least in the ones I have some idea about). Example in Polish:

"Spotkałem kolegę" - "I [male] met a [male] friend."

"Spotkałam kolegę" - "I [female] met a [male] friend."

"Spotkałem koleżankę" - "I [male] met a [female] friend."

"Spotkałam koleżankę" - "I [female] met a [female] friend."

Notice the different ending (em/am) depending on whether a male or female is speaking.