"For some the “fear” goes beyond ignorance and is based on an underlying prejudice that trans women are actually just men “pretending” to be women"
You could not be more wrong. This shows how little you have engaged in conversation with women who oppose the easing to 'self ID' along with the ability to legally change sex on documents such as birth certificates, driving licence.
The ignorance is all yours as you approach none of the impacts on single sex based protections legally definable by record of sex. Namely, sports, employment & equal opportunities, data collection crucial to understanding womens health, crimes by and perpetrated against, scholarships and other such sex protected funding streams which were designed to safeguard women from having fewer opportunities than men.
There are so many areas in which having such a right to legally change ones sex impacts negatively upon rights women have fought to be recognised and finally had acknowledged as a justifiable leveller of opportunities and recognition.
To plainly refuse to see these risks to womens rights, to plainly shut women out of the conversation by villifying women who dare speak up on this is as misogynistic in it's most directly underhand form as it is disturbing.