No, we actually do want to have sex with all those women. They are not invisible to us, even if we are not pursuing them.
It is a matter of how you look at the threshold question. For women, a man has to be qualified to sleep with. For a man, a woman has to be disqualified to not sleep with.
So when a woman looks at a man she starts from the assumption that something about him needs to be good enough that she would want to sleep with him. For a man we our default is that we want to sleep with her, unless we can find a good reason why we would not sleep with her. It's opt in vs opt out.
The reason this may not make sense to you is that in a room of 100 women I might want to sleep with 95 of them, but I will only look twice at 40 of them and I will only attempt to speak to 10 of them. Those ten will be the ones that I think are most likely within my reach, not the top ten percent.
A woman might see me only looking twice at 40% of the women and interpret that as me not seeing or wanting to sleep with 60% of the women. This is not true, it is just a matter of the amount of effort vs the potential reward, and me coming to terms with the fact that my time and effort are not unlimited.
The fact that I may not pursue a woman does not mean that given the chance I would not sleep with her. Relationship status affects things as well. I am married, so I am not going to sleep with another woman behind my wife's back, so in that room of 100 women I am not going to make a move on any of them. That does not remove the desire though. I do still want to sleep with 95 of those 100 women. As an outside observer though, it is going to be hard for you to see that I desire to sleep with almost the entire room because you can only judge me by my actions, which will reflect the fact that I am not going to cheat on my wife.