I guess in that case the scenario is an nb person being open about their gender identity from the get-go and the counterpart being at least open to the idea of having a gender non-conforming partner or open to being educated about it through discussion and maybe some readings...

Both are true.

worry about things like 'am I not straight if I date a genderqueer person?'

I've always considered myself a straight male. I've always been attracted to people who have feminine physical characteristics from my end. The problem isn't wondering if I'm straight. The problem is wondering if harming or disrespecting a nonbinary person to whom I am attracted by subconsciously treating them as female. If that makes sense.

it can be really empowering and liberating if you care about a gnc person and want to have a relationship with them

I will always care about them. You don't get close to someone without doing that.

at the end of the day labels should not matter that much when love is on the table

That's my opinion too.

If I have a concept of myself, they have a concept of theirself, and all that is compatible... I'm good with it.