Who's being intolerant, the one demanding others change their behavior to suit their own ideals or the one who is unwilling to change their behavior?
The question as you pose it is outright not applicable to the situation. Pastors are not the individuals responsible for the registration or recognition of civil marriages, and religious institutions and their representatives such as Pastors are free to choose whom to marry, and/or whose marriages to religiously ordain or not.
It all comes down to this being utterly wrong:
Because there is no such thing. Marriage is an institution, ordained by God, that is the joining of a man and a woman. The government really has nothing to do with it and never should have.
Holy Matrimony in the Christian faith, or more broadly, religious marriage, are a set of institutions dictated by various faith regarding the relationship between an institution of faith (usually but not always a god or a spiritual entity) and two (or more) people. It however isn't the only type of marriage to exist, and it's important to recognise that it is not the same thing as Civil Marriage. The government in most modern countries (both inside and outside the west) does not have a say in religious marriages, but it does have it's own form of marriage called Civil Marriage; which exists to deal with various legal aspects that became associated with marriage over time, and to specifically also make sure those aspects become and stay available regardless of faith, or faith specific requirements to marriage.
In simple terms: Civil Marriage exists, in order to both protect the rights and protections of marriage, while also being able to protect religious freedom, including that right of the irreligious, those of minority religions, those whom religious movements might not want to marry and those of the religious movements themselves.
It is funny you say "If you want to involve the govt. just make it about that. Why not just make it a civil union... ...and silence the arguing?" - Because that is quite literally what Civil Marriage is. A form of union (usually between two people) dictated by Civil Law (hence the name "Civil" Marriage) and registered by the government, rather then recognised and ordained by a religious institution.
That's not some newfangled thing that just came along with same-sex marriage either. There's a good chance if you live anywhere outside the middle east, that Civil marriage has been available in your local area for over a century.
All of this leaves you in a very odd position with the statements you've made: You essentially started off with a false claim about the nature of marriage, stated opposition to same-sex marriage, then continued to state you'd be fine with government recognised unions for same-sex couples that are themselves not about religion which is what Civil Marriage already is.
Not only did you tie yourself in a logical pretzel there, you did it twice:
Who's being intolerant, the one demanding others change their behavior to suit their own ideals or the one who is unwilling to change their behavior?
Remember what I said about Civil Marriage existing because of Religious Freedom, and how Civil Marriage protects legal rights and protections for Married couples even if religious institutions don't approve? That applies no less to same-sex couples. Those protections Civil Marriage offer are not arbitrary. If religious institutions/people/pastors/anyone would get their way with religious marriage arguments against the legality/recognition of same-sex marriage (which again, would be about civil marriage recognition), that'd force same-sex couples to behave differently in the way they form and protect their relationships and families. - And much so to their detriment, and in violation of their religious freedom.
Even within your own reasoning: Those taking the stance against legal same-sex marriage, that you're defending.