You say no one would ever wear it but this is a should question not a would/could question. It's a formal, government event -- there has to be some limitation. Should someone be allowed to masturbate or whatever at a ceremony? It doesn't hurt anyone, right?
I personally wouldn't because I don't see the KKK outfit or the SS uniform or whatever inherently offensive -- it is what they represent and at the Oath you're representing Canada and her values. These don't represent Canada's views so not allowing them is legitimate.
It isn't flawed logic because to a Neo-Nazi, they aren't intentionally trying to provoke anyone, just trying to express their view that Jews should all die or whatever. And it's not absurd to compare the SS uniform or whatever to the niqab because it wouldn't happen because the only different is that here are a hell of a lot more Muslims than there are Neo Nazis. Large masses of people genuinely believe that the niqab is rooted in sexism and a form of hate speech or whatever. Why should Canada uphold values that it doesn't in a ceremony that is about upholding Canadian values?
You say liberty but in Canada, we have restrictions on our freedoms. There is free speech but I can't participate in hate speech or give you a death threat. Just like you have the right to express yourself anyway you want, there are still restrictions. What are the restrictions in this case?
Formality - This is a formal event. Canadian formal, not what is formal somewhere else... only Canada. It must be demanded that in a formal event in Canada, especially one involving the government, Canadian formal attire must be worn. The niqab is not formal in Canada.
I swear (or affirm) that I will be faithful and bear true allegiance to Her Majesty Queen Elizabeth II, Queen of Canada, Her Heirs and Successors, and that I will faithfully observe the laws of Canada and fulfil my duties as a Canadian citizen.
Oath - One of the parts of the Oath is to faithfully observe the laws of Canada and fufil my duties as a Canadian citizen. Now what does that latter part mean? It means upholding Canadian values from what I dissect. What are Canadian values? If we look at the anthem or Canada's general culture one of them is equality, one of them is secularism and one of them liberty.
Let's look at secularism first - it is the separation of government and religion. I could argue that wearing the niqab, religious expression, at a government ceremony is not secular and thus not upholding a Canadian value and thus not legitimate when it comes to the oath. You are representing the Queen and Canada when you wear the niqab -- Canada demands this separation and wearing the niqab defeats this.
Next is equality and liberty. Now I'm not saying I hold this view, I'm not saying I don't hold this view but the niqab can easily be argued to be rooted in sexism. Why don't men wear it? Why are women supposed to wear it? Many arguments are based around women wanting to not be valued only for looks and so wear the niqab for this purpose.
Humans are social animals and looking at someone's face is natural, healthy and there are almost no drawbacks to simply looking at a human's face. Covering it up, only when pertaining to a women by law of a holy book written by God can be argued as oppression of women. This is not liberty, this is not equality.
If someone wore a KKK outfit or a Nazi outfit and did the Hitler salute or whatever, you're not standing for Canada's values and fulfilling your duties as a Canadian. There would be hysteria if this happened but there is a debate going on regarding the niqab only because there are a lot of Muslims. We shouldn't accept the niqab as religious expression because it is an attire born from sexism and doesn't uphold Canadian views.
(again, I'm not saying I hold/do not hold any of these views)