They could choose to live under Islamic rules, Unaffiliated Rules, Christians rules, etc. Or maybe have their own ex-Muslim millet and draft their own community rules.
My point was about the widespread notion that Islam calls for the execution of apostates. I've met many Muslims who think that simply leaving Islam is enough of a crime for the person to be killed. Heck, there was a thread about this on /r/islam recently.
In a popular vote majority wins. Here in America, a solid 40% of the country don't like the laws the Democrats enacted, but the Democrats won the popular vote, so they make the rules. The fact that the disgruntled minority doesn't approve doesn't give them leave to break the law.
That's a mischaracterization of the US system. The Democrats, if they won the majority of seats in Congress, cannot make whatever laws they want. They are limited by provisions in the Constitution and a series of judicial decisions by the Supreme Court that are meant to create limits to Congress's power. Moreover, there are many areas of law where Congress cannot interfere because they are considered state matters.
Moreover, the US has a dual legal system. It has the federal system, and the state system. The Constitution is supposed to lay out which powers the federal branch has, and which powers the state courts have. However, there is no way that the founding fathers could have predicted all the possible issues that might arise in this matter. Plus, a century later the Industrial Revolution reached America and all new sorts of legal claims were created. So the Supreme Court is supposed to be the body that is responsible for interpreting the Constitution for the federal courts and the state courts, and which areas of law is each responsible for. Ultimately, there are areas where the federal courts cannot intervene in the state's courts. Essentially, each state has a legal system of its own. They have their own constitutions and their own hierarchical structure of courts.
The point here is that, in the US system, there is one judicial body that reigns supreme over all others: The Supreme Court. It has multiple functions, but one of them is being a last resort court. Plus the power of the Supreme Court itself is limited by the Constitution.
The US system is extremely different from what you're envisioning.
Plus, there is a point you're not addressing. How do you create ways to sustain the system? Your imaginary state is a Muslim majority state. What if Muslims decide that shariah doesn't allow multiple legal systems and that the other systems have to be abolished? What if they decide shariah demands that it is the supreme law of the land and all must be subjected to it?