Thats the point, you will never get the same rights and agreements as EU members

That’s true of literally every federation of confederation. Membership equals representation which equals a voice at the table to protect and promote your interests. Members will always have that voice, whereas non-members won’t. This means members are better able to force compromise instead of getting all-or-nothing deals.

The trade off is that membership in any federation or confederation requires that you give up some sovereignty. Non-members don’t face that problem, as their sovereignty remains fully in tact, leaving them free to fully accept or reject any proposal sought by the federation/confederation, which is a luxury the members don’t have (as proposals can, to a large degree, be forced upon members).

Your complaint is that you want the benefits of both membership (negotiation power) and non-membership (full sovereignty), and complain when you suffer the necessary detriment that must come with one of those. Yes, the Swiss have less negotiating power with the EU, but they also retain greater sovereignty than if they were an EU member. This is a choice your government has made, and there are consequences that come with that choice, just as there would be if you were members.