Worked freelance for most of my life but in a slightly different position where it comes to clients. Meaning usually they need me more than I need them. Because of this I never really face your predicament but perhaps part of the answer for you still lies in that. You need them more than they need you. At least, that's how it feels to you. From a business perspective it's never good to rely on just one client so I think what you should do from a business perspective is change that.
Your current gig is a place many people want to work, that's a good thing because that place is now on your cv. Use it to open doors for yourself in to other companies.
As for emotion, emotion in business does play a huge role when you're in the negotiation phase of things - so when you talked with them and decided on a fee. When dealing with clients after that phase you need to keep it practical. By this I mean communicate clearly what you need to succeed in solving their problem.
As far as "asserting yourself" goes, I probably have a very different take on it than most people. I've been in leadership positions and honestly, I don't give 2 fucks whether someone under my management is assertive. All I care about is if they get the job done that they are being paid for. So I care about skill. If there's something they need to know in order to do their job effectively I expect clear communication about it. If they bring emotion in to that communication, I usually think of one of 2 possible reasons (in semi-random order):
they're in over their head
they're having problems in their private life
they want to do their job effectively but someone or something is preventing them from doing it
1 and 2 make people a potential liability so that's what sticks in the back of my mind. I won't act on it immediately but I will as soon as I see that performance goes down. How I deal with that depends on a lot of factors but it's important to know that all of those factors relate to how safeguard the project or work that needs to be done. It is after all a business, not a therapy session. What I will say though is that it can still end well. If someone is having personal problems and communicates clearly about that and gives me the impression they can solve them it's fine. That's just life. Note though that I say "gives me the impression**. I have to feel like they can solve it. Promises that they will are something completely unrelated.
In case 3 I will investigate what can be done to remedy the problem. But do note that replacing someone is still an option there. From a business perspective what matters is which solution takes the least time and/or money.
Now obviously there's room for being human in all of this! But I wouldn't be doing my job if I let that seriously jeopardize the primary objective of a business.
Hope it makes sense.
TL;DR: find more clients and keep emotion confined to negotiations :)