You want him to be more assertive without being a dick is what it sounds like, which a lot of guys find to be a difficult line to balance on.

First off, your boyfriend is not perfect. No one is, and it's a bad idea to idealize someone that way. Especially when there are very real things you're not especially pleased with regarding his personality. He has his flaws, as does everyone else.

It's a tough situation since he's already sensitive about appearing weak/vulnerable, so I'd take it easy on him. Also, telling someone who is naturally shy or meek "I want you to be more assertive/confident" can be kinda like telling a fish "I want you to live on land." So be prepared for this to be a long process if he decides he wants to change for you, where he will struggle to be more outspoken and confident for fear of upsetting you (even if you tell him he won't) or out of general anxiety.

I'd say you should start with the bedroom since it's less likely that this will impact his current fear of feeling like he's weak. Tell him you like to feel controlled during sex, or that you want him to be a bit rough with you. Discuss it first. Things you like, things you don't like. Things he's comfortable with doing for/to you and things he isn't. Establishing a safe word is a good idea to make both of you feel more comfortable with it if you're going to be exploring any level of power play.

If he goes for that and things in the bedroom improve over time, just continue to encourage that more dominant side of him. Make sure he's aware of how much it excites you when he takes control. If he finds he enjoys being in control in that aspect of things it may help him be a bit more assertive outside the bedroom as well, especially if he knows you like it as much as he does and, more importantly, that you trust him not to cross the line.

If he can't seem to find it in him to be more assertive or outspoken or whatever it is you're wishing he could be then I suppose you've got to decide if that's important enough to you to end the relationship and see other people.