Absolutely do not have every member of your wedding party give a speech. That will go on way too long. They won't be offended. Typically if you are farther down in the wedding party ordering you do not expect to give a speech.
The most you would want to do is each set of parents, one groomsman, and one bridesmaid.
First, decide whether you want them to speak at the wedding or rehearsal dinner. You could also do both, such as parents at the wedding and wedding party at the rehearsal dinner or vice versa. I think this mainly comes down to personal preference, such as how big your rehearsal dinner is going to be and how much time you have in your reception timeline.
Next, figure out who you want to speak, or ask them. Some people love speaking and some hate it. I think it is fine these days to have someone speak who is not the person standing closest next to you if that's what you all decide, especially since you don't have a designated MoH.
Finally, decide when this is going to happen. We talked to our venue who recommended toasts during salads.
To give you an example of how we are doing it and the reasons behind it (everything in wedding planning is so effing complicated, amirite?):
Each group gets 2.5 minutes, enforced by the DJ, for a total of 10 minutes, which is really the longest you want speeches to go on.
FH and I are going to be speaking at the rehearsal dinner.