I was a gifted kid who stayed in a mainstream classroom. My brother (one year older) was a gifted kid who was in the gifted program. It worked out well for both of us.
In my school region, the gifted class was not as diverse (socioeconomically or racially) as the mainstream class and also seemed to attract a certain type of kid - most were Type A, very driven for success, others were a quirky type - innovators and big idea kids. That wasn't me. I was a laid back , easy going kid who cared more about socializing at school, and learning about people than academic achievement. My brother was a great fit for the gifted program.
My parents made the initial decisions but I had a couple chances over the years to switch into the gifted program and I always chose not to. The workload in the gifted program was higher and I wanted to play sports, have an active social life, volunteer, travel and generally have lots of free time/down time.
In elementary school I did do some pull out enrichment classes at various times and was given advanced work when I finished my work early. I was bored a lot but I didn't really care.
All the way through school teachers etc told me I was a gifted underachiever, that if I worked harder and focused more on academic achievement and success I could have been an A+ student. I did well in school with very little effort and had no desire to really do better. My definition of success in life was never high grades. I never really understood the fixation on needing to have my identity or prioritize my time around grades and school work. I had other priorities and still do!
In the end it has all worked out well. Both my brother and I have been very successful. Years later I went on to graduate school and now work in academia! I am still laid back, and still only see my work as one part of who I am, and still don't define success as work success.
I think gifted programs are great for certain kids but if a child is truly gifted they have the capabilities, talents and skills to ultimately do well. It really all depends on the fit for the child and how you define success.