It will be different, but still challenging in new ways you might not even imagine.
I went from a super low-ses district where all my students were in public housing and there was zero parental support. I saw a lot of the destructive/disrespectful behaviors you mention, and I was relieved when I landed a job in a 'nicer' district.
There are still challenges here, but completely different ones. No one wanted to go to my old school, so class sizes were small and I only had 100 students total. Now I have 30+ in a class and 190 students I'm responsible for. Instead of working closely with ~15 teachers there are over 40 teachers at my site and I barely know some of them.
At the old school there was zero parental support which meant no homework getting done, no one answering if I called, etc. Now there's a touch too much parental support i.e. when I upload new grades I get a cascade of e-mails wanting to know why Jimmy lost 3 points on his lab, they read (* cough* did) the whole thing and it was amazing, and should we schedule a meeting to discuss his progress!
Behavior is better in that I don't have to call security on anybody, but the constant battle to win attention away from their phones and socializing is difficult. If you push them to hard and things end up in the Principals office it will be framed as your fault, not theirs.
Instead of writing dozens of referrals and behavior plans, you'll be filling out private school recommendations and behavior analysis to support their parents divorce proceedings.
There will be challenges wherever you go. Tough schools need good teachers, but not everyone is cut out for the day-in-day-out struggles with little reward. Some people can't handle the high-stakes and intense scrutiny of a high-performing school. There's nothing wrong with trying out the other side for a while. There will always be jobs in those tough schools if you decide it was a better place for you!