They do stop you, this is a large part of why people consider things like access to housing, food etc a human right.

If a group of people takes over an entire country, won't let you sleep in forests etc because they "own" them and makes it impossible for you to live your life, the solution is to kill them all. To avoid this outcome governments make it easier for us to live so everyone benefits from their presence and we're motivated to follow their rules. If it ever becomes more efficient to build your own house than buy one or grow your own food than work for it, they are failing to uphold their end of the deal. We reached that point a long time ago in a lot of countries and the reason we put up with it is mostly because capitalists erode communities & education and encourage hyper-specialisation so that we're so isolated, useless and dependant on the system to live, we can't imagine an alternative.