For rape specifically I clarified in my other comment you responded to why it's a little more nuanced in that scenario. And the conviction to kill people comes from places that punishment would not be able to adhere to. If I see someone getting bullied, I'm more than willing to punch the bully across the face in spite of the zero tolerance policy we have at our school.

As for other crimes like theft, theft only exists (for 90% of the part) because someone is impoverished and can't provide for himself and his family without resorting to stealing. The over emphasis on material/ consumer identity in capitalism also plays a role. Within an anarchist framework, there'd be no need for theft and a lot of other crimes, so there would be a drastic reduction without any police force or secret agents to keep people in line