Hey there, I'm sorry that you've been going through so much. Please don't lose trust in yourself and in therapy. Sometimes things don't work out, don't blame yourself too harshly. My experience with AVPD blurred the lines between having feeling for someone and becoming dangerously dependent on them. I'm not in any way doubting that you have feelings for him, but as you say your fear of rejection makes the relationship complicated and ultimately becomes a self-fulfilling prophecy by nature of our disorder. Remember that this is his job and that he was trained to deal with situations like this. In my experience, it is almost always the case that we exaggerate anxious ideas and feelings internally to a very large extent. He most certainly doesn't see you the way you see yourself. I don't know either of you, but I'm absolutely sure about that.

What's done is done. It may be hard to accept but it is important that you try to accept it for your sake as well as his. I'm a guy, and I remember specifically asking for a male therapist because I was afraid of developing feelings for a female therapist. I do feel dependent on my current male therapist but it's not the same. I've learned that when relationships become difficult, it help a lot to take some space from those people and (at least temporarily) remove them entirely from your life and as much as possible from your mind. It will give you time to heal, gain perspective on what happened and get a chance to stop focusing exclusively on the negatives. That being said, I wouldn't recommend stopping therapy. You made a very important step in consulting a therapist in the first place and I really think you should continue therapy with a new therapist. Focus on yourself, get better and down the road if you want reach out to him when you feel that you're not afraid of his response hurting you. At that point you will know that you've made some great progress and he will be happy to see that.

I'm in no way knowledgeable about psychology or therapy, but I did find this article to be helpful. I also bought the book called 'Fear of intimacy' by Robert Firestone, which has some great insights into relationships in general but as the title suggests, I felt that its very appropriate to me as a person with AvPD.

Hang in there. It won't be like this forever. You always learn and adapt. It might not feel like it, but you always do. And if you ever feel about self-harming, please talk to someone about it. Take yourself out of that situation immediately, go outside or call a friend to change your state of mind. And please consult a professional. There is always hope. Remember that you always have the power to take action against dangerous urges that you may have. :)