I went through the same experience that you did and coincidentally, I took on the challenge of audio programming. I had to deal with threads and lock free queues from the get go. I offered myself a very challenging task and was frustrated for weeks.
Eventually, things started to click. The rules around Rust's Ownership system started to become "natural". As in, I would automatically reason about them without thinking about them, while writing the code. It was like driving a car for miles then suddenly coming to and you cant remember what the last 5 mins of the drive was. You were on autopilot.
The type system is the most powerful and complex that I've ever used. At first, the sheer complexity and the combinatorics of the many different ways to solve a problem was overwhelming. This too however started to become "natural". Progressively, after much use of the language I began to see patterns of common ways I've used the type system to solve problems. I would still have to think about what I was designing, you always have to think about design, but now the patterns just came to me.
The audio programming I was doing involved pipelining raw input into a chain of function calls where for example an array of PCM bytes can be feed through transformation function calls `pcm(data).bitrate_transform(16).16_bit_to_byte_chunks(1024).map(|| // some other transform)`. In any other language I would have to make all sorts of runtime assertions that the API was being used with properly aligned types (compiler must statically prevent calling 16_bit_to_byte_chunks() on arrays not of 16 bitrate audio). For more details on that particular pattern look at the Type State pattern and Iterators. I could write that constraint in Rust! That is just amazing to me.
My advice, no.. Not advice, I IMPLORE you to keep going. Work through the growing pains. It will hurt. You will stumble and think that you will never just understand it. But I can assure you this is wrong. I was wrong when I felt like this, probably more than you. I gave up on Rust and was ready to bury the language. I came back for round 2 and now here I am.