bro, this is a cool project. Nothing meaningful though. A lot of your bottlenecks is because of the core missing ingredient. An external module that handles neuro-symbolic relationships. Memory issues. You need to improve your text/token chucking strategy and design a custom solution...attention is all you need. This is pretty simple to be honest. I'm working on my own solution. Which won't see the light for 4-6 months. Using my crypto profits to train LLM from stratch[30b trained on 3 trillion tokens, 10% will be synthetic data, 64k context window{flash attention}], with some SOTA architectural optimizations. I'm being super anal about my pretraining dataset. Data quality is super important. I'm building a proto-AGI. able to handle complex task[ex: prototype on LLaMa was able to generate instagram clone using next.js and mysql{extensive finetuning, and other tricks[Judge0]}]. It will only have access to a few tools but it will have implicit tool use. I'm going to tokenify it and the goal it use build out a distributed community node network, where the community runs inference and is paid. A lot to work out. But IMO proto-AGI is possible now, and by that I mean autonomous complex task completion. Expand your approach bro, this isnt really any different that a langchain chatbot with hard coded emotion vectors.