Hello! I’m following the short course on MemGPT and I’m interested in putting it in practice in a real project. I want this project to be scalable, and have the possibility to instantiate N agents and then destroy them when I don’t need them anymore. I believe I might use a structure that is similar to what you provide for the course: for each agent, a jupyter notebook and, I imagine, a docker image behind? Can I ask you more information on the technical infrastructure that you use? Is the LLM called via API from the docker machine? Thank you in advance
Letta runs as a service - you can see instructions for setting up a server on the Letta docs.
The server connects to one or more LLM providers (e.g. OpenAI, Anthropic, etc.) and a database backend (Postgres or SQLite).
All agents are backed by a single server, so you only need one. The number of agents you can create will probably be bottlenecked by your DB size and LLM rate limits, but you can scale up the number of containers running to back the service to scale up.
1 Like
Hi Sarah, thank you for your reply! In your experience, for the sake of scalability, how many agents do you suggest to host in a single container? If I understand correctly, you suggest to keep the DB inside of the container, right? In my use case, I envisage to develop an ‘orchestrator’ as an external application, where I put in place a scenario, and from which I can instantiate N agents, give them human characteristics, a job, a task (like daily schedule) and personality traits, and run a disruptive event (like a flood, some news, etc) , let them interact and see what happens. From your experience, do you have suggestions on the possible architecture, considering that I may require thousands of agents at a certain stage?