Deploying agents as a service
How to deploy a MemGPT server on the cloud.
You can use the MemGPT docker container to deploy MemGPT on cloud providers or on k8. We provide an example of how to deploy the container on fly.io below.
Fly.io
Requirements for Fly.io deployment
Currently, we only support deploying with Fly.io if you are using OpenAI for your LLM/embedding model backend, and SQLite and Chroma for your database backends. We will be adding support for other model provider and storage options soon.
- Install
flyctl
- Clone the git repository:
git clone https://github.com/cpacker/MemGPT.git
- Edit the server config at
configs/configs/server_config.yaml
to use Chroma & SQLite:
[defaults]
preset = memgpt_chat
persona = sam_pov
human = basic
[model]
model = gpt-4
model_endpoint = https://api.openai.com/v1
model_endpoint_type = openai
model_wrapper = null
context_window = 8192
[embedding]
embedding_endpoint_type = openai
embedding_endpoint = https://api.openai.com/v1
embedding_model = text-embedding-ada-002
embedding_dim = 1536
embedding_chunk_size = 300
[archival_storage]
type = chroma
path = /root/.memgpt/chroma
uri = null
[recall_storage]
type = sqlite
path = /root/.memgpt
uri = null
[metadata_storage]
type = sqlite
path = /root/.memgpt
uri = null
[client]
anon_clientid = 00000000-0000-0000-0000-000000000000
- Create a
fly.toml
file:
app = 'memgpt-server-app'
primary_region = 'sjc'
[env]
MEMGPT_SERVER_PASS = 'password' # TODO: your admin password
OPENAI_API_KEY = 'sk-...' # TODO: your OpenAI key
[build]
[http_service]
internal_port = 8083
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
processes = ['app']
[[vm]]
memory = '1gb'
cpu_kind = 'shared'
cpus = 1
- Launch the server with
fly launch
Updated about 2 months ago