Deploying agents as a service

How to deploy a MemGPT server on the cloud.

You can use the MemGPT docker container to deploy MemGPT on cloud providers or on k8. We provide an example of how to deploy the container on fly.io below.

Fly.io

⚠️

Requirements for Fly.io deployment

Currently, we only support deploying with Fly.io if you are using OpenAI for your LLM/embedding model backend, and SQLite and Chroma for your database backends. We will be adding support for other model provider and storage options soon.

1. Install flyctl
2. Clone the git repository: git clone https://github.com/cpacker/MemGPT.git
3. Edit the server config at configs/configs/server_config.yaml to use Chroma & SQLite:

YAML

[defaults]
preset = memgpt_chat
persona = sam_pov
human = basic

[model]
model = gpt-4
model_endpoint = https://api.openai.com/v1
model_endpoint_type = openai
model_wrapper = null
context_window = 8192

[embedding]
embedding_endpoint_type = openai
embedding_endpoint = https://api.openai.com/v1
embedding_model = text-embedding-ada-002
embedding_dim = 1536
embedding_chunk_size = 300

[archival_storage]
type = chroma
path = /root/.memgpt/chroma
uri = null

[recall_storage]
type = sqlite
path = /root/.memgpt
uri = null

[metadata_storage]
type = sqlite
path = /root/.memgpt
uri = null

[client]
anon_clientid = 00000000-0000-0000-0000-000000000000

4. Create a fly.toml file:

yaml

app = 'memgpt-server-app'
primary_region = 'sjc'

[env]
MEMGPT_SERVER_PASS = 'password' # TODO: your admin password 
OPENAI_API_KEY = 'sk-...' # TODO: your OpenAI key 

[build]

[http_service]
  internal_port = 8083
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[[vm]]
  memory = '1gb'
  cpu_kind = 'shared'
  cpus = 1

5. Launch the server with fly launch