- Admin Creation: The first account created on Manta gains Administrator privileges, controlling user management and system settings.
- User Registrations: Subsequent sign-ups start with Pending status, requiring Administrator approval for access.
- Privacy and Data Security: All your data, including login details, is locally stored on your device. Manta ensures strict confidentiality and no external requests for enhanced privacy and security.
- All models are private by default. Models must be explicitly shared via groups or by being made public. If a model is assigned to a group, only members of that group can see it. If a model is made public, anyone on the instance can see it.
Choose your preferred installation method below:
- Docker: Officially supported and recommended for most users
- Python: Suitable for low-resource environments or those wanting a manual setup
- Kubernetes: Ideal for enterprise deployments that require scaling and orchestration
- Docker
- Kubernetes
- Third Party
- Docker
- Docker Compose
- Podman
- Docker Swarm
Quick Start with Docker π³β
Follow these steps to install Manta with Docker.
Step 1: Pull the Required Docker Imageβ
Start by pulling the latest Manta Docker image used by Manta from the GitHub Container Registry.
docker pull ghcr.io/cirrascalecloudservices/manta:main
Step 2: Run the Containerβ
Run the container with default settings. This command includes a volume mapping to ensure persistent data storage.
docker run -d -p 3000:8080 -v manta:/app/backend/data --name manta ghcr.io/cirrascalecloudservices/manta:main
Important Flagsβ
- Volume Mapping (
-v manta:/app/backend/data): Ensures persistent storage of your data. This prevents data loss between container restarts. - Port Mapping (
-p 3000:8080): Exposes the Manta WebUI on port 3000 of your local machine.
Using GPU Supportβ
For Nvidia GPU support, add --gpus all to the docker run command:
docker run -d -p 3000:8080 --gpus all -v manta:/app/backend/data --name manta ghcr.io/cirrascalecloudservices/manta:cuda
Single-User Mode (Disabling Login)β
To bypass the login page for a single-user setup, set the WEBUI_AUTH environment variable to False:
docker run -d -p 3000:8080 -e WEBUI_AUTH=False -v manta:/app/backend/data --name manta ghcr.io/cirrascalecloudservices/manta:main
:::warning
You cannot switch between single-user mode and multi-account mode after this change.
:::
#### Advanced Configuration: Connecting to Ollama on a Different Server
To connect Manta to an Ollama server located on another host, add the `OLLAMA_BASE_URL` environment variable:
```bash
docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://example.com -v manta:/app/backend/data --name manta --restart always ghcr.io/cirrascalecloudservices/manta:main
Access the WebUIβ
After the container is running, access Manta at:
For detailed help on each Docker flag, see Docker's documentation.
Updatingβ
To update your local Docker installation to the latest version, you can either use Watchtower or manually update the container.
Option 1: Using Watchtowerβ
With Watchtower, you can automate the update process:
docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once manta
(Replace manta with your container's name if it's different.)
Option 2: Manual Updateβ
-
Stop and remove the current container:
docker rm -f manta -
Pull the latest version:
docker pull ghcr.io/cirrascalecloudservices/manta:main -
Start the container again:
docker run -d -p 3000:8080 -v manta:/app/backend/data --name manta ghcr.io/cirrascalecloudservices/manta:main
Both methods will get your Docker instance updated and running with the latest build.
Docker Compose Setup
Using Docker Compose simplifies the management of multi-container Docker applications.
If you don't have Docker installed, check out our Docker installation tutorial.
Docker Compose requires an additional package, docker-compose-v2.
Warning: Older Docker Compose tutorials may reference version 1 syntax, which uses commands like docker-compose build. Ensure you use version 2 syntax, which uses commands like docker compose build (note the space instead of a hyphen).
Example docker-compose.ymlβ
Here is an example configuration file for setting up Manta with Docker Compose: This setup uses the official Manta Docker image.
version: '3'
services:
manta:
image: ghcr.io/cirrascalecloudservices/manta:main
ports:
- "3000:8080"
volumes:
- manta:/app/backend/data
volumes:
manta:
Starting the Servicesβ
To start your services, run the following command:
docker compose up -d
Helper Scriptβ
A useful helper script called run-compose.sh is included with the codebase. This script assists in choosing which Docker Compose files to include in your deployment, streamlining the setup process.
Note: For Nvidia GPU support, you change the image from ghcr.io/cirrascalecloudservices/manta:main to ghcr.io/cirrascalecloudservices/manta:cuda and add the following to your service definition in the docker-compose.yml file:
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
This setup ensures that your application can leverage GPU resources when available.
Using Podman
Podman is a daemonless container engine for developing, managing, and running OCI Containers.
Basic Commandsβ
-
Run a Container:
podman run -d --name manta -p 3000:8080 -v manta:/app/backend/data ghcr.io/cirrascalecloudservices/manta:main -
List Running Containers:
podman ps
Networking with Podmanβ
If networking issues arise, use slirp4netns to adjust the pod's network settings to allow the container to access your computer's ports.
Ensure you have slirp4netns installed, remove the previous container if it exists using podman rm, and start a new container with
podman run -d --network=slirp4netns:allow_host_loopback=true --name manta -p 3000:8080 -v manta:/app/backend/data ghcr.io/cirrascalecloudservices/manta:main
If you are using Ollama from your computer (not running inside a container),
Once inside Manta, navigate to Settings > Admin Settings > Connections and create a new Ollama API connection to http://10.0.2.2:[OLLAMA PORT]. By default, the Ollama port is 11434.
Refer to the Podman documentation for advanced configurations.
Docker Swarmβ
This installation method requires knowledge on Docker Swarms, as it utilizes a stack file to deploy 3 seperate containers as services in a Docker Swarm.
It includes isolated containers of ChromaDB, Ollama, and Manta container for Manta. Additionally, there are pre-filled Environment Variables to further illustrate the setup.
Choose the appropriate command based on your hardware setup:
-
Before Starting:
Directories for your volumes need to be created on the host, or you can specify a custom location or volume.
The current example utilizes an isolated dir
data, which is within the same dir as thedocker-stack.yaml.-
For example:
mkdir -p data/manta data/chromadb data/ollama
-
-
With GPU Support:
Docker-stack.yamlβ
version: '3.9'
services:
manta:
image: ghcr.io/cirrascalecloudservices/manta:main
depends_on:
- chromadb
- ollama
volumes:
- ./data/manta:/app/backend/data
environment:
DATA_DIR: /app/backend/data
OLLAMA_BASE_URLS: http://ollama:11434
CHROMA_HTTP_PORT: 8000
CHROMA_HTTP_HOST: chromadb
CHROMA_TENANT: default_tenant
VECTOR_DB: chroma
WEBUI_NAME: Awesome ChatBot
CORS_ALLOW_ORIGIN: "*" # This is the current Default, will need to change before going live
RAG_EMBEDDING_ENGINE: ollama
RAG_EMBEDDING_MODEL: nomic-embed-text-v1.5
RAG_EMBEDDING_MODEL_TRUST_REMOTE_CODE: "True"
ports:
- target: 8080
published: 8080
mode: overlay
deploy:
replicas: 1
restart_policy:
condition: any
delay: 5s
max_attempts: 3
chromadb:
hostname: chromadb
image: chromadb/chroma:0.5.15
volumes:
- ./data/chromadb:/chroma/chroma
environment:
- IS_PERSISTENT=TRUE
- ALLOW_RESET=TRUE
- PERSIST_DIRECTORY=/chroma/chroma
ports:
- target: 8000
published: 8000
mode: overlay
deploy:
replicas: 1
restart_policy:
condition: any
delay: 5s
max_attempts: 3
healthcheck:
test: ["CMD-SHELL", "curl localhost:8000/api/v1/heartbeat || exit 1"]
interval: 10s
retries: 2
start_period: 5s
timeout: 10s
ollama:
image: ollama/ollama:latest
hostname: ollama
ports:
- target: 11434
published: 11434
mode: overlay
deploy:
resources:
reservations:
generic_resources:
- discrete_resource_spec:
kind: "NVIDIA-GPU"
value: 0
replicas: 1
restart_policy:
condition: any
delay: 5s
max_attempts: 3
volumes:
- ./data/ollama:/root/.ollama-
Additional Requirements:
- Ensure CUDA is Enabled, follow your OS and GPU instructions for that.
- Enable Docker GPU support, see Nvidia Container Toolkit
- Follow the Guide here on configuring Docker Swarm to with with your GPU
- Ensure GPU Resource is enabled in
/etc/nvidia-container-runtime/config.tomland enable GPU resource advertising by uncommenting theswarm-resource = "DOCKER_RESOURCE_GPU". The docker daemon must be restarted after updating these files on each node.
-
-
With CPU Support:
Modify the Ollama Service within
docker-stack.yamland remove the lines forgeneric_resources:ollama:
image: ollama/ollama:latest
hostname: ollama
ports:
- target: 11434
published: 11434
mode: overlay
deploy:
replicas: 1
restart_policy:
condition: any
delay: 5s
max_attempts: 3
volumes:
- ./data/ollama:/root/.ollama -
Deploy Docker Stack:
docker stack deploy -c docker-stack.yaml -d super-awesome-ai
- Helm
- Kustomize
Helm Setup for Kubernetes
This page is currently under development.
Kustomize Setup for Kubernetes
This page is currently under development.
- Pinokio.computer
Pinokio.computer Installation
This page is currently under development.
Next Stepsβ
After installing, visit:
- http://localhost:3000 to access Manta.
- or http://localhost:8080/ when using a Python deployment.
You are now ready to start using Manta!
Using Manta with Ollamaβ
If you're using Manta with Ollama, be sure to check out our Starting with Ollama Guide to learn how to manage your Ollama instances with Manta.
Join the Communityβ
Need help? Have questions? Join our community:
Stay updated with the latest features, troubleshooting tips, and announcements!