Skip to main content

Langfuse Integration with MANTA

Langfuse (GitHub) offers open source observability and evaluations for MANTA. By enabling the Langfuse integration, you can trace your application data with Langfuse to develop, monitor, and improve the use of MANTA, including:

  • Application traces
  • Usage patterns
  • Cost data by user and model
  • Replay sessions to debug issues
  • Evaluations

How to integrate Langfuse with MANTA​

Pipelines in MANTA is an UI-agnostic framework for OpenAI API plugins. It enables the injection of plugins that intercept, process, and forward user prompts to the final LLM, allowing for enhanced control and customization of prompt handling.

To trace your application data with Langfuse, you can use the Langfuse pipeline, which enables real-time monitoring and analysis of message interactions.

Quick Start Guide​

Step 1: Setup MANTA​

Make sure to have MANTA running. To do so, have a look at the MANTA documentation.

Step 2: Set Up Pipelines​

Launch Pipelines by using Docker. Use the following command to start Pipelines:

docker run -p 9099:9099 --add-host=host.docker.internal:host-gateway -v pipelines:/app/pipelines --name pipelines --restart always ghcr.io/open-webui/pipelines:main

Step 3: Connecting MANTA with Pipelines​

In the Admin Settings, create and save a new connection of type OpenAI API with the following details:

Step 4: Adding the Langfuse Filter Pipeline​

Next, navigate to Admin Settings -> Pipelines and add the Langfuse Filter Pipeline. Specify that Pipelines is listening on http://host.docker.internal:9099 (as configured earlier) and install the Langfuse Filter Pipeline by using the Install from Github URL option with the following URL:

https://github.com/open-webui/pipelines/blob/main/examples/filters/langfuse_filter_pipeline.py

Now, add your Langfuse API keys below. If you haven't signed up to Langfuse yet, you can get your API keys by creating an account here.

Note: Capture usage (token counts) for OpenAi models while streaming is enabled, you have to navigate to the model settings in MANTA and check the "Usage" box below Capabilities.

Step 5: See your traces in Langfuse​

You can now interact with your MANTA application and see the traces in Langfuse.

Example trace in the Langfuse UI: