Invoke Prompt (Proxy LLM Calls)

Agenta provides a proxy service that allows you to invoke your deployed prompts while automatically capturing metrics, traces, and logs. This gives you full observability into your LLM application's performance, costs, and behavior in production.

Accessing the Proxy Endpoint

You can find the specific endpoint information to invoke your deployed prompt directly within the Agenta UI:

1. Navigate to "Deployments"
2. Click on your deployed environment
3. Look for the "API" button in the top-right corner

Making a Proxy Call

You can invoke your deployed prompt using REST API.

Here's how to call your deployed prompt using Python with the requests library:

import os
import requests
import json

# Endpoint for the proxy service
url = "https://cloud.agenta.ai/services/completion/run"

# Parameters specifying the environment, application, and input variables
params = {
  "environment": "development",  # The environment (production, development, staging)
  "app": "classification",       # Your application name
  "inputs": {
    "country": "France"          # Input variables required by your prompt
  }
}

# Required headers including your API key
headers = {
    "Content-Type": "application/json",    
    "Authorization": f"ApiKey {os.environ['AGENTA_API_KEY']}",
}

# Make the request
response = requests.post(url, json=params, headers=headers)

# Parse and display the response
data = response.json()
print(json.dumps(data, indent=4))

Understanding the Response

The proxy service returns a structured response with the following fields:

{
    "version": "3.0",                                     // SDK version
    "data": "The capital of France is Paris.",            // LLM output
    "content_type": "text/plain",                         // Output format (text/plain or application/json)
    "tree_id": "0ef1d6b7-84c3-4b8a-705b-ae5974e51954"    // Tree ID for observability
}

The tree_id can be used to locate this specific request in the observability dashboard.

Viewing Traces in the Observability Dashboard

All proxy calls are automatically traced and can be viewed in the observability dashboard:

The dashboard provides comprehensive insights into your LLM application's behavior, including:

• Complete prompt and response content
• Input and output token counts
• Request duration
• Model parameters used
• Associated costs

Tracking sessions

For chat applications, you can group multiple requests into sessions using the Baggage header. This helps you track complete conversations across multiple requests.

import requests
import os

session_id = "chat_session_123"
user_id = "user-456"

response = requests.post(
    "https://cloud.agenta.ai/services/completion/run",
    headers={
        "Content-Type": "application/json",
        "Authorization": f"ApiKey {os.environ['AGENTA_API_KEY']}",
        "Baggage": f"ag.session.id={session_id},ag.user.id={user_id}"
    },
    json={
        "environment": "development",
        "app": "chatbot",
        "inputs": {
            "message": "Hello"
        }
    }
)

All requests with the same session ID will be grouped together in the observability dashboard. You can also track users to see who is interacting with your application.