Running an MCP Gateway in 30 Minutes

When an MCP client keeps its connection open after the first request, the difference from a basic LLM proxy becomes clear: sessions. A chat completions request is stateless. Send it, receive a response, and the connection ends. MCP keeps an SSE connection open for initialization, tool calls, and several roundtrips. The gateway must manage that persistent flow differently from an ordinary request-response proxy.

This guide gets an MCP gateway running in under 30 minutes with Zuplo. It assumes an MCP server is already running somewhere and that you want authentication and rate limiting in front of it without changing the server.

How MCP Traffic Reaches the Gateway

An MCP client connecting through a gateway produces this sequence:

1. An HTTP GET or POST to the MCP endpoint opens an SSE stream
2. An initialize JSON-RPC message where the client sends its capabilities
3. An initialized response from the server
4. Repeated tools/list and tools/call exchanges
5. At the end, the session terminates

The gateway checks authentication at step 1, as the connection opens. The session remains authenticated afterward. Rate limits can apply per session through concurrency, meaning open sessions per key, and through a per-session token budget, meaning the total tokens that session may consume.

Step 1: Create the Route

In zuplo.routes.json, add a route for the MCP server endpoint. Unlike a regular proxy route, it must enable SSE passthrough so streaming responses are not buffered by the gateway:

{
  "label": "MCP server proxy",
  "method": "GET",
  "path": "/mcp/:sessionId",
  "handler": {
    "export": "urlForwardHandler",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "baseUrl": "http://your-mcp-server.internal:8080",
      "forwardHeaders": true,
      "streaming": true
    }
  },
  "policies": {
    "inbound": []
  }
}

The streaming: true setting forwards SSE frames as they arrive instead of waiting for the complete response. Without it, streaming output can sit in a buffer until timeout, and long tool calls may take 30+ seconds.

Step 2: Add Authentication

MCP routes use authentication like other gateway routes. The API key policy checks the key before forwarding the request. Make it the first inbound policy:

"policies": {
  "inbound": [
    {
      "name": "require-api-key",
      "policyName": "api-key-inbound",
      "config": {
        "allowUnauthenticated": false
      }
    }
  ]
}

The key is checked when the initial connection opens. Since MCP sessions persist, each tool call inside that session is not checked again. A key revoked during the session therefore remains usable until the session ends and a new connection is made. That is acceptable for most cases. Immediate mid-session revocation requires session invalidation in the MCP server, not the gateway.

Step 3: Add Concurrency Limits

Concurrency is the key MCP rate limit because sessions remain open. A consumer with 50 simultaneous MCP sessions holds 50 persistent connections. Set a cap at the gateway:

{
  "name": "session-concurrency-limit",
  "policyName": "rate-limit-inbound",
  "config": {
    "rateLimitBy": "apiKey",
    "maxConcurrentConnections": 5,
    "rejectMessage": "Maximum concurrent MCP sessions reached for this API key"
  }
}

Five concurrent sessions per key is a reasonable starting point. Interactive developer tools commonly use 2 to 5. A server-to-server integration managing multiple sessions may need 10 to 20. Adjust the value to match what your MCP server can support per client.

Step 4: Add a Token Budget

An MCP session may make many tool calls, and each call can trigger LLM completions on the server. If the MCP server makes those calls and you pay for the tokens, set a token budget per session or key:

{
  "name": "daily-token-budget",
  "policyName": "token-budget-inbound",
  "config": {
    "rateLimitBy": "apiKey",
    "tokenBudget": 200000,
    "windowSeconds": 86400
  }
}

The limit covers every session for a key during the day. Reaching it does not close existing sessions. Instead, the gateway rejects new connections, while current sessions continue until completion or a natural disconnect. Long-running agent sessions are not cut off mid-task.

Session State and Gateway Limits

The gateway controls access and resource limits for MCP traffic, but it does not interpret MCP message contents. It cannot examine tool arguments or results, apply prompt-level rules, or act on which tools a session calls.

That boundary is deliberate. The gateway asks, "does this caller have permission to use this resource at this rate?" It does not ask, "is this tool call semantically valid?" Put content rules in the MCP server when needed, not in the gateway.

Verify the Setup

After deploying the config, test authentication and rate limits before using the real MCP client:

# Test with a valid key (should connect and receive SSE events)
curl -H "Authorization: Bearer zpl_live_abc123" \
  -H "Accept: text/event-stream" \
  https://your-gateway.gozuplo.org/mcp/test-session

# Test without a key (should return 401)
curl -H "Accept: text/event-stream" \
  https://your-gateway.gozuplo.org/mcp/test-session

# Test with a revoked key (should return 401)
curl -H "Authorization: Bearer zpl_live_revoked" \
  -H "Accept: text/event-stream" \
  https://your-gateway.gozuplo.org/mcp/test-session

On an auth failure, the SSE connection drops immediately instead of hanging. Confirm that the MCP client handles a 401 at connection time, since some clients built for naive MCP servers assume every connection succeeds.

Production Checks

With live MCP traffic, watch session duration and per-key session concurrency. Sessions that remain open indefinitely usually indicate a client or server bug. Set a maximum duration, with 10 to 15 minutes as a reasonable upper bound for interactive agents, then have the gateway close sessions that exceed it.

A concurrency spike from one key opening many sessions at once often indicates a retry loop. The client is failing to handle errors and starts new sessions instead of resuming. The concurrency limit contains the behavior, but the client holds the root cause. Include enough detail in the 429 response body for the developer to diagnose it without opening a support ticket.

A complete MCP gateway in Zuplo uses one routes.json file and three policy entries. The 30-minute estimate covers enough MCP spec reading to understand initialization and testing your server's SSE transport. The configuration itself is simple.