A team building a multi-tenant LLM product needs its gateway to fit the rest of its engineering workflow. Before Zuplo, I spent years on platform infrastructure and API platforms, and kept seeing the same issue: the gateway sat outside normal development. It was managed through a UI, proprietary CLI, or SDK that wrote hidden control-plane state. Everything else lived in version control. The gateway did not.
The Dashboard Problem
A web UI makes gateway setup look simple at first, but several problems become costly as a team grows.
First, the configuration is hidden from anyone who is not signed into the dashboard. A new engineer cannot inspect the repository to learn how the API is secured. They must log in, move through several screens, and reconstruct the current state. A YAML file or JSON config in the repo is different. It takes five minutes to read. The UI can take a day, and details may still be missed.
Second, dashboard edits skip code review. Anyone with access can change a rate limit, turn off an auth policy, or create a route without another engineer seeing it. In practice, "just click and save" is a common path to outages. Not through bad intent, but because the edit seemed minor and a PR felt like too much friction.
Third, reproducing the setup is unreliable. Rebuilding the gateway or creating a staging environment that matches production means manually copying settings or depending on the vendor's export feature. Some vendors provide one. Many do not. Even a "decent export" seldom means a format that can deploy without a person involved.
What Config-as-Code Actually Means
By config-as-code, I mean that files in your repository describe the complete gateway state. Not a base file with dashboard overrides. Complete state. Delete the control plane, redeploy from the repo, and the behavior is the same.
In Zuplo, the primary config file is zuplo.routes.json. It contains every route, each attached policy, and every policy option:
{
"routes": [
{
"path": "/api/chat",
"methods": ["POST"],
"handler": "upstream-http",
"upstream": "https://api.openai.com/v1/chat/completions",
"policies": [
{
"name": "api-key-auth",
"handler": "api-key-auth-inbound"
},
{
"name": "token-budget",
"handler": "token-budget-inbound",
"options": {
"key": "api-key",
"budget_tokens": 100000,
"window": "1d"
}
}
]
}
]
}The file lives in git. Pull requests expose each change, and the diff shows what deployment will do. A rollback is a revert followed by a redeploy. You need neither special tooling nor a dashboard login.
The Code Review Problem It Solves
The often-missed benefit of config-as-code is that gateway edits follow the same review path as application code.
Changing a rate limit from 100 requests per minute to 1000 is an important operational decision. A dashboard click gets no review, no question about the reason, and no check that the upstream can handle the extra load.
A pull request gives a teammate room to ask, "why are we raising this?" That question can prevent an incident. The PR is not bureaucracy. It is the engineering discipline infrastructure decisions need.
We see this with teams moving from dashboard-configured gateways. During the first few weeks, engineers open PRs such as "raise rate limit to 1000 because we need it". Reviewers ask questions, some edits improve, and a few stop because the question exposes a misunderstanding. Review catches considerations the original engineer missed.
Environment Parity as a Side Effect
Putting gateway config in a repository makes environment parity manageable. Staging has a routes.json, and production has a routes.json. Diff them to see exactly where staging differs from production and why.
For teams running LLM applications, this helps testing. Staging should have rate limits close enough to production for load tests to mean something. With a dashboard, staging and production drift because "someone adjusted it for testing and forgot to sync the change back". With config-as-code, a mismatch exists because someone deliberately made the environments different, and the repository shows it.
What We Had to Get Right
Config-as-code is not automatically easy to implement well. We learned several parts through experience.
Policy options need sensible defaults. Requiring all 15 options makes a config file intimidating and invites mistakes. The common case should need as little configuration as possible. Advanced options can remain available without being mandatory.
Validate the config schema at deploy time, not when requests arrive. A bad policy should fail deployment rather than quietly wait for production traffic to find it. Our deployment pipeline validates the schema and policy options before traffic moves over.
Local tooling matters. If developers cannot test gateway config locally before deployment, the workflow pushes them back toward dashboard-driven tools. We invested early in a local dev server that reads the same routes.json and runs the same policy engine as production.
Not a Universal Answer
Config-as-code is not right for every team. A solo developer putting an API key before one endpoint, with no second person ever editing the config, may find a git workflow adds friction without much return. For that case, a dashboard is quicker to set up, and that is fine.
Its value grows with team size and organizational complexity. Two engineers sharing a codebase begin to notice the benefit. Four engineers across service teams feel it clearly. A platform team managing gateway config for multiple application teams cannot operate without it.
Building Zuplo this way was a bet that developer tooling would treat infrastructure config like application code. That bet still feels right. Teams with the most confidence in their API infrastructure can answer configuration questions by inspecting the repository. That is what we set out to make possible.