Langfuse’s November 13 updates added Google Vertex AI support and restrictions on which models individual AI Gateway keys can call. Together, the changes offer a useful opportunity to separate experimental access from production access, without assuming that every connected model should be available to every application.
This guide outlines a rollout plan, not a hands-on performance review. The gateway remains a public beta. Start with a noncritical application and test the specific route it will use before changing a production service.
Write the access plan before changing keys
Create a short inventory with one row per application: its owner, environment, approved provider connection, model, expected request format and rollback contact. Use separate entries for a developer’s local tool and a shared service, even if both currently call the same model.
Langfuse’s per-key restriction announcement says administrators can narrow model access when creating a gateway key or edit the restrictions afterward. Use that capability to express the inventory, rather than issuing a broad credential first and hoping usage monitoring catches an unwanted route later.
Define what the application must not do as well as what it needs. A prototype might be approved for one connection but not for a connection carrying production credentials. A model access rule cannot replace a decision about which information the application is allowed to transmit.
Check the Vertex route, not just the provider name
The separate Langfuse Vertex AI announcement limits support to models available through Vertex AI’s OpenAI-compatible Chat Completions endpoint. It names Gemini and hosted open models, while explicitly saying Claude on Vertex AI is not yet available through this integration.
Do not turn a successful Gemini test into an assumption that every Vertex-hosted model works through the same path. Record the actual requested model and API format in the rollout checklist. An endpoint compatibility failure and a permission failure need different fixes.
Use synthetic, non-sensitive input for the initial request. Check that the response comes back through the intended connection and that an error is visible to the application. Keep the old route available until the new one passes the workflow’s normal tests.
Make model names unambiguous
Langfuse’s models documentation uses connection-qualified identifiers in the form connection:model. It also says key patterns only narrow the models administrators have enabled; they cannot enable a disabled model. A key without patterns can reach every enabled model, including additions enabled later.
Prefer an explicit permitted set for a service with a narrow job. Broad wildcard rules can be convenient during exploration, but their future meaning may be wider than their current effect. Have the owner review how a new connection or model would interact with the rule before approving it.
Run two small tests: one against an approved model and one against a deliberately excluded model. The second should fail in a way the caller handles cleanly. Save the result without exposing the credential. A successful permitted request alone does not prove the restriction is working.
Choose ownership that matches the workload
Langfuse’s authentication documentation distinguishes personal user keys from organization-owned service account keys. A user’s key stops when its owner loses gateway access; service account keys are not tied to membership changes. The documentation identifies shared services and continuous integration as service-account use cases.
Give each shared integration a named operational owner, even when the credential belongs to the organization. Specify who rotates it, who receives failure alerts and how the application retrieves its replacement. An owner field in a spreadsheet is useful only if someone remains responsible for it.
For temporary trials, agree an end date and a cleanup task before enabling access. Keep development credentials out of source code and avoid copying secrets into debugging transcripts. Test reports should show which configuration was exercised, not the authentication material used to exercise it.
Plan failures and later changes
The models documentation says a request targets one connection and the gateway does not automatically retry it through another connection when that target fails. Decide whether the application should pause, show an error or use an explicitly approved alternative. Do not silently broaden access to make a test pass.
Before rollout, review the results with the application owner: permitted calls work, excluded calls fail, ownership is correct and rollback is documented. Repeat those checks after connection renames, model changes or credential rotation. The useful outcome is a known route with bounded access, not simply a green response from an API.



