Designing a Secure Agentic Development Platform
While on parental leave from my job, I built Alpine Agentic, an AI application development platform. The concept of the platform is kind of a combination of vibe-coding sites like Lovable and working with high-touch software consultants. It combines the ease of setup of vibe-coding sites (none at all) with a more collaborative experience that is focused less on trying to one-shot a pretty web app and more on trying to understand the user and their business to build reliable, useful workloads that require minimal modifications.
The target ICP for this type of platform is businesses with the following properties:
- Large enough to have a sizeable number of ongoing operations and data
- Small enough to have a limited IT or software team and therefore lack resources to explore existing AI tooling like coding agents or vibe-coding sites
- Products and services offered are more physical than digital — potentially reducing the familiarity and utility of traditional SaaS business products
- Industry is niche enough where existing solutions might not solve every problem
But this blog post is not focused on the product experience (the marketing site should explain that). Instead, we'll talk about the architectural decisions and security considerations of designing the platform.
Bird's eye view
The platform is hosted on GCP. The primary tenant boundary is GCP projects within the Alpine Agentic organization which provides built-in resource and permissions isolation between tenant data planes.
The control plane is in a separate GCP project in the organization. The control plane service has permissions to administer resources in the tenant projects and contains the following primary infra:
control-plane-service: The brain and primary management system of the platform- Inference Gateway: Proxies all tenant agent inference traffic to LLM providers
- User portal: User-facing web app to chat with the technical account manager agent (Rocky) to plan workloads and manage their organization
The tenant data planes contains the following primary infrastructure:
- Workloads: The custom-built applications that augment the tenant's business operations and provide value
- Implementation runtimes: Runs the implementation engineer agent which receives workload specifications to build and test
I built the platform using cloud-native services like Cloud Run Services and Jobs for compute and Firestore for NoSQL databases. This allowed me to get the platform off the ground quickly rather than managing Kubernetes or maintaining persistent servers.
A shared control plane manages the platform and orchestrates implementation agents in isolated tenant projects. Each tenant has custom workloads and implementation runtimes. Agent inference goes through the shared inference gateway to LLM providers.
- Alpine Agentic — GCP organization: user-portal, control-plane-service, inference-gateway, Custom workloads, Implementation runtimes.
- Control plane — Dedicated GCP project · shared platform services: user-portal, control-plane-service, inference-gateway.
- Tenant data planes — One isolated GCP project per tenant: Custom workloads, Implementation runtimes.
- user-portal — The console where users chat with Rocky, the technical account manager agent, to plan workloads, administer tenant users, review plans and build status, and manage the organization.
- control-plane-service — The brain and primary management system of the platform. Handles authentication and agent orchestration and has permission to administer resources in tenant projects.
- inference-gateway — Proxies all tenant agent inference traffic to LLM providers.
- Custom workloads — Custom applications that augment the tenant's operations, isolated inside its own GCP project.
- Implementation runtimes — Runs the implementation engineer agent, which receives specifications from Rocky to build and test. Agent inference goes through the shared inference gateway.
- LLM providers — External LLM providers receive tenant agent inference requests through the inference gateway.
- user-portal → control-plane-service
- control-plane-service → Implementation runtimes: Administer & orchestrate. The control plane service administers tenant resources and orchestrates agents across the project boundary. The dashed path represents control plane responsibility.
- Implementation runtimes → inference-gateway: Agent inference. Tenant agent inference requests return to the shared inference gateway before reaching model providers.
- inference-gateway → LLM providers: LLM API
Securing public-facing surfaces
A global Application Load Balancer serves as the public entry point for the user portal and tenant applications. It terminates HTTPS and performs hostname routing to tenant applications, supporting custom domains or platform managed subdomains.
The control-plane-service and inference gateway are internal services. A Shared VPC connects the control-plane with implementation and workload runtimes. Internal service calls are routed through the Shared VPC and Private Google Access. Any caller must also possess the necessary invocation permissions. Each service’s caller-policy.json declares which callers may access which API paths, and Terraform uses that policy to build path-scoped roles/run.invoker IAM grants for service-to-service calls.
Internet traffic reaches the portal and tenant applications through a global external Application Load Balancer and separate backends. The control plane, inference gateway, and implementation runtimes have no public ingress. Dashed green connections are private, IAM-authenticated service calls over the Shared VPC and Private Google Access.
- Public ingress — HTTPS termination & hostname routing: Global external ALB, Portal backend, Tenant backend.
- Shared VPC — Private Google Access: User portal, Control plane service, Inference gateway, Tenant applications, Implementation runtimes.
- Control plane — GCP project: User portal, Control plane service, Inference gateway.
- Tenant — GCP project: Tenant applications, Implementation runtimes.
- Internet
- Global external ALB — The global external Application Load Balancer terminates HTTPS and routes portal, platform subdomain, and custom-domain hostnames to their backends.
- Portal backend
- Tenant backend
- User portal — The internet-facing Cloud Run portal accepts requests through its load-balancer backend and calls the control plane privately.
- Control plane service — Internal-only Cloud Run service. There is no load-balancer or internet route to this service; callers need private network access and invocation authorization.
- Inference gateway — Internal-only Cloud Run inference proxy, reachable through private networking by authorized service callers.
- Tenant applications — Public Cloud Run applications receive traffic through the tenant backend, including their platform subdomains or custom domains.
- Implementation runtimes — Private implementation services and jobs. They have Shared VPC connectivity and do not receive public ingress.
- Internet → Global external ALB: HTTPS
- Global external ALB → Portal backend: Portal hostname
- Global external ALB → Tenant backend: Workload hostname
- Portal backend → User portal: Public ingress
- Tenant backend → Tenant applications: Public ingress
- User portal → Control plane service: IAM: portal API. The portal uses its service identity for the private control-plane invocation. Network reachability does not grant API access by itself.
- Implementation runtimes → Inference gateway: IAM: agent inference. Implementation runtimes call the inference gateway privately, using authorized service identities. This dashed path is private service traffic, not public ingress.
Authentication
The platform uses Google Cloud Identity Platform to maanage authentication for tenant and workload users. It's cheap and provides Google-grade credential storage, account lifecycle emails, a login SDK, and MFA.
Sign-in happens through the user portal, which acts as a backend for frontend (BFF). The client sends credentials directly to Google using the Identity Platform browser SDK. After authentication, the browser presents the resulting identity token to the portal, which exchanges it through the control plane for a cookie. The portal forwards the cookie with backend requests to the control plane. The portal itself has no direct datastore access or identity-administration permissions.
Steps 1–7 establish a portal session. The browser authenticates directly with Identity Platform, then submits its ID token to the portal. The control plane verifies the token and creates the session. Steps 8–11 validate subsequent requests. The user's token or cookie is distinct from the portal's own service identity, which authenticates every portal-to-control-plane call.
- Establish a portal session — User credential: ID token · Service caller: portal identity: Identity Platform, Browser, User portal / BFF, Control plane.
- Subsequent requests — User credential: portal cookie · Service caller: portal identity: Browser, User portal / BFF, Control plane.
- Control-plane state — Portal sessions: Session store.
- Identity Platform — Google Identity Platform receives credentials directly from the browser SDK and returns the user's identity token.
- Browser
- User portal / BFF — Receives the identity token and delegates session creation to the control plane. It has no direct session datastore or identity-administration access.
- Control plane — Verifies the Identity Platform token and creates an application session. The portal authenticates this private call with its own service identity.
- Session store — The control plane owns session state. Only the control plane is connected to this store in the sign-in and authorization flows.
- Browser
- User portal / BFF — Forwards the user's session credential while authenticating to the control plane as the portal service, not as the user.
- Control plane — Validates the session and determines the user's tenant authorization before allowing the portal request.
- Browser → Identity Platform: 1. Sign in
- Identity Platform → Browser: 2. ID token
- Browser → User portal / BFF: 3. ID token
- User portal / BFF → Control plane: 4. Token + caller ID. One private request carries the user's ID token and authenticates the caller with the portal's service identity. The control plane verifies both the caller and the user token.
- Control plane → Session store: 5. Create session
- Control plane → User portal / BFF: 6. Session created
- User portal / BFF → Browser: 7. Portal cookie
- Browser → User portal / BFF: 8. Portal cookie
- User portal / BFF → Control plane: 9. Cookie + caller ID. The portal forwards the user's session cookie while authenticating the invocation with its own service identity. These are separate credentials with separate purposes.
- Control plane → Session store: 10. Check session
- Control plane → User portal / BFF: 11. Tenant access
Tenant applications use the same sign-in experience. When an application requires authentication, it redirects the browser to the portal. After sign-in or reuse of an existing session, the platform returns a single-use grant tied to that workload. The workload exchanges the grant through its authenticated control-plane connection and establishes its own session cookie.
Authentication and authorization is separate in workloads. Identity Platform establishes who the person is. The control plane determines whether that person is a member of the Alpine Agentic tenant and what portal roles they have. Each workload enforces its own application RBAC, and the policy is defined by the implementation agent based on business requirements enumerated by Rocky.
The tenant application redirects an unauthenticated browser to the portal. The portal reuses its own session or completes sign-in with Identity Platform. The control plane issues a short-lived, single-use grant bound to the target workload. The browser returns to that workload's callback; the workload redeems the grant using its own service identity, establishes its own session cookie, and enforces application membership and roles. Portal and workload cookies remain separate, including on custom domains.
- Reuse the portal session or sign in — Portal cookie stays on the portal domain: Browser, Tenant application, Identity Platform, User portal, Control plane.
- Establish the workload session — Separate cookie on the workload or custom domain: Browser, Tenant application, Control plane, Membership & roles.
- Browser
- Tenant application — An unauthenticated request redirects the browser to the user portal, identifying the target workload.
- Identity Platform — Only needed if no existing portal session can be reused. Credentials go directly from the browser to Google, following the portal login flow above.
- User portal — Reuses the portal-domain session cookie, or completes the portal login flow. Requests a workload-bound grant from the control plane.
- Control plane — Authorizes the portal session and issues a short-lived, single-use grant tied to the requested workload.
- Browser
- Tenant application — Its callback receives the grant and redeems it through an authenticated private connection to the control plane, using the workload's own service identity.
- Control plane — Validates both the calling workload's identity and the grant's target, expiry, and single-use status before returning a workload-scoped session credential.
- Membership & roles — Each workload enforces its own application membership and roles, according to the policy implemented for its business requirements. A valid identity or portal session alone does not grant application access.
- Browser → Tenant application: 1. Open workload
- Tenant application → Browser: 2. Portal redirect
- Browser → User portal: 3. Use portal session
- Browser → Identity Platform: 4. Sign in if needed. Optional browser-to-Identity-Platform sign-in. When a portal session already exists, this step is skipped; otherwise the portal authentication flow establishes that session.
- User portal → Control plane: 5. Request grant. The portal authenticates to the control plane as its own service identity and presents the user's session and target workload.
- Control plane → User portal: 6. Single-use grant. A short-lived, single-use grant is bound to the target workload; it is not a shared browser cookie.
- User portal → Browser: 7. Return with grant. The browser follows the redirect to the target workload's callback, including when it is hosted on a custom domain.
- Browser → Tenant application: 8. Callback grant
- Tenant application → Control plane: 9. Grant + caller ID. This private call is authenticated with the workload's service identity, separate from the user's single-use grant.
- Control plane → Tenant application: 10. App session
- Tenant application → Browser: 11. App cookie. The workload sets its own browser session cookie for its own domain. Portal, workload, and custom domains do not share cookies.
- Tenant application → Membership & roles: 12. Check app roles
Workload implementation
This is what makes the platform "agentic". As previously mentioned, there are two main agents:
- Rocky: Chats with users to understand their processes and what kind of applications can help support their business.
- Implementation engineer: Runs in the tenant project, writes and tests the workload code.
Planning workload implementation
Rocky is a custom harness with tooling to work with the user and draft product requirement documents (PRDs). It uses OpenAI and Gemini native function calling which is less error-prone than directly parsing tool call markup. GPT 6 Luna has been my go-to for Rocky — it is an extremely cheap model with sufficient intelligence for planning small to medium sized workloads. The agent loop lives in the control-plane-service and user messages in the portal trigger a run. Once planning is complete, Rocky requests an implementation job.
If the PRD revision is different from the last deployment, a new implementation request is created. GCP Eventarc detects a new "build request" object in the control plane firestore database and requests to start a new implementation run. If a run for that workload is already in progress, Eventarc will send staggered retries until the lock is cleared and a new run can begin.
User messages reach Rocky through the portal. Rocky's agent loop lives inside the control-plane service, which also admits implementation runs. Workload PRDs and build requests are stored together in the control-plane Firestore database. If a PRD differs from the last deployment, the service creates a build request. Eventarc detects the request and calls the same service to start a run. A per-workload lock admits one implementation job at a time; busy workloads receive staggered retries until the lock clears.
- Control plane — GCP project · planning & run admission: User portal, Control plane service · Rocky, Control plane Firestore, Eventarc.
- Tenant project — Isolated implementation: Implementation job.
- User
- User portal
- Control plane service · Rocky — Rocky's planning agent loop runs inside this service. It drafts and refines PRDs, writes a build request when the plan differs from the last deployment, and admits at most one implementation run per workload using a run lock.
- Control plane Firestore — The same control-plane Firestore database stores workload PRDs and build request objects. A new build request is created only when the PRD differs from the plan used for the last deployment.
- Eventarc — Detects the new Firestore build request and requests an implementation run. A busy workload causes staggered retries until its run lock is cleared.
- Implementation job — The implementation engineer starts in a Cloud Run Job in the tenant project, using the planned workload requirements.
- User → User portal: Plan workload
- User portal → Control plane service · Rocky: User message
- Control plane service · Rocky → Control plane Firestore: PRDs & build requests. Draft and refine the workload PRD in Firestore. Create a new build request in that database only if the PRD differs from the last deployment.
- Control plane Firestore → Eventarc: New build request
- Eventarc → Control plane service · Rocky: Start run
- Control plane service · Rocky → Eventarc: Busy → retry. A held workload lock prevents admission. Eventarc retries on a staggered schedule until the active run finishes and the lock clears.
- Control plane service · Rocky → Implementation job: Lock available
The implementation runtime
The implementation engineer runs in a Cloud Run Job and uses Pi as a harness. I like Pi's simplicity (only bash and file editing tools by default) and customizability as a headless coding agent more than something heavier like Opencode. Model calls are routed through the inference gateway which is the only service with read access to the platform-owned model provider keys, stored in GCP Secrets Manager. Currently it supports any model on Gemini, OpenAI, xAI, Fireworks, and Together AI. For the implementation engineer, I'm primarily using models in the middle of the Intelligence / Cost pareto line. Usually, a chinese open weight model is the most economical option, but lately OpenAI's GPT 6.1 Sol has had impressive efficiency.
Each workload has a test database and GCS folder, allowing the agent to test functionality without altering production data. The implementation filesystem is seeded with a template that contains adapters for authentication and data stores, allowing the engineer to focus mainly on implementing workload business logic. Once the code passes validation, the agent loop exits, and the runtime wrapper commits the source and saves a Git bundle and result artifact in the tenant’s delivery bucket. The result object creation notifies the control plane through Pub/Sub. The control plane starts Cloud Build in the tenant project, which builds the app image, publishes to artifact registry, and deploys to the workload service.
A tenant Cloud Run Job runs the Pi implementation agent in a workspace with the template code seeded from a bucket in the control-plane project, using isolated test data and model calls through the shared inference gateway. After validation passes, the job commits the source and saves both a Git bundle and a result artifact in the tenant delivery bucket. The result object notifies the control plane through Pub/Sub. The control plane starts tenant Cloud Build, which uses the source bundle to build and publish the app image, then deploy it to the workload service.
- Control plane — Shared GCP project: Workload template, Inference gateway, Control plane service.
- Tenant project — Implementation, test data & delivery: Implementation job, Test database, Delivery bucket, Cloud Build, Artifact Registry, Workload service, Test storage bucket.
- Workload template — A bucket in the control-plane project stores the workload template. It seeds the implementation filesystem with adapters for authentication and data stores, allowing the agent to focus on the workload's business logic.
- Implementation job — A Cloud Run Job runs the implementation engineer using the Pi harness and Bash, Read, Edit, and Write tools. After validation passes and the agent loop exits, the job commits the source and saves a Git bundle and result artifact in the delivery bucket.
- Test database — The workload's dedicated test database allows functional testing without production database access.
- Delivery bucket — Holds both the Git bundle containing committed workload source and the result artifact. The result object notifies the control plane through Pub/Sub; Cloud Build uses the source bundle.
- Cloud Build — Started by the control plane in the tenant project. Builds the application image from the delivered source, publishes it to Artifact Registry, and deploys the workload service.
- Artifact Registry — Stores the workload's application image built and published by Cloud Build.
- Workload service — Cloud Build deploys the published application image to the tenant's Cloud Run workload service.
- Inference gateway — Proxies the implementation engineer's model calls to the configured providers. The implementation job does not receive provider keys.
- Control plane service — Receives implementation results through Pub/Sub and starts Cloud Build in the tenant project.
- Test storage bucket — The workload's dedicated GCS test folder allows storage testing without production storage access.
- Workload template → Implementation job: Seed workspace
- Implementation job → Test database: Test workload. Uses the dedicated test database without accessing production data.
- Implementation job → Inference gateway: Model calls
- Implementation job → Delivery bucket: Bundle & result. After validation succeeds, the implementation job commits the source and writes both the Git bundle and result artifact to the tenant delivery bucket.
- Delivery bucket → Control plane service: Result via Pub/Sub. The result object stored in the delivery bucket notifies the control plane through Pub/Sub.
- Control plane service → Cloud Build: Start build
- Delivery bucket → Cloud Build: Source bundle
- Cloud Build → Artifact Registry: Build & push
- Artifact Registry → Workload service: App image. The deployed workload uses the application image published to Artifact Registry.
- Cloud Build → Workload service: Deploy service. Cloud Build deploys the published image to the tenant's Cloud Run workload service.
- Implementation job → Test storage bucket: Test storage
Prompt injection / Arbitrary code execution
Obviously, the platform directly feeds user input into agent instructions which forces the platform threat model to assume that the implementation and workload runtimes are completely compromised. I talked about some of the risks that RCE-as-a-service platforms like this have in my BSides Las Vegas talk. If the tenant environment is assumed compromised there are a few risks that must be addressed:
| Category | Concerns | Mitigations | Assumed risks |
|---|---|---|---|
| VPC access | - Can the compromised service reach any other private/internal services? - Are those endpoints secure? |
Path-based least privilege IAM policies for every service | Potential vulnerabilities in control-plane-service or inference gateway exploitable from a tenant project |
| Cloud privilege escalation | Does the workload or implementation service account have unnecessary privileges that can be abused or exploited to gain additional access? | Limited to: -Read/write permissions on the workload's data stores - Invoker on chat completions and authentication endpoints |
Right now there are few, but I can see things getting more risky if workloads eventually need to interact |
| The runtime OS | - Is there any data or processes running on the same machine that the user should not have access to? - Are there any secrets on the file system that can be enumerated? |
- Dedicated implementation service for each workload - Credentials used in upstream gateways |
Primarily abuse related — threat actors can create applications to use for malicious activity |
| Resource abuse | Can a user create a flood of data that overloads the system (and costs money)? | - Consistent monitoring of workload storage usage - Potentially a "Data broker" service in the tenant project which proxies writes and enforces limits |
- Monitoring cannot stop runaway writes before they occur - Exploits/bypasses in the data broker |
Integrating with external systems
A critical component of the platform is allowing workloads to integrate with the customer's existing systems of record, such as productivity tooling, ERPs, CRMs, project management suites, etc. A Connector Gateway service in each tenant project enables this. Workloads call schema-defined operations and the gateway constructs the external request. This keeps user-provided secrets out of implementation and application code.
The shared control-plane GCP project configures connections and grants in an isolated tenant GCP project. Using a tenant connection-manager identity, the control-plane service can create secrets and enroll credentials but cannot read them. At runtime, the tenant's Cloud Run Connector Gateway checks workload identity, connection grants, and allowed operations; exchanges named operations with workloads; reads credentials and stores rotated tokens in Secret Manager; uses tenant Firestore for connection metadata, grants, and audit; and calls external providers for API operations and OAuth token exchange. Provider credentials are never supplied to workloads. Green dashed paths are configuration; blue solid paths are runtime exchanges.
- Shared control-plane — GCP project: User Portal, Control-Plane Service.
- Isolated tenant — GCP project: Workload, Connector Gateway, Secret Manager, Tenant Firestore.
- User Portal
- Control-Plane Service — Manages tenant connections and workload grants. Secret enrollment uses the tenant connection-manager identity, which can create secrets but cannot read their contents.
- Workload — Exchanges authenticated, named operations with the tenant Connector Gateway without receiving provider credentials.
- Connector Gateway — Checks the workload identity, connection grants, and allowed operations before constructing provider requests. Reads credentials and stores rotated OAuth tokens within the tenant project.
- Secret Manager — The control plane enrolls credentials using a tenant connection-manager identity that cannot read secrets. The Connector Gateway reads credentials and stores rotated OAuth tokens.
- Tenant Firestore — Stores tenant connection metadata, workload grants, and sanitized audit records.
- External Providers — External systems of record reached through the gateway's provider API calls and OAuth token exchange. These services are outside both GCP project boundaries.
- User Portal → Control-Plane Service: Configure API key or OAuth.
- Control-Plane Service → Connector Gateway: Manage connections and grants.
- Control-Plane Service → Secret Manager: Create secret / enroll credentials.
- Workload ↔ Connector Gateway: Authenticated, named operations.
- Connector Gateway ↔ Secret Manager: Read credentials / store rotated tokens.
- Connector Gateway ↔ Tenant Firestore: Connection metadata, update and review grants, audit.
- Connector Gateway ↔ External Providers: Provider API calls / OAuth token exchange.
Discovery agent
To effectively plan workloads that integrate with external systems, Rocky needs to answer questions that users often times cannot provide, such as technical questions about the schema of the system APIs. To address this, I implemented a "Discovery" subagent that Rocky can task with internet research, integration read operations, and inspection of workload test applications.
The Discovery harness runs in a dedicated Cloud Run service inside the tenant’s project. The agent returns its findings and results to Rocky, who uses them to inform workload planning. Internet searches goes through an authenticated control-plane proxy for Tavily, a web search results provider.
The shared control-plane service contains Rocky, Discovery orchestration, and the Tavily proxy. Its inference gateway is a separate service. Discovery orchestration dispatches tasks to the Discovery harness in the tenant Cloud Run service. The harness saves progress in Discovery Workspace Storage. Its agent explores a local workload instance and reads external data through the Connector Gateway. The local workload instance has temporary scoped access to workload test datastores. Model calls go to the inference gateway, while search goes through the control-plane Tavily proxy to the external Tavily API. On completion, the control-plane service validates stored findings and resumes Rocky.
- Shared control plane — GCP project: Inference gateway, Rocky & Discovery orchestration, Tavily proxy.
- Tenant project — Isolated GCP project: Discovery Workspace Storage, Connector Gateway, Workload test datastores, Discovery agent, Local workload instance.
- Control-Plane Service: Rocky & Discovery orchestration, Tavily proxy.
- Discovery harness — Cloud Run service: Discovery agent, Local workload instance.
- Rocky & Discovery orchestration — Both Rocky and Discovery orchestration live inside the control-plane service. The service dispatches Discovery tasks, validates the stored findings on completion, and resumes Rocky's planning and user communication.
- Inference gateway — A separate shared service handles Discovery's model calls; it is outside the control-plane service container.
- Tavily proxy — An authenticated proxy inside the control-plane service calls the external Tavily API for Discovery's internet research.
- Discovery agent — The tenant-local Discovery agent gathers evidence and stores findings for validation by the control-plane service.
- Local workload instance — A local workload instance inside the Discovery harness uses temporary, scoped access to workload test datastores.
- Discovery Workspace Storage — Private tenant storage holds Discovery inputs, supporting evidence, and checkpoints.
- Connector Gateway — Discovery can use approved integration read operations only when the connection and access have been configured.
- Workload test datastores — Workload test datastores are accessed by the local workload instance with temporary, scoped access for the Discovery task.
- Tavily — The external Tavily API is outside both GCP project boundaries and is accessed through the control-plane proxy.
- Rocky & Discovery orchestration → Discovery harness: Dispatch task
- Discovery agent → Rocky & Discovery orchestration: Report findings and resume Rocky. The completion path returns to Discovery orchestration inside the control-plane service, which validates the stored findings before resuming Rocky.
- Discovery agent → Inference gateway: Model calls
- Discovery agent → Tavily proxy: Authenticated search
- Tavily proxy → Tavily: Search API
- Discovery harness ↔ Discovery Workspace Storage: save progress. Reads private task inputs and checkpoints, and saves findings, supporting evidence, and progress in the same tenant bucket.
- Discovery agent → Connector Gateway: external data reads
- Discovery agent → Local workload instance: explore workload
- Local workload instance → Workload test datastores
Development in the age of LLMs
Obviously the majority of the platform code is AI-generated — but I don't think I would call my development strategy "vibe-coding". My definition of vibe-coding is literally having zero understanding of the underlying code or architecture and just continuously requesting more user-facing changes from a bot. My methodology is essentially:
- Plan a new component of the system as best I can. Use AI to research options, fill in my own knowledge gaps, and then articulate the plan.
- Generate structured requirements to be implemented as standalone "work units" within the component sprint.
- Task an agent to implement the first work unit.
- Review the diff: review core logic, documentation, and repo structure.
- Test manually and deliberately.
- Request refinements, review, test again, then move on to the next work unit.
By far, the most time-consuming part of the development cycle was planning and testing. The time it takes for AI to write the code itself is almost negligible - which is incredible considering if I had to hand-jam everything it would probably take several years to build the same platform (and become proficient in multiple coding languages). I found that if I rushed planning I would usually end up regretting it and have to go back and start from scratch. The agents almost alway over-engineer hacky solutions to compensate for gaps in planning.
Admin interface
I also built an admin portal that used my admin permissions to interact directly with the control plane service and database (I still need to add a dedicated operator role and make admin access break-glass only 😬). Building the admin portal was trivial using AI but gives me any type of visibility I want into platform operations. While building out the platform I added the following utilities into the admin-site:
- Tenant metadata, storage, user management
- Rocky message, compaction, and context lab
- Workload infrastructure tracking
- Implementation agent testing and session tracing
- System-wide configuration like primary/alternate models
CI/CD
I did all of my testing in a dev environment and manually deployed changes to it. For deploying changes to prod, GitHub Actions workflows run as service accounts with only permissions to publish and update service images. I have not built a pipeline to apply Terraform plans when new infrastructure changes are required, such as creating a new service, changing resource configurations, etc. That pipeline would require really strict security as infra changes require near admin privileges. I similarly need to create new tenant projects manually (there is no self-sign up in the platform for now).
Lessons learned
Building a platform like this was a good experience. I built agents from scratch and refined their prompts, tool schemas, harnesses, and experimented with different models. But the most enjoyable part was architecting the overall system. Obviously AI can quickly generate and refine front-ends, API contracts, and core logic (if well planned), but managing the tradeoffs of the overall infrastructure, such as cost, security, scalability, and maintenance, is not something that should be delegated to the model.
I know for a fact "vibe-coding" parts of the platform would have resulted in security issues. When I was initially planning out the inference gateway, I wanted to add time and cost-bound sessions. For example, the implementation engineer had one hour and $10 of LLM spend to finish building a workload. This was before I added a monthly usage cap per tenant.
GPT 5.5 created a plan for the inference gateway. The control-plane-service would mint sessions on an /internal endpoint. And implementation runtimes send LLM traffic to the chat completions endpoint, which is forwarded to the providers. However, the model forgot to add service-based authorization for those endpoints. So an agent that regularly processes untrusted instructions could have been able to mint as many inference sessions as it wanted with unlimited time and budget.
In the vulnerable proposal, only the control-plane service should mint inference sessions through the gateway's /internal endpoint. Missing service authorization also lets the tenant implementation agent, following untrusted user instructions, mint sessions with arbitrary expiry and cost limits, then use them for provider-backed model calls. Green shows the intended minting path; the dashed amber path shows the bypass. This illustrates the planning flaw described above.
- Tenant project: Untrusted agent.
- Inference gateway — Shared service: /internal, Chat completions.
- Control service — The control-plane service is the intended caller of the internal session-minting endpoint. It sets the implementation run's authorized duration and cost limit.
- Untrusted agent — The tenant implementation agent follows untrusted user instructions. In the flawed proposal it could call the internal minting endpoint itself and obtain sessions with self-selected time and cost limits.
- /internal — The proposed session-minting endpoint lacked service-based authorization. It would accept the implementation agent as a caller instead of restricting minting to the control-plane service.
- Chat completions — The agent uses its self-minted session for model calls, bypassing the time and cost limits that the control-plane service intended to impose.
- LLM providers — The inference gateway proxies model calls to upstream providers.
- Control service → /internal: Mint session
- Untrusted agent → /internal: Missing authorization. Any limits. The implementation agent can request and receive sessions with arbitrary time and cost limits because the internal endpoint does not check the calling service's authority to mint sessions.
- Untrusted agent → Chat completions: Uncapped calls. The agent submits chat-completions requests using the session it minted for itself.
- Chat completions → LLM providers: Model calls
Just one of many examples of why you don't trust agents with decisions that need to be made by someone you can fire or sue.