---
title: Designing a Secure Agentic Development Platform
slug: agentic-platform
date: 2026-10-23
description: Reviewing the technical architecture and decisions of the Alpine
Agentic platform
cover: /agentic-platform/cover_image.png
coverAlt: Alpine Agentic cover image
url: https://benkofman.com/agentic-platform.html
---
While on parental leave from my job, I built [Alpine Agentic](https://alpineagentic.com), an AI application development platform. The concept of the platform is kind of a combination of vibe-coding sites like Lovable and working with high-touch software consultants. It combines the ease of setup of vibe-coding sites (none at all) with a more collaborative experience that is focused less on trying to one-shot a pretty web app and more on trying to understand the user and their business to build reliable, useful workloads that require minimal modifications.
The target ICP for this type of platform is businesses with the following properties:
* Large enough to have a sizeable number of ongoing operations and data
* Small enough to have a limited IT or software team and therefore lack resources to explore existing AI tooling like coding agents or vibe-coding sites
* Products and services offered are more physical than digital — potentially reducing the familiarity and utility of traditional SaaS business products
* Industry is niche enough where existing solutions might not solve every problem
But this blog post is not focused on the product experience (the marketing site should explain that). Instead, we'll talk about the architectural decisions and security considerations of designing the platform.
## Bird's eye view
The platform is hosted on GCP. The primary tenant boundary is [GCP projects](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects?hl=en) within the Alpine Agentic organization which provides built-in resource and permissions isolation between tenant data planes.
The control plane is in a separate GCP project in the organization. The control plane service has permissions to administer resources in the tenant projects and contains the following primary infra:
* `control-plane-service`: The brain and primary management system of the platform
* Inference Gateway: Proxies all tenant agent inference traffic to LLM providers
* User portal: User-facing web app to chat with the technical account manager agent (Rocky) to plan workloads and manage their organization
The tenant data planes contains the following primary infrastructure:
* Workloads: The custom-built applications that augment the tenant's business operations and provide value
* Implementation runtimes: Runs the implementation engineer agent which receives workload specifications to build and test
I built the platform using cloud-native services like [Cloud Run Services](https://docs.cloud.google.com/run/docs/deployment-options-for-services?hl=en) and Jobs for compute and Firestore for NoSQL databases. This allowed me to get the platform off the ground quickly rather than managing Kubernetes or maintaining persistent servers.
```architecture
version: 2
title: Platform high-level overview
description: A shared control plane manages the platform and orchestrates implementation agents in isolated tenant projects. Each tenant has custom workloads and implementation runtimes. Agent inference goes through the shared inference gateway to LLM providers.
grid: { nodeWidth: 180 }
sections:
organization:
label: Alpine Agentic
subtitle: GCP organization
details: The organization contains the control plane project and a separate data plane project for each tenant.
control-plane:
label: Control plane
subtitle: Dedicated GCP project · shared platform services
in: organization
tone: sage
details: A separate project contains the platform's primary management service, inference gateway, and user portal. The control plane service handles authentication and agent orchestration and can administer resources in tenant projects.
tenant-projects:
label: Tenant data planes
subtitle: One isolated GCP project per tenant
in: organization
tone: blue
repeat: true
details: Each tenant project contains custom workloads and implementation runtimes. Project boundaries isolate tenant resources and permissions. This shows one representative tenant, repeated independently for every tenant.
nodes:
portal:
label: user-portal
at: [0, 0]
in: control-plane
tone: sage
details: The console where users chat with Rocky, the technical account manager agent, to plan workloads, administer tenant users, review plans and build status, and manage the organization.
control:
label: control-plane-service
at: [1, 0]
width: 224
in: control-plane
tone: sage
details: The brain and primary management system of the platform. Handles authentication and agent orchestration and has permission to administer resources in tenant projects.
inference:
label: inference-gateway
at: [2, 0]
in: control-plane
tone: sage
details: Proxies all tenant agent inference traffic to LLM providers.
workloads:
label: Custom workloads
at: [0.3, 1.7]
in: tenant-projects
tone: blue
details: Custom applications that augment the tenant's operations, isolated inside its own GCP project.
runtime:
label: Implementation runtimes
at: [1, 1.7]
width: 224
in: tenant-projects
tone: blue
details: Runs the implementation engineer agent, which receives specifications from Rocky to build and test. Agent inference goes through the shared inference gateway.
models:
label: LLM providers
shape: external
at: [3.3, 0]
tone: amber
details: External LLM providers receive tenant agent inference requests through the inference gateway.
edges:
- from: portal
to: control
tone: sage
- from: control.bottom
to: runtime.top
label: Administer & orchestrate
style: dashed
tone: sage
details: The control plane service administers tenant resources and orchestrates agents across the project boundary. The dashed path represents control plane responsibility.
- from: runtime.top
fromOffset: 0.8
to: inference.bottom
toOffset: 0.3
bend: 0.35
label: Agent inference
tone: amber
details: Tenant agent inference requests return to the shared inference gateway before reaching model providers.
- from: inference
to: models
label: LLM API
tone: amber
```
## Securing public-facing surfaces
A global Application Load Balancer serves as the public entry point for the user portal and tenant applications. It terminates HTTPS and performs hostname routing to tenant applications, supporting custom domains or platform managed subdomains.
The `control-plane-service` and inference gateway are internal services. A Shared VPC connects the control-plane with implementation and workload runtimes. Internal service calls are routed through the Shared VPC and [Private Google Access](https://docs.cloud.google.com/vpc/docs/private-google-access). Any caller must also possess the necessary invocation permissions. Each service’s `caller-policy.json` declares which callers may access which API paths, and Terraform uses that policy to build path-scoped `roles/run.invoker` IAM grants for service-to-service calls.
```architecture
version: 2
title: Public ingress and private service calls
description: Internet traffic reaches the portal and tenant applications through a global external Application Load Balancer and separate backends. The control plane, inference gateway, and implementation runtimes have no public ingress. Dashed green connections are private, IAM-authenticated service calls over the Shared VPC and Private Google Access.
grid: { column: 300, row: 150, nodeWidth: 200 }
sections:
public-ingress:
label: Public ingress
subtitle: HTTPS termination & hostname routing
tone: amber
private-network:
label: Shared VPC
subtitle: Private Google Access
tone: sage
details: Private service traffic remains on Google's network. Network access alone is insufficient; callers also need invocation permissions and must satisfy each service's caller policy.
control-project:
label: Control plane
subtitle: GCP project
labelAlign: right
in: private-network
tone: sage
tenant-project:
label: Tenant
subtitle: GCP project
labelAlign: right
in: private-network
tone: blue
nodes:
internet:
label: Internet
shape: external
at: [1.2, 0]
width: 224
tone: amber
load-balancer:
label: Global external ALB
at: [1.2, 1.1]
width: 224
in: public-ingress
tone: amber
details: The global external Application Load Balancer terminates HTTPS and routes portal, platform subdomain, and custom-domain hostnames to their backends.
portal-backend:
label: Portal backend
at: [0, 2.3]
in: public-ingress
tone: amber
tenant-backend:
label: Tenant backend
at: [2.4, 2.3]
in: public-ingress
tone: amber
portal:
label: User portal
at: [0, 4.1]
in: control-project
tone: sage
details: The internet-facing Cloud Run portal accepts requests through its load-balancer backend and calls the control plane privately.
control:
label: Control plane service
at: [0, 5.5]
width: 224
in: control-project
tone: sage
details: Internal-only Cloud Run service. There is no load-balancer or internet route to this service; callers need private network access and invocation authorization.
inference:
label: Inference gateway
at: [1, 5.5]
in: control-project
tone: sage
details: Internal-only Cloud Run inference proxy, reachable through private networking by authorized service callers.
applications:
label: Tenant applications
at: [2.4, 4.1]
in: tenant-project
tone: blue
details: Public Cloud Run applications receive traffic through the tenant backend, including their platform subdomains or custom domains.
implementation:
label: Implementation runtimes
at: [2.4, 5.5]
width: 224
in: tenant-project
tone: blue
details: Private implementation services and jobs. They have Shared VPC connectivity and do not receive public ingress.
edges:
- from: internet.bottom
to: load-balancer.top
label: HTTPS
tone: amber
- from: load-balancer.bottom
fromOffset: 0.2
to: portal-backend.top
label: Portal hostname
tone: amber
- from: load-balancer.bottom
fromOffset: 0.8
to: tenant-backend.top
label: Workload hostname
tone: amber
- from: portal-backend.bottom
fromOffset: 0.75
to: portal.top
toOffset: 0.75
label: Public ingress
tone: amber
- from: tenant-backend.bottom
fromOffset: 0.75
to: applications.top
toOffset: 0.75
label: Public ingress
tone: amber
- from: portal.bottom
fromOffset: 0.35
to: control.top
toOffset: 0.3125
label: "IAM: portal API"
style: dashed
tone: sage
details: The portal uses its service identity for the private control-plane invocation. Network reachability does not grant API access by itself.
- from: implementation.left
to: inference.right
label: "IAM: agent inference"
style: dashed
tone: sage
details: Implementation runtimes call the inference gateway privately, using authorized service identities. This dashed path is private service traffic, not public ingress.
```
### Authentication
The platform uses Google Cloud [Identity Platform](https://docs.cloud.google.com/identity-platform/docs/multi-tenancy) to maanage authentication for tenant and workload users. It's cheap and provides Google-grade credential storage, account lifecycle emails, a login SDK, and MFA.
Sign-in happens through the user portal, which acts as a backend for frontend (BFF). The client sends credentials directly to Google using the Identity Platform browser SDK. After authentication, the browser presents the resulting identity token to the portal, which exchanges it through the control plane for a cookie. The portal forwards the cookie with backend requests to the control plane. The portal itself has no direct datastore access or identity-administration permissions.
```architecture
version: 4
title: Portal sign-in and session authorization
description: Steps 1–7 establish a portal session. The browser authenticates directly with Identity Platform, then submits its ID token to the portal. The control plane verifies the token and creates the session. Steps 8–11 validate subsequent requests. The user's token or cookie is distinct from the portal's own service identity, which authenticates every portal-to-control-plane call.
grid: { column: 320, row: 200, nodeWidth: 180 }
sections:
sign-in:
label: Establish a portal session
subtitle: "User credential: ID token · Service caller: portal identity"
tone: sage
requests:
label: Subsequent requests
subtitle: "User credential: portal cookie · Service caller: portal identity"
tone: blue
session-state:
label: Control-plane state
subtitle: Portal sessions
tone: sage
nodes:
identity:
label: Identity Platform
shape: external
at: [ 0, 0 ]
in: sign-in
tone: amber
details: Google Identity Platform receives credentials directly from the browser SDK and returns the user's identity token.
browser:
label: Browser
at: [ 0, 1.2 ]
in: sign-in
tone: blue
shape: browser
portal:
label: User portal / BFF
at: [ 1, 1.2 ]
in: sign-in
tone: sage
details: Receives the identity token and delegates session creation to the control plane. It has no direct session datastore or identity-administration access.
control:
label: Control plane
at: [ 2, 1.2 ]
in: sign-in
tone: sage
details: Verifies the Identity Platform token and creates an application session. The portal authenticates this private call with its own service identity.
store:
label: Session store
shape: database
at: [ 3.2, 1.8 ]
in: session-state
tone: sage
details: The control plane owns session state. Only the control plane is connected to this store in the sign-in and authorization flows.
request-browser:
label: Browser
at: [ 0, 3 ]
in: requests
tone: blue
shape: browser
request-portal:
label: User portal / BFF
at: [ 1, 3 ]
in: requests
tone: sage
details: Forwards the user's session credential while authenticating to the control plane as the portal service, not as the user.
request-control:
label: Control plane
at: [ 2, 3 ]
in: requests
tone: sage
details: Validates the session and determines the user's tenant authorization before allowing the portal request.
edges:
- from: browser.top
fromOffset: 0.2
to: identity.bottom
toOffset: 0.2
label: 1. Sign in
tone: amber
- from: identity.bottom
fromOffset: 0.8
to: browser.top
toOffset: 0.8
label: 2. ID token
tone: amber
- from: browser.right
fromOffset: 0.25
to: portal.left
toOffset: 0.25
label: 3. ID token
tone: blue
- from: portal.right
fromOffset: 0.25
to: control.left
toOffset: 0.25
label: 4. Token + caller ID
tone: sage
details: One private request carries the user's ID token and authenticates the caller with the portal's service identity. The control plane verifies both the caller and the user token.
- from: control.right
fromOffset: 0.25
to: store.left
toOffset: 0.25
label: 5. Create session
tone: sage
- from: control.left
fromOffset: 0.8
to: portal.right
toOffset: 0.8
label: 6. Session created
tone: sage
- from: portal.left
fromOffset: 0.8
to: browser.right
toOffset: 0.8
label: 7. Portal cookie
tone: blue
- from: request-browser.right
fromOffset: 0.25
to: request-portal.left
toOffset: 0.25
label: 8. Portal cookie
tone: blue
- from: request-portal.right
fromOffset: 0.25
to: request-control.left
toOffset: 0.25
label: 9. Cookie + caller ID
tone: sage
details: The portal forwards the user's session cookie while authenticating the invocation with its own service identity. These are separate credentials with separate purposes.
- from: request-control.right
fromOffset: 0.25
to: store.left
toOffset: 0.75
label: 10. Check session
tone: sage
- from: request-control.left
fromOffset: 0.8
to: request-portal.right
toOffset: 0.8
label: 11. Tenant access
tone: sage
```
Tenant applications use the same sign-in experience. When an application requires authentication, it redirects the browser to the portal. After sign-in or reuse of an existing session, the platform returns a single-use grant tied to that workload. The workload exchanges the grant through its authenticated control-plane connection and establishes its own session cookie.
Authentication and authorization is separate in workloads. Identity Platform establishes who the person is. The control plane determines whether that person is a member of the Alpine Agentic tenant and what portal roles they have. Each workload enforces its own application RBAC, and the policy is defined by the implementation agent based on business requirements enumerated by Rocky.
```architecture
version: 4
title: Workload single sign-on
description: The tenant application redirects an unauthenticated browser to the portal. The portal reuses its own session or completes sign-in with Identity Platform. The control plane issues a short-lived, single-use grant bound to the target workload. The browser returns to that workload's callback; the workload redeems the grant using its own service identity, establishes its own session cookie, and enforces application membership and roles. Portal and workload cookies remain separate, including on custom domains.
grid: { column: 320, row: 200, nodeWidth: 200 }
sections:
portal-session:
label: Reuse the portal session or sign in
subtitle: Portal cookie stays on the portal domain
tone: sage
workload-session:
label: Establish the workload session
labelAlign: right
subtitle: Separate cookie on the workload or custom domain
tone: blue
nodes:
browser:
label: Browser
at: [ 0, 0 ]
in: portal-session
tone: blue
shape: browser
entry-workload:
label: Tenant application
at: [ 1, 0 ]
in: portal-session
tone: blue
details: An unauthenticated request redirects the browser to the user portal, identifying the target workload.
identity:
label: Identity Platform
shape: external
at: [ 0, 1.2 ]
in: portal-session
tone: amber
details: Only needed if no existing portal session can be reused. Credentials go directly from the browser to Google, following the portal login flow above.
portal:
label: User portal
at: [ 1, 1.2 ]
in: portal-session
tone: sage
details: Reuses the portal-domain session cookie, or completes the portal login flow. Requests a workload-bound grant from the control plane.
issuer:
label: Control plane
at: [ 2, 1.2 ]
in: portal-session
tone: sage
details: Authorizes the portal session and issues a short-lived, single-use grant tied to the requested workload.
callback-browser:
label: Browser
at: [ 0, 2.9 ]
in: workload-session
tone: blue
shape: browser
workload:
label: Tenant application
at: [ 1, 2.9 ]
in: workload-session
tone: blue
details: Its callback receives the grant and redeems it through an authenticated private connection to the control plane, using the workload's own service identity.
redeemer:
label: Control plane
at: [ 2, 2.9 ]
in: workload-session
tone: sage
details: Validates both the calling workload's identity and the grant's target, expiry, and single-use status before returning a workload-scoped session credential.
permissions:
label: Membership & roles
at: [ 1, 4.1 ]
in: workload-session
tone: blue
details: Each workload enforces its own application membership and roles, according to the policy implemented for its business requirements. A valid identity or portal session alone does not grant application access.
edges:
- from: browser.right
fromOffset: 0.25
to: entry-workload.left
toOffset: 0.25
label: 1. Open workload
tone: blue
- from: entry-workload.left
fromOffset: 0.8
to: browser.right
toOffset: 0.8
label: 2. Portal redirect
tone: blue
- from: browser.bottom
fromOffset: 0.8
to: portal.top
label: 3. Use portal session
tone: blue
- from: browser.bottom
fromOffset: 0.2
to: identity.top
toOffset: 0.2
label: 4. Sign in if needed
style: dashed
tone: amber
details: Optional browser-to-Identity-Platform sign-in. When a portal session already exists, this step is skipped; otherwise the portal authentication flow establishes that session.
- from: portal.right
fromOffset: 0.25
to: issuer.left
toOffset: 0.25
label: 5. Request grant
tone: sage
details: The portal authenticates to the control plane as its own service identity and presents the user's session and target workload.
- from: issuer.left
fromOffset: 0.8
to: portal.right
toOffset: 0.8
label: 6. Single-use grant
tone: sage
details: A short-lived, single-use grant is bound to the target workload; it is not a shared browser cookie.
- from: portal.bottom
fromOffset: 0.25
to: callback-browser.top
toOffset: 0.75
label: 7. Return with grant
tone: blue
details: The browser follows the redirect to the target workload's callback, including when it is hosted on a custom domain.
- from: callback-browser.right
fromOffset: 0.25
to: workload.left
toOffset: 0.25
label: 8. Callback grant
tone: blue
- from: workload.right
fromOffset: 0.25
to: redeemer.left
toOffset: 0.25
label: 9. Grant + caller ID
tone: sage
details: This private call is authenticated with the workload's service identity, separate from the user's single-use grant.
- from: redeemer.left
fromOffset: 0.8
to: workload.right
toOffset: 0.8
label: 10. App session
tone: sage
- from: workload.left
fromOffset: 0.8
to: callback-browser.right
toOffset: 0.8
label: 11. App cookie
tone: blue
details: The workload sets its own browser session cookie for its own domain. Portal, workload, and custom domains do not share cookies.
- from: workload.bottom
to: permissions.top
label: 12. Check app roles
tone: blue
```
## Workload implementation
This is what makes the platform "agentic". As previously mentioned, there are two main agents:
* Rocky: Chats with users to understand their processes and what kind of applications can help support their business.
* Implementation engineer: Runs in the tenant project, writes and tests the workload code.
### Planning workload implementation
Rocky is a custom harness with tooling to work with the user and draft product requirement documents (PRDs). It uses OpenAI and Gemini native function calling which is less error-prone than directly parsing tool call markup. GPT 6 Luna has been my go-to for Rocky — it is an extremely cheap model with sufficient intelligence for planning small to medium sized workloads. The agent loop lives in the `control-plane-service` and user messages in the portal trigger a run. Once planning is complete, Rocky requests an implementation job.
If the PRD revision is different from the last deployment, a new implementation request is created. GCP Eventarc detects a new "build request" object in the control plane firestore database and requests to start a new implementation run. If a run for that workload is already in progress, Eventarc will send staggered retries until the lock is cleared and a new run can begin.
```architecture
version: 4
title: Rocky planning and implementation request cycle
description: User messages reach Rocky through the portal. Rocky's agent loop lives inside the control-plane service, which also admits implementation runs. Workload PRDs and build requests are stored together in the control-plane Firestore database. If a PRD differs from the last deployment, the service creates a build request. Eventarc detects the request and calls the same service to start a run. A per-workload lock admits one implementation job at a time; busy workloads receive staggered retries until the lock clears.
grid: { column: 320, row: 180, nodeWidth: 200 }
sections:
control-project:
label: Control plane
subtitle: GCP project · planning & run admission
labelAlign: right
tone: sage
tenant-project:
label: Tenant project
subtitle: Isolated implementation
labelAlign: right
tone: blue
nodes:
user:
label: User
at: [ 0, -1.2 ]
tone: blue
portal:
label: User portal
at: [ 0, 0 ]
in: control-project
tone: sage
control:
label: Control plane service · Rocky
width: 272
at: [ 1, 0 ]
in: control-project
tone: sage
details: Rocky's planning agent loop runs inside this service. It drafts and refines PRDs, writes a build request when the plan differs from the last deployment, and admits at most one implementation run per workload using a run lock.
firestore:
label: Control plane Firestore
shape: database
width: 240
at: [ 2.6, 0 ]
in: control-project
tone: sage
details: The same control-plane Firestore database stores workload PRDs and build request objects. A new build request is created only when the PRD differs from the plan used for the last deployment.
eventarc:
label: Eventarc
width: 240
at: [ 2.6, 1.6 ]
in: control-project
tone: amber
details: Detects the new Firestore build request and requests an implementation run. A busy workload causes staggered retries until its run lock is cleared.
implementation:
label: Implementation job
width: 272
at: [ 1, 3.2 ]
in: tenant-project
tone: blue
details: The implementation engineer starts in a Cloud Run Job in the tenant project, using the planned workload requirements.
edges:
- from: user.bottom
to: portal.top
label: Plan workload
tone: blue
- from: portal.right
to: control.left
label: User message
tone: sage
- from: control.right
to: firestore.left
label: PRDs & build requests
tone: sage
details: Draft and refine the workload PRD in Firestore. Create a new build request in that database only if the PRD differs from the last deployment.
- from: firestore.bottom
to: eventarc.top
label: New build request
tone: amber
- from: eventarc.left
fromOffset: 0.25
to: control.bottom
toOffset: 0.8
label: Start run
tone: amber
- from: control.bottom
fromOffset: 0.6
to: eventarc.left
toOffset: 0.8
label: Busy → retry
style: dashed
tone: amber
details: A held workload lock prevents admission. Eventarc retries on a staggered schedule until the active run finishes and the lock clears.
- from: control.bottom
fromOffset: 0.2
to: implementation.top
toOffset: 0.2
label: Lock available
tone: blue
```
### The implementation runtime
The implementation engineer runs in a Cloud Run Job and uses [Pi](https://github.com/earendil-works/pi) as a harness. I like Pi's simplicity (only bash and file editing tools by default) and customizability as a headless coding agent more than something heavier like Opencode. Model calls are routed through the inference gateway which is the only service with read access to the platform-owned model provider keys, stored in GCP Secrets Manager. Currently it supports any model on Gemini, OpenAI, xAI, Fireworks, and Together AI. For the implementation engineer, I'm primarily using models in the middle of the [Intelligence / Cost pareto line](https://artificialanalysis.ai/models/#intelligence-per-task-tabs). Usually, a chinese open weight model is the most economical option, but lately OpenAI's GPT 6.1 Sol has had impressive efficiency.
Each workload has a test database and GCS folder, allowing the agent to test functionality without altering production data. The implementation filesystem is seeded with a template that contains adapters for authentication and data stores, allowing the engineer to focus mainly on implementing workload business logic. Once the code passes validation, the agent loop exits, and the runtime wrapper commits the source and saves a Git bundle and result artifact in the tenant’s delivery bucket. The result object creation notifies the control plane through Pub/Sub. The control plane starts Cloud Build in the tenant project, which builds the app image, publishes to artifact registry, and deploys to the workload service.
```architecture
version: 4
title: Implementing, testing, and deploying workloads
description: A tenant Cloud Run Job runs the Pi implementation agent in a workspace with the template code seeded from a bucket in the control-plane project, using isolated test data and model calls through the shared inference gateway. After validation passes, the job commits the source and saves both a Git bundle and a result artifact in the tenant delivery bucket. The result object notifies the control plane through Pub/Sub. The control plane starts tenant Cloud Build, which uses the source bundle to build and publish the app image, then deploy it to the workload service.
grid: { column: 300, row: 180, nodeWidth: 200 }
sections:
control-project:
label: Control plane
subtitle: Shared GCP project
tone: sage
tenant-project:
label: Tenant project
subtitle: Implementation, test data & delivery
tone: blue
nodes:
template:
label: Workload template
shape: bucket
width: 175
at: [ 0.05, 0 ]
in: control-project
tone: sage
details: A bucket in the control-plane project stores the workload template. It seeds the implementation filesystem with adapters for authentication and data stores, allowing the agent to focus on the workload's business logic.
agent:
label: Implementation job
at: [ 1.7, 0 ]
in: tenant-project
tone: blue
details: A Cloud Run Job runs the implementation engineer using the Pi harness and Bash, Read, Edit, and Write tools. After validation passes and the agent loop exits, the job commits the source and saves a Git bundle and result artifact in the delivery bucket.
test-data:
label: Test database
shape: database
at: [ 2.9, 0 ]
in: tenant-project
tone: blue
details: The workload's dedicated test database allows functional testing without production database access.
delivery:
label: Delivery bucket
shape: bucket
at: [ 1.7, 1.6 ]
in: tenant-project
tone: blue
details: Holds both the Git bundle containing committed workload source and the result artifact. The result object notifies the control plane through Pub/Sub; Cloud Build uses the source bundle.
build:
label: Cloud Build
at: [ 1.7, 3.1 ]
in: tenant-project
tone: blue
details: Started by the control plane in the tenant project. Builds the application image from the delivered source, publishes it to Artifact Registry, and deploys the workload service.
registry:
label: Artifact Registry
shape: service
at: [ 2.9, 3.1 ]
in: tenant-project
tone: blue
details: Stores the workload's application image built and published by Cloud Build.
workload:
label: Workload service
at: [ 2.9, 4.5 ]
in: tenant-project
tone: blue
details: Cloud Build deploys the published application image to the tenant's Cloud Run workload service.
inference:
label: Inference gateway
width: 224
at: [ 0, 1.2 ]
in: control-project
tone: sage
details: Proxies the implementation engineer's model calls to the configured providers. The implementation job does not receive provider keys.
control:
label: Control plane service
width: 224
at: [ 0, 2.4 ]
in: control-project
tone: sage
details: Receives implementation results through Pub/Sub and starts Cloud Build in the tenant project.
test-bucket:
label: Test storage bucket
shape: bucket
at: [ 2.9, 1.25 ]
in: tenant-project
tone: blue
details: The workload's dedicated GCS test folder allows storage testing without production storage access.
edges:
- from: template.right
to: agent.left
label: Seed workspace
tone: sage
- from: agent.right
to: test-data.left
label: Test workload
tone: blue
details: Uses the dedicated test database without accessing production data.
fromOffset: 0.25
toOffset: 0.25
- from: agent.bottom
fromOffset: 0.2
to: inference.right
label: Model calls
tone: sage
- from: agent.bottom
fromOffset: 0.7
to: delivery.top
toOffset: 0.7
label: Bundle & result
tone: blue
details: After validation succeeds, the implementation job commits the source and writes both the Git bundle and result artifact to the tenant delivery bucket.
- from: delivery.left
fromOffset: 0.25
to: control.right
toOffset: 0.25
label: Result via Pub/Sub
tone: amber
details: The result object stored in the delivery bucket notifies the control plane through Pub/Sub.
- from: control.right
fromOffset: 0.8
to: build.left
toOffset: 0.25
label: Start build
tone: sage
- from: delivery.bottom
to: build.top
label: Source bundle
tone: blue
- from: build.right
to: registry.left
label: Build & push
tone: blue
- from: registry.bottom
to: workload.top
label: App image
tone: blue
details: The deployed workload uses the application image published to Artifact Registry.
- from: build.bottom
to: workload.left
label: Deploy service
tone: blue
details: Cloud Build deploys the published image to the tenant's Cloud Run workload service.
- from: agent.right
fromOffset: 0.8
to: test-bucket.left
label: Test storage
tone: blue
```
### Prompt injection / Arbitrary code execution
Obviously, the platform directly feeds user input into agent instructions which forces the platform threat model to assume that the implementation and workload runtimes are completely compromised. I talked about some of the risks that RCE-as-a-service platforms like this have in my BSides Las Vegas [talk](https://www.youtube.com/watch?v=peo0GXhMreM). If the tenant environment is assumed compromised there are a few risks that must be addressed:
| Category {width=200px} | Concerns {width=400px} | Mitigations {width=400px} | Assumed risks {width=400px} |
| -------- | ------- | ------- | ------- |
| VPC access | - Can the compromised service reach any other private/internal services?
- Are those endpoints secure? | Path-based least privilege IAM policies for every service | Potential vulnerabilities in control-plane-service or inference gateway exploitable from a tenant project |
| Cloud privilege escalation | Does the workload or implementation service account have unnecessary privileges that can be abused or exploited to gain additional access? | Limited to:
-Read/write permissions on the workload's data stores
- Invoker on chat completions and authentication endpoints | Right now there are few, but I can see things getting more risky if workloads eventually need to interact |
| The runtime OS | - Is there any data or processes running on the same machine that the user should not have access to?
- Are there any secrets on the file system that can be enumerated? | - Dedicated implementation service for each workload
- Credentials used in upstream gateways | Primarily abuse related — threat actors can create applications to use for malicious activity |
| Resource abuse | Can a user create a flood of data that overloads the system (and costs money)? | - Consistent monitoring of workload storage usage
- Potentially a "Data broker" service in the tenant project which proxies writes and enforces limits | - Monitoring cannot stop runaway writes before they occur
- Exploits/bypasses in the data broker |
{.wide}
## Integrating with external systems
A critical component of the platform is allowing workloads to integrate with the customer's existing systems of record, such as productivity tooling, ERPs, CRMs, project management suites, etc. A Connector Gateway service in each tenant project enables this. Workloads call schema-defined operations and the gateway constructs the external request. This keeps user-provided secrets out of implementation and application code.
```architecture
version: 4
title: Workload Integrations using the Connector Gateway
description: The shared control-plane GCP project configures connections and grants in an isolated tenant GCP project. Using a tenant connection-manager identity, the control-plane service can create secrets and enroll credentials but cannot read them. At runtime, the tenant's Cloud Run Connector Gateway checks workload identity, connection grants, and allowed operations; exchanges named operations with workloads; reads credentials and stores rotated tokens in Secret Manager; uses tenant Firestore for connection metadata, grants, and audit; and calls external providers for API operations and OAuth token exchange. Provider credentials are never supplied to workloads. Green dashed paths are configuration; blue solid paths are runtime exchanges.
grid: { column: 280, row: 160, nodeWidth: 224 }
sections:
control-project:
label: Shared control-plane
subtitle: GCP project
tone: sage
tenant-project:
label: Isolated tenant
subtitle: GCP project
tone: blue
nodes:
portal:
label: User Portal
at: [ 0, 0 ]
in: control-project
tone: sage
control:
label: Control-Plane Service
at: [ 0, 1.6 ]
in: control-project
tone: sage
details: Manages tenant connections and workload grants. Secret enrollment uses the tenant connection-manager identity, which can create secrets but cannot read their contents.
workload:
label: Workload
width: 288
at: [ 1.6, 0 ]
in: tenant-project
tone: blue
details: Exchanges authenticated, named operations with the tenant Connector Gateway without receiving provider credentials.
gateway:
label: Connector Gateway
width: 288
at: [ 1.6, 1.6 ]
in: tenant-project
tone: blue
details: Checks the workload identity, connection grants, and allowed operations before constructing provider requests. Reads credentials and stores rotated OAuth tokens within the tenant project.
secrets:
label: Secret Manager
shape: vault
width: 200
at: [ 3, 0 ]
in: tenant-project
tone: blue
details: The control plane enrolls credentials using a tenant connection-manager identity that cannot read secrets. The Connector Gateway reads credentials and stores rotated OAuth tokens.
metadata:
label: Tenant Firestore
shape: database
width: 204
at: [ 1.75, 2.9 ]
in: tenant-project
tone: blue
details: Stores tenant connection metadata, workload grants, and sanitized audit records.
providers:
label: External Providers
shape: external
at: [ 3.9, 1.6 ]
tone: amber
details: External systems of record reached through the gateway's provider API calls and OAuth token exchange. These services are outside both GCP project boundaries.
edges:
# Configuration: green, dashed, one-way.
- from: portal.bottom
to: control.top
label: Configure API key or OAuth.
labelMaxWidth: 190
style: dashed
tone: sage
- from: control.right
to: gateway.left
label: Manage connections and grants.
labelAt: [ 1.2, 1.8 ]
labelMaxWidth: 140
style: dashed
tone: sage
- from: control.right
fromOffset: 0.1
to: secrets.top
via: [ [ 1.2, -0.75 ] ]
label: Create secret / enroll credentials.
labelAt: [ 1.2, 1 ]
labelMaxWidth: 180
style: dashed
tone: sage
# Runtime: blue, solid, bidirectional.
- from: workload.bottom
to: gateway.top
direction: both
label: Authenticated, named operations.
labelMaxWidth: 180
tone: blue
- from: gateway.right
fromOffset: 0.1
to: secrets.left
direction: both
label: Read credentials / store rotated tokens.
labelMaxWidth: 160
tone: blue
- from: gateway.bottom
to: metadata.top
direction: both
label: Connection metadata, update and review grants, audit.
labelMaxWidth: 160
tone: blue
- from: gateway.right
to: providers.left
direction: both
label: Provider API calls / OAuth token exchange.
labelMaxWidth: 200
tone: blue
```
### Discovery agent
To effectively plan workloads that integrate with external systems, Rocky needs to answer questions that users often times cannot provide, such as technical questions about the schema of the system APIs. To address this, I implemented a "Discovery" subagent that Rocky can task with internet research, integration read operations, and inspection of workload test applications.
The Discovery harness runs in a dedicated Cloud Run service inside the tenant’s project. The agent returns its findings and results to Rocky, who uses them to inform workload planning. Internet searches goes through an authenticated control-plane proxy for [Tavily](https://www.tavily.com), a web search results provider.
```architecture
version: 5
title: Discovery agent architecture
description: The shared control-plane service contains Rocky, Discovery orchestration, and the Tavily proxy. Its inference gateway is a separate service. Discovery orchestration dispatches tasks to the Discovery harness in the tenant Cloud Run service. The harness saves progress in Discovery Workspace Storage. Its agent explores a local workload instance and reads external data through the Connector Gateway. The local workload instance has temporary scoped access to workload test datastores. Model calls go to the inference gateway, while search goes through the control-plane Tavily proxy to the external Tavily API. On completion, the control-plane service validates stored findings and resumes Rocky.
grid: { column: 260, row: 140, nodeWidth: 200 }
sections:
control-project:
label: Shared control plane
subtitle: GCP project
tone: sage
control-service:
label: Control-Plane Service
in: control-project
tone: sage
tenant-project:
label: Tenant project
subtitle: Isolated GCP project
tone: blue
discovery-runtime:
label: Discovery harness
subtitle: Cloud Run service
in: tenant-project
tone: blue
nodes:
orchestration:
label: Rocky & Discovery orchestration
width: 280
at: [ 0, 0 ]
in: control-service
tone: sage
details: Both Rocky and Discovery orchestration live inside the control-plane service. The service dispatches Discovery tasks, validates the stored findings on completion, and resumes Rocky's planning and user communication.
inference:
label: Inference gateway
at: [ 0.25, 2.7 ]
in: control-project
tone: sage
details: A separate shared service handles Discovery's model calls; it is outside the control-plane service container.
width: 200
tavily-proxy:
label: Tavily proxy
width: 200
at: [ 0, 1.45 ]
in: control-service
tone: sage
details: An authenticated proxy inside the control-plane service calls the external Tavily API for Discovery's internet research.
discovery:
label: Discovery agent
width: 240
at: [ 1.8, 1.45 ]
in: discovery-runtime
tone: blue
details: The tenant-local Discovery agent gathers evidence and stores findings for validation by the control-plane service.
local-workload:
label: Local workload instance
width: 240
at: [ 1.8, 2.7 ]
in: discovery-runtime
tone: blue
details: A local workload instance inside the Discovery harness uses temporary, scoped access to workload test datastores.
storage:
label: Discovery Workspace Storage
shape: bucket
at: [ 3.4, 0 ]
in: tenant-project
tone: blue
details: Private tenant storage holds Discovery inputs, supporting evidence, and checkpoints.
width: 224
gateway:
label: Connector Gateway
at: [ 3.4, 1.45 ]
in: tenant-project
tone: blue
details: Discovery can use approved integration read operations only when the connection and access have been configured.
width: 200
test-resources:
label: Workload test datastores
shape: database
width: 240
at: [ 3.4, 2.7 ]
in: tenant-project
tone: blue
details: Workload test datastores are accessed by the local workload instance with temporary, scoped access for the Discovery task.
tavily:
label: Tavily
shape: external
at: [ -0.2, 3.5 ]
tone: amber
details: The external Tavily API is outside both GCP project boundaries and is accessed through the control-plane proxy.
width: 144
edges:
- from: orchestration.right
fromOffset: 0.25
to: discovery-runtime.top
toOffset: 0.5
label: Dispatch task
tone: sage
- from: discovery.left
fromOffset: 0.25
to: orchestration.right
toOffset: 0.8
label: Report findings and resume Rocky
tone: sage
details: The completion path returns to Discovery orchestration inside the control-plane service, which validates the stored findings before resuming Rocky.
labelMaxWidth: 140
- from: discovery.left
fromOffset: 0.85
to: inference.right
label: Model calls
labelAt: [ 1.215, 2.9286 ]
tone: sage
labelMaxWidth: 120
- from: discovery.left
to: tavily-proxy.right
label: Authenticated search
labelAt: [ 1.2846, 1.6786 ]
tone: sage
- from: tavily-proxy.bottom
fromOffset: 0.1
to: tavily.top
toOffset: 0.5
label: Search API
tone: amber
labelMaxWidth: 100
- from: discovery-runtime.right
fromOffset: 0.15
to: storage.left
toOffset: 0.5
label: save progress
tone: blue
details: Reads private task inputs and checkpoints, and saves findings, supporting evidence, and progress in the same tenant bucket.
direction: both
labelAt: [ 3.1327, 0.8 ]
labelMaxWidth: 120
- from: discovery.right
to: gateway.left
label: external data reads
labelAt: [ 3.0615, 1.6786 ]
tone: blue
labelMaxWidth: 150
- from: discovery.bottom
to: local-workload.top
label: explore workload
tone: blue
- from: local-workload.right
to: test-resources.left
style: dashed
tone: blue
```
## Development in the age of LLMs
Obviously the majority of the platform code is AI-generated — but I don't think I would call my development strategy "vibe-coding". My definition of vibe-coding is literally having zero understanding of the underlying code or architecture and just continuously requesting more user-facing changes from a bot. My methodology is essentially:
1. Plan a new component of the system as best I can. Use AI to research options, fill in my own knowledge gaps, and then articulate the plan.
2. Generate structured requirements to be implemented as standalone "work units" within the component sprint.
3. Task an agent to implement the first work unit.
4. Review the diff: review core logic, documentation, and repo structure.
5. Test manually and deliberately.
6. Request refinements, review, test again, then move on to the next work unit.
By far, the most time-consuming part of the development cycle was planning and testing. The time it takes for AI to write the code itself is almost negligible - which is incredible considering if I had to hand-jam everything it would probably take several years to build the same platform (and become proficient in multiple coding languages). I found that if I rushed planning I would usually end up regretting it and have to go back and start from scratch. The agents almost alway over-engineer hacky solutions to compensate for gaps in planning.
### Admin interface
I also built an admin portal that used my admin permissions to interact directly with the control plane service and database (I still need to add a dedicated operator role and make admin access break-glass only 😬). Building the admin portal was trivial using AI but gives me any type of visibility I want into platform operations. While building out the platform I added the following utilities into the admin-site:
* Tenant metadata, storage, user management
* Rocky message, compaction, and context lab
* Workload infrastructure tracking
* Implementation agent testing and session tracing
* System-wide configuration like primary/alternate models
### CI/CD
I did all of my testing in a dev environment and manually deployed changes to it. For deploying changes to prod, GitHub Actions workflows run as service accounts with only permissions to publish and update service images. I have not built a pipeline to apply Terraform plans when new infrastructure changes are required, such as creating a new service, changing resource configurations, etc. That pipeline would require really strict security as infra changes require near admin privileges. I similarly need to create new tenant projects manually (there is no self-sign up in the platform for now).
## Lessons learned
Building a platform like this was a good experience. I built agents from scratch and refined their prompts, tool schemas, harnesses, and experimented with different models. But the most enjoyable part was architecting the overall system. Obviously AI can quickly generate and refine front-ends, API contracts, and core logic (if well planned), but managing the tradeoffs of the overall infrastructure, such as cost, security, scalability, and maintenance, is not something that should be delegated to the model.
I know for a fact "vibe-coding" parts of the platform would have resulted in security issues. When I was initially planning out the inference gateway, I wanted to add time and cost-bound sessions. For example, the implementation engineer had one hour and $10 of LLM spend to finish building a workload. This was before I added a monthly usage cap per tenant.
GPT 5.5 created a plan for the inference gateway. The control-plane-service would mint sessions on an `/internal` endpoint. And implementation runtimes send LLM traffic to the chat completions endpoint, which is forwarded to the providers. However, the model forgot to add service-based authorization for those endpoints. So an agent that regularly processes untrusted instructions could have been able to mint as many inference sessions as it wanted with unlimited time and budget.
```architecture
version: 5
title: Session admin authorization vulnerability
description: In the vulnerable proposal, only the control-plane service should mint inference sessions through the gateway's /internal endpoint. Missing service authorization also lets the tenant implementation agent, following untrusted user instructions, mint sessions with arbitrary expiry and cost limits, then use them for provider-backed model calls. Green shows the intended minting path; the dashed amber path shows the bypass. This illustrates the planning flaw described above.
grid: { column: 260, row: 160, nodeWidth: 200 }
sections:
tenant:
label: Tenant project
tone: blue
gateway:
label: Inference gateway
subtitle: Shared service
tone: sage
nodes:
control:
label: Control service
at: [ 0, 0 ]
tone: sage
details: The control-plane service is the intended caller of the internal session-minting endpoint. It sets the implementation run's authorized duration and cost limit.
agent:
label: Untrusted agent
at: [ 0, 1.3 ]
in: tenant
tone: amber
details: The tenant implementation agent follows untrusted user instructions. In the flawed proposal it could call the internal minting endpoint itself and obtain sessions with self-selected time and cost limits.
sessions:
label: /internal
at: [ 1.6, 0 ]
in: gateway
tone: amber
details: The proposed session-minting endpoint lacked service-based authorization. It would accept the implementation agent as a caller instead of restricting minting to the control-plane service.
chat:
label: Chat completions
at: [ 1.6, 1.3 ]
in: gateway
tone: sage
details: The agent uses its self-minted session for model calls, bypassing the time and cost limits that the control-plane service intended to impose.
providers:
label: LLM providers
shape: external
width: 176
at: [ 3, 1.3 ]
tone: neutral
details: The inference gateway proxies model calls to upstream providers.
edges:
- from: control.right
to: sessions.left
label: Mint session
tone: sage
- from: agent.right
fromOffset: 0.2
to: sessions.left
toOffset: 0.8
label: Missing authorization
annotation: Any limits
labelMaxWidth: 160
style: dashed
tone: amber
details: The implementation agent can request and receive sessions with arbitrary time and cost limits because the internal endpoint does not check the calling service's authority to mint sessions.
- from: agent.right
to: chat.left
label: Uncapped calls
labelAt: [ 1.1846, 1.5 ]
tone: amber
details: The agent submits chat-completions requests using the session it minted for itself.
- from: chat.right
to: providers.left
label: Model calls
tone: sage
```
Just one of many examples of why you don't trust agents with decisions that need to be made by someone you can fire or sue.