Managed Compute
Managed compute is the set of cloud services that run code while the provider operates part of the underlying fleet. The useful distinction is not “serverless versus servers”; it is startup latency, concurrency model, runtime limit, state, accelerator access, network control, and operational ownership. A model serving API, a nightly batch job, and a GPU trainer have different compute shapes even if all run containers.
Matching workload to compute
Use the workload contract:
flowchart LR Shape[Request or job shape] --> Runtime[Runtime, state, and hardware] Runtime --> Scaling[Scaling signal] Scaling --> Deployment[Deployment unit] Deployment --> Failure[Failure behavior]
Functions fit short event handlers. Cloud Run-style containers fit stateless HTTP services with configurable concurrency. Kubernetes fits teams that need custom scheduling, sidecars, or portability. Batch services fit finite jobs. GPU instances or managed ML jobs fit GPU systems. The choice feeds cost management: high concurrency can reduce instances, but only if application code is actually safe and efficient under parallel requests.
Worked concurrency check
For a stateless HTTP service handling 300 requests/s with 180 ms service time, Little’s Law gives about in-flight requests. With a 70% target utilization, the required instance count is
| Per-instance concurrency | Required instances |
|---|---|
| 1 | 78 |
| 8 | 10 |
| 80 | 1 |
Cloud Run documentation explicitly treats concurrency as a scaling and cost control. The table is not a recommendation to set concurrency to 80 blindly; CPU-bound Python code or shared mutable state can require a lower setting. That is a scalability test, not a console default.
Caveats
Managed compute still needs IAM, quotas, logs, rollbacks, and health checks. Cold starts matter for bursty APIs. Hidden retries can duplicate side effects unless the handler is idempotent. Regional outages still require reliability design, and managed GPU endpoints can be capacity-constrained in ways a generic autoscaler cannot fix.
References
Nav