Why Gitpod is Leaving Kubernetes

Gitpod has decided to transition away from Kubernetes for managing cloud development environments, opting instead for a custom-built solution better suited to their needs. While Kubernetes is powerful for orchestrating stateless application workloads, Gitpod identified several challenges that made it less ideal for their dynamic, stateful workloads.

Key Challenges of Kubernetes

• Resource Overhead: Kubernetes introduces significant complexity and resource consumption, which is inefficient for scaling ephemeral development environments.

• Latency in Scaling: The time required to scale pods and handle stateful workloads can slow down developer workflows that demand near-instant provisioning.

• Stateful Workloads: Kubernetes is designed for stateless applications, and adapting it for stateful environments adds operational complexity.

• Cost Inefficiency: Running dynamic workloads on Kubernetes incurs higher operational costs due to the constant need for scaling and resource orchestration.

• Security Concerns: Managing multi-tenant security on Kubernetes is challenging, requiring considerable effort to ensure workload isolation and permission control.

• Operational Complexity: Maintaining Kubernetes clusters at scale involves a significant operational burden, including updates, monitoring, and configuration management.

Gitpod is now focusing on Gitpod Flex, a new solution tailored to better meet the demands of developers, offering improved scalability, efficiency, and simplicity.

tl;dw (best DevOps tools in 2023)

1. Low-budget cloud computing : Civo (close to Scaleway)
2. Infrastructure and Service Management: Crossplane
3. App Management - manifests : cdk8s (yes, not Kustomize or Helm)
4. App Management - k8s operators: tie between Knative and Crossplane
5. App Management - managed services: Google Cloud Run
6. Dev Envs: Okteto (yeap, not GitPod)
7. CI/CD: GitHub Actions (as it's simplest to use)
8. GitOps (CD): Argo CD (wins with Flux due to its adoption rate)
9. Policy Management: Kyverno (simpler to use than industry's most powerful tool: OPA / Gatekeeper)
10. Observability: OpenTelemetry (instrumentation of apps), VictoriaMetrics (metrics - yes not Prometheus), Grafana / Loki (logs), Grafana Tempo (tracing), Grafana (dashboards), Robusta (alerting), Komodor (troubleshooting)