• Core Value Proposition:

  • Combines Amazon EKS Auto Mode (automating infrastructure and compute layer management) with Istio Ambient Mesh (automating service-to-service networking and security) to significantly reduce manual operational overhead and strengthen security.

• Amazon EKS Auto Mode Key Components:

  • Managed Instances: AWS fully controls the lifecycle, patching, and security configurations of nodes; direct SSH access is removed in favor of Kubernetes-native troubleshooting.
  • Bottlerocket-based OS: Nodes utilize Bottlerocket, a minimal, immutable, and container-optimized Linux distribution enforcing strict security boundaries via SELinux.
  • Built-in System Components: Core add-ons—including Amazon VPC CNI, kube-proxy, Amazon EBS CSI driver, CoreDNS, and AWS Load Balancer Controller—are managed directly by AWS as system processes to eradicate version compatibility friction and minimize the threat surface.
  • Karpenter-powered Scaling: Employs a custom integrated version of Karpenter that dynamically provisions right-sized instances based on pod requests, continuously evaluates consolidation opportunities, and optimizes workloads onto Spot instances where possible.

• Istio Ambient Mesh Capabilities:

  • Sidecarless Architecture: Shifts traffic security and policy enforcement out of individual application pods into a split infrastructure model, decoupling service networking from application lifecycles and significantly cutting down resource overhead.
  • Layer 4 Security (ztunnel): Uses a secure overlay network via node-level ztunnel proxies to enforce zero-trust capabilities like automatic mutual TLS (mTLS) encryption, L4 authorization policies, and TCP-level observability.
  • Layer 7 Capabilities (Waypoint proxies): Provisions optional Layer 7 Waypoint proxies externally to implement advanced traffic routing, circuit breaking, and rich cryptographic or application-layer policy enforcement without sidecars.

• Integration and Workload Onboarding:

  • Unified Automation: Offloads data-plane management entirely to AWS while simultaneously removing sidecar proxy complexities from the application architecture.
  • Incremental Mesh Onboarding: Workloads can be added selectively to the ambient mesh simply by applying the label istio.io/dataplane-mode=ambient at either the namespace level or to individual target pods.

• Kubernetes (K8s) has become nearly universal across companies of all sizes, including early-stage startups that do not have immediate technical scaling needs.
• The shift toward K8s is primarily driven by organizational benefits rather than pure technical necessity:
  • It enforces uniform deployment processes across different applications.
  • It standardizes engineering knowledge, making the infrastructure architecture highly visible via YAML manifests rather than keeping it locked in individual developers' heads.
  • It provides end-to-end traceability and predictability through GitOps workflows (e.g., using Argo CD or Flux CD).
• The universal adoption has been accelerated because managed K8s offerings (such as AWS EKS, Google GKE, and Azure AKS) have matured significantly, and the engineering talent pool has heavily flipped in favor of K8s familiarity.
• Despite its organizational strengths, the author still recommends that early-stage startups start without Kubernetes because debugging cluster complexities can heavily drain energy that should be focused on building the core product.

Hacker News Discussion

• Many commenters warn that Kubernetes is far from "batteries-included." Setting up a basic functional cluster requires installing and self-managing numerous third-party controllers (like ingress, cert-manager, external-dns, and CoreDNS), creating significant hidden operational overhead.
• Maintaining Kubernetes involves a severe maintenance loop, forcing teams to perform major cluster and controller upgrades every few months while constantly navigating breaking changes and configuration churn.
• A common critique is the massive shift of complexity from application programming languages to configuration management (e.g., YAML, Helm, Kustomize), which is often harder to validate for correctness and can easily become a "Byzantine mesh" of plug-ins.
• Some participants argue that there is a massive gap in the market for a simpler alternative that allows declarative, containerized deployments to a pool of instances without the full operational complexity of K8s, though others counter that user demands for endless customization always inevitably drive simple tools to become complex orchestrators.
• Proponents note that while K8s doesn't stop engineers from making poor architectural decisions (like packaging unencrypted secrets inside images), it structurally makes it much easier to implement and adhere to best practices compared to traditional VPS or bare SSH workflows.

• Viability: Plain Docker Compose remains a viable option for production workloads in 2026, especially for single-node deployments, edge computing, or internal services that don't require the complexity of Kubernetes.
• Addressing Operational Gaps: Success depends on manually closing gaps that Compose leaves open:
  • Orphan Containers: Use the --remove-orphans flag during up and down commands to ensure containers removed from the YAML file are actually stopped and cleared.
  • Disk Management:
    • Implement log rotation in daemon.json (e.g., max-size: 10m) to prevent unbounded log files from filling disks.
    • Establish a schedule for pruning unused images and build caches (docker image prune).
    • Exercise caution with docker volume prune to avoid accidental data loss from detached volumes.
  • Health Checks: Native Docker health checks only report status; they do not automatically restart containers. Use a sidecar like docker-autoheal or a dedicated agent to act on "unhealthy" states.
  • Image Pinning: Avoid using mutable tags like :latest. Instead, pin images using their immutable SHA256 digests (image: myapp@sha256:...) to ensure consistency across different host pulls.
• Security Risks: Mounting /var/run/docker.sock provides a container with effective root privileges on the host. Minimize its use, consider rootless Docker, or use a socket proxy to limit API exposure.
• Scaling Updates: For managing multiple environments, tools like Watchtower (polling) or pull-based agents are necessary, as Docker has no native mechanism to "push" updates to remote Compose hosts.
• Growth Path: When requirements outgrow a single node, Kubernetes is the industry standard for migration. Docker Swarm is an alternative that reuses Compose syntax but has a smaller ecosystem.

• Docker Inc. has faced multiple identity crises since revolutionizing containerization, struggling to monetize its commoditized open-source core technology.
• Pivoted from orchestration (sold Swarm after Kubernetes dominance) to developer tools like Docker Scout (supply chain security via Atomist acquisition) and Testcontainers (integration testing).
• Shifted to AI with Model Runner, Compose for agents, Offload for GPUs, partnerships (Google Cloud, Azure), and MCP Defender acquisition for AI security.
• Released 1,000+ free Apache 2.0 Hardened Images (95% fewer vulnerabilities) in response to Chainguard, questioning future revenue from security features.
• CEO change to ex-Oracle exec sparks acquisition speculation by cloud giants; technology endures as infrastructure despite company uncertainty.

Hacker News Discussion

• Docker commoditized itself via open source; hard to monetize core tech, leading to pivots—enterprise ignored rootless/cgroups needs early, enabling Podman/Quay rise.
• Criticism of Docker Desktop licensing "gotcha" tactics pushing users to Rancher/Colima/OrbStack; developer tools market tough as devs resist paying.
• Defenses: maintained OCI runtimes (runc/containerd origins), free hardened images generous; AI/exploration necessary amid hyperscaler dominance.
• Swarm never meant to rival K8s; Podman docs lacking, but rootless superior; alternatives like gVisor, Apple container, Nix praised.
• Broader: open infra hard to profit (e.g., Redis forks); VCs expect quick exits; Docker success lives on despite company struggles.

• Kubernetes functions as a runtime for declarative infrastructure with a type system, rather than just a container orchestrator.
• Users declare desired state via manifests; the system continuously reconciles actual state to match intent through a cycle: declare → persist → reconcile → place → execute.
• Kubernetes provides a type system with resource kinds like Pod, Deployment, Service—each with strict definitions, semantics, and behaviors; CRDs extend this by defining custom types.
• API server validates and persists declarations as durable state; controllers watch for changes, compare spec (desired) vs. status (observed), and act to converge them.
• Continuous reconciliation prevents drift: manual changes get reverted if they conflict with declared intent.
• GitOps integrates naturally—Git holds source of truth, GitOps controller reconciles cluster state to Git, treating kubectl as a debugging tool for managed resources.
• Practical advice: change desired state not symptoms, let reconciliation handle work, make ownership explicit, use the type system properly.

• AWS EKS costs can escalate due to massive, parallel workloads in life sciences/drug development (e.g., genomic sequencing, molecular modeling).
• Default Kubernetes scheduler uses leastAllocated strategy, spreading pods across many nodes for fairness/high availability.
• leastAllocated strategy causes many partially utilized nodes, preventing autoscalers from scaling down idle nodes, increasing costs.
• mostAllocated scheduling strategy "packs" pods onto fewer nodes, maximizing utilization and enabling autoscalers like Karpenter to remove idle nodes.
• Switching to mostAllocated can reduce runtime costs significantly (e.g., ~10% in UAT, 43% in PROD environments).
• Custom scheduler deployment on AWS EKS requires creating a service account, ClusterRoleBindings, RoleBinding, a ConfigMap with the mostAllocated scoring strategy, and a deployment with a matching Kubernetes version container image.
• Resource weights can prioritize packing of expensive resources (e.g., high weight on GPUs for ML workloads).
• Testing in non-production environments is recommended before full rollout.
• Implementing mostAllocated scheduling can dramatically optimize costs by enabling cluster autoscalers to shut down unused nodes.

• DRA goes mainstream: stable core with ResourceClaim/DeviceClass and structured parameters — finally flexible, sane GPU/accelerator management without node‑selector tricks or quota hacks.
• Ops bonuses: production‑ready kubelet/API Server tracing (end‑to‑end debugging), new Deployment pod replacement policies (faster vs. resource‑conservative), PreferSameNode/SameZone traffic distribution, KYAML output in kubectl, and per‑HPA configurable tolerance.
• Zero breaking changes: no deprecations or removals — smooth upgrade for production teams.
• Release planned for late August

Summary: Navigating Failures in Pods With Devices

This article examines the unique challenges Kubernetes faces in managing specialized hardware (e.g., GPUs, accelerators) within AI/ML workloads, and explores current pain points, DIY solutions, and the future roadmap for more robust device failure handling.

Why AI/ML Workloads Are Different

• Heavy Dependence on Specialized Hardware: AI/ML jobs require devices like GPUs, with hardware failures causing significant disruptions.
• Complex Scheduling: Tasks may consume entire machines or need coordinated scheduling across nodes due to device interconnects.
• High Running Costs: Specialized nodes are expensive; idle time is wasteful.
• Non-Traditional Failure Models: Standard Kubernetes assumptions (like treating nodes as fungible, or pods as easily replaceable) don’t apply well; failures can trigger large-scale restarts or job aborts.

Major Failure Modes in Kubernetes With Devices

1. Kubernetes Infrastructure Failures

   • Multiple actors (device plugin, kubelet, scheduler) must work together; failures can occur at any stage.
   • Issues include pods failing admission, poor scheduling, or pods unable to run despite healthy hardware.
   • Best Practices: Early restarts, close monitoring, canary deployments, use of verified device plugins and drivers.

2. Device Failures

   • Kubernetes has limited built-in ability to handle device failures—unhealthy devices simply reduce the allocatable count.
   • Lacks correlation between device failure and pod/container failure.
   • DIY Solutions:
     • Node Health Controllers: Restart nodes if device capacity drops, but these can be slow and blunt.
     • Pod Failure Policies: Pods exit with special codes for device errors, but support is limited and mostly for batch jobs.
     • Custom Pod Watchers: Scripts or controllers watch pod/device status, forcibly delete pods attached to failed devices, prompting rescheduling.

3. Container Code Failures

   • Kubernetes can only restart containers or reschedule pods, with limited expressiveness about what counts as failure.
   • For large AI/ML jobs: Orchestration wrappers restart failed main executables, aiming to avoid expensive full job restart cycles.

4. Device Degradation

   • Not all device issues result in outright failure; degraded performance now occurs more frequently (e.g., one slow GPU dragging down training).
   • Detection and remediation are largely DIY; Kubernetes does not yet natively express "degraded" status.

Current Workarounds & Limitations

• Most device-failure strategies are manual or require high privileges.
• Workarounds are often fragile, costly, or disruptive.
• Kubernetes lacks standardized abstractions for device health and device importance at pod or cluster level.

Roadmap: What’s Next for Kubernetes

SIG Node and Kubernetes community are focusing on:

• Improving core reliability: Ensuring kubelet, device manager, and plugins handle failures gracefully.
• Making Failure Signals Visible: Initiatives like KEP 4680 aim to expose device health at pod status level.
• Integration With Pod Failure Policies: Plans to recognize device failures as first-class events for triggering recovery.
• Pod Descheduling: Enabling pods to be rescheduled off failed/unhealthy devices, even with restartPolicy: Always.
• Better Handling for Large-Scale AI/ML Workloads: More granular recovery, fast in-place restarts, state snapshotting.
• Device Degradation Signals: Early discussions on tracking performance degradation, but no mature standard yet.

Key Takeaway

Kubernetes remains the platform of choice for AI/ML, but device- and hardware-aware failure handling is still evolving. Most robust solutions are still "DIY," but community and upstream investment is underway to standardize and automate recovery and resilience for workloads depending on specialized hardware.

See the summary below this highlight

• Introduction

  • Containers are widely used for scalable applications but face challenges with startup times for large images (e.g., AI/ML workloads).
  • Pulling large images from Amazon Elastic Container Registry (ECR) can take several minutes, impacting performance.
  • Bottlerocket, an AWS open-source Linux OS optimized for containers, offers a solution to reduce container startup time.

• Solution Overview

  • Bottlerocket's data volume feature allows prefetching container images locally, eliminating the need for downloading during startup.
  • Prefetching is achieved by creating an Amazon Elastic Block Store (EBS) snapshot of Bottlerocket's data volume and mapping it to new Amazon EKS nodes.
  • Steps to implement:
  • Spin up an Amazon EC2 instance with Bottlerocket AMI.
  • Pull application images from the repository.
  • Create an EBS snapshot of the data volume.
  • Map the snapshot to Amazon EKS node groups.

• Benefits of Bottlerocket

  • It separates OS and container data volumes, ensuring consistency and security during updates.
  • Prefetched images significantly reduce startup times for large containers.

• Implementation Walkthrough

  • Step 1: Build EBS Snapshot
    • Automate snapshot creation using a script.
    • Prefetch images like Jupyter-PyTorch and Kubernetes pause containers.
    • Export the snapshot ID for use in node group configuration.
  • Step 2: Setup Amazon EKS Cluster
    • Create two node groups:
    • no-prefetch-mng: Without prefetched images.
    • prefetch-mng: With prefetched images mapped via EBS snapshot.
  • Step 3: Deploy Pods
    • Test deployment on both node groups.
    • Prefetched nodes start pods in just 3 seconds, compared to 49 seconds without prefetching.

• Results

  • Prefetching reduced container startup time from 49 seconds to 3 seconds, improving efficiency and user experience.

• Further Enhancements

  • Use Karpenter for automated scaling with Bottlerocket nodes.
  • Automate snapshot creation in CI pipelines using GitHub Actions.

• Cleaning Up

  • Delete AWS resources (EKS cluster, Cloud9 environment, EBS snapshots) to avoid charges after testing.

• Conclusion

  • Bottlerocket's data volume prefetching dramatically enhances container startup performance for large workloads on Amazon EKS.

• Why use Kubernetes

  • Best for running multiple processes/servers/jobs with redundancy and load balancing
  • Enables infrastructure-as-code configuration for service relationships
  • Outsourced infrastructure management via cloud providers (e.g., Google Kubernetes Engine)

• What they use

  • Core resources: Deployments (with rolling updates), Services (ClusterIP/LoadBalancer), CronJobs
  • Configuration: ConfigMaps and Secrets via Pulumi (TypeScript) instead of raw YAML
  • Cautious adoptions: StatefulSets for limited persistence, RBAC only when necessary

• What they avoid

  • Hand-written YAML and Helm charts ("fragility for no gain")
  • Operators, custom resources, service meshes, and most third-party controllers
  • Local k8s stack replication (prefer Docker Compose for local dev)

• Key insights

  • "A human should never wait for a pod" - unsuitable for interactive workloads requiring fast startup
  • Use managed databases/storage for critical data instead of k8s volumes
  • Alternatives like Railway/Render may be better for simpler SaaS apps
  • Recently adopted Ingress controllers for Cloud Armor integration despite initial reservations

• Figma's Initial Infrastructure Challenges:

  • Figma's monolithic architecture struggled with resource allocation inefficiencies and limited scalability.
  • High traffic spikes from collaborative design workflows required more robust solutions for resource autoscaling and failover.

• Why Kubernetes Was Chosen:

  • Kubernetes' container orchestration capabilities promised better resource management and service isolation.
  • Features like Horizontal Pod Autoscaling (HPA), robust networking via Kubernetes Services, and support for StatefulSets made it an ideal fit for Figma’s needs.
  • The platform also wanted better alignment with cloud-native practices and modern CI/CD workflows.

• Incremental Migration Approach:

  • Step 1: Non-Critical Services: Figma migrated stateless services first, allowing experimentation without risking core functionality.
  • Step 2: Custom Tooling: Internal tooling was built to manage Kubernetes manifests and automate Helm chart creation for standardization.
  • Step 3: Stateful Services: For databases and other stateful components, Figma relied on Kubernetes' StatefulSets and persistent volumes (PVs) to ensure data integrity during the migration.
  • Step 4: Observability Enhancements: Kubernetes-native tools like Prometheus and Grafana were integrated to provide detailed metrics and system insights.

• Key Technical Adjustments During Migration:

  • Service Discovery: Transitioned to Kubernetes-native DNS for internal service communication, replacing legacy methods.
  • Load Balancing: Leveraged Kubernetes Ingress and external load balancers (e.g., NGINX or cloud-native solutions) for traffic routing.
  • Networking Complexity: Resolved challenges around multi-cluster networking using Kubernetes CNI plugins like Calico.
  • Resource Management: Used Resource Quotas and Limits to prevent pod overcommitment and optimize cluster utilization.

• Challenges Faced:

  • Stateful Services: Ensuring zero-downtime migration for databases required careful orchestration of PersistentVolumeClaims (PVCs) and StatefulSets.
  • Networking: Handling cross-region traffic and external dependencies required tweaking Kubernetes Ingress configurations.
  • Resource Constraints: Balancing costs and performance involved tuning cluster-autoscaler configurations and evaluating node pool setups.

• Benefits Realized Post-Migration:

  • Scalability: Kubernetes' HPA allowed Figma to scale pods dynamically based on traffic patterns, ensuring consistent performance.
  • Deployment Efficiency: CI/CD pipelines integrated seamlessly with Kubernetes, enabling faster and more reliable rollouts using tools like Argo CD.
  • Reliability: Self-healing capabilities, such as pod restarts and node failover, reduced downtime during failures.
  • Observability: Improved system monitoring with Kubernetes' native metrics server and integrations with Prometheus and Grafana.

• Future Enhancements Planned:

  • Service Mesh Integration: Adoption of Istio or Linkerd to enhance observability, security (e.g., mutual TLS), and traffic management.
  • Cost Optimization: Further tuning autoscaling policies and resource limits to minimize waste.
  • Edge Improvements: Deploying Kubernetes clusters closer to end-users for reduced latency, potentially using Kubernetes' Cluster Federation.

• Overview of Tesla's Data Infrastructure Challenges:

  • Modern Tesla vehicles generate an enormous volume of telemetry data related to sensor readings, driver behavior, energy consumption, and more.
  • The primary challenge is ingesting, processing, and analyzing this data at scale while maintaining real-time capabilities.

• Kubernetes for Orchestration:

  • Tesla uses Kubernetes to manage containerized microservices across a distributed cloud environment.
  • Kubernetes ensures dynamic scaling to handle fluctuating workloads, providing high availability for critical services.
  • Each microservice is encapsulated in its own container, improving isolation and deployability.

• Kafka for Real-Time Event Streaming:

  • Apache Kafka is the backbone of Tesla’s data pipeline, managing trillions of events daily from globally distributed vehicles.
  • Kafka topics are structured to partition and replicate data efficiently, ensuring fault tolerance and high throughput.
  • Producers (vehicles) send data to Kafka brokers, while consumers (analytics systems, data lakes) process these streams in real-time.

• Data Processing Pipelines:

  • Data from Kafka is ingested into processing systems for real-time analytics, anomaly detection, and predictive maintenance.
  • Stream processing frameworks (e.g., Apache Flink or Kafka Streams) analyze data for immediate feedback.
  • Batch systems handle aggregation and storage in Tesla’s data lake for long-term insights.

• Key Technical Advantages:

  • Scalability: Kubernetes dynamically allocates resources based on the volume of incoming data and computational requirements.
  • Resilience: Kafka’s replication factor ensures that no single broker failure impacts the system.
  • Low Latency: Data streams from Kafka enable Tesla to act on insights in milliseconds, critical for safety and performance monitoring.

• Simplified Management:

  • The platform supports multi-cluster Kubernetes configurations for geographic data segregation.
  • A central control plane monitors system health, manages deployments, and ensures compliance with data regulations.

• Future Goals and Improvements:

  • Enhancing AI-driven analytics to derive deeper insights from vehicle data.
  • Further optimizing Kafka’s cluster topology to improve fault tolerance and reduce operational costs.
  • Expanding edge processing capabilities in vehicles to pre-filter data, reducing bandwidth requirements to the cloud.

• Kubernetes Configuration Drift Issue:

  • Reddit experienced a significant outage on March 13, 2022, during a Kubernetes upgrade from version 1.23 to 1.24.
  • The outage was caused by configuration drift, where unintended changes accumulated over time, leading to inconsistencies across clusters.

• New Platform Abstraction with Declarative APIs:

  • Reddit adopted a declarative approach, leveraging Kubernetes controllers to manage configurations and enforce consistency.
  • The implementation of these controllers enabled Reddit to abstract platform complexities and ensure a uniform deployment environment.

• Centralized Control Plane for Multi-Cluster Management:

  • The team built a centralized control plane to manage multiple Kubernetes clusters effectively.
  • Cluster provisioning time was drastically reduced from over 30 hours to approximately 2 hours.
  • Centralization facilitated standardized configurations and reduced operational overhead.

• Development of Achilles SDK:

  • Achilles, an SDK developed in-house by Reddit, simplified the creation of Kubernetes controllers.
  • It allowed infrastructure engineers to automate and manage resources programmatically without deep Kubernetes expertise.
  • The SDK supported a more proactive approach to problem-solving, preventing drift by design.

• Benefits and Lessons Learned:

  • The new system ensured robust monitoring, minimized manual intervention, and improved scalability.
  • Configuration drift was effectively mitigated, providing a more stable and predictable infrastructure.
  • The experience highlighted the importance of using Kubernetes-native solutions and declarative configurations for managing large-scale deployments.

• Future Goals:

  • Further refinement of the platform to address edge cases and improve developer experience.
  • Continued investment in tools and processes to maintain infrastructure consistency at scale.

Why Gitpod is Leaving Kubernetes

Gitpod has decided to transition away from Kubernetes for managing cloud development environments, opting instead for a custom-built solution better suited to their needs. While Kubernetes is powerful for orchestrating stateless application workloads, Gitpod identified several challenges that made it less ideal for their dynamic, stateful workloads.

Key Challenges of Kubernetes

• Resource Overhead: Kubernetes introduces significant complexity and resource consumption, which is inefficient for scaling ephemeral development environments.

• Latency in Scaling: The time required to scale pods and handle stateful workloads can slow down developer workflows that demand near-instant provisioning.

• Stateful Workloads: Kubernetes is designed for stateless applications, and adapting it for stateful environments adds operational complexity.

• Cost Inefficiency: Running dynamic workloads on Kubernetes incurs higher operational costs due to the constant need for scaling and resource orchestration.

• Security Concerns: Managing multi-tenant security on Kubernetes is challenging, requiring considerable effort to ensure workload isolation and permission control.

• Operational Complexity: Maintaining Kubernetes clusters at scale involves a significant operational burden, including updates, monitoring, and configuration management.

Gitpod is now focusing on Gitpod Flex, a new solution tailored to better meet the demands of developers, offering improved scalability, efficiency, and simplicity.

The introduction of the Image Volume Source feature in Kubernetes 1.31 allows MLOps practitioners to mount OCI-compatible artifacts, such as large language model weights or machine learning models, directly into pods without embedding them in container images. This streamlines model deployment, enhances efficiency, and leverages OCI distribution mechanisms for effective model management.

The blog post by Cliff Malmborg from Loft Labs discusses optimizing Kubernetes costs using multi-tenancy and virtual clusters. With Kubernetes expenses rising rapidly at scale, traditional cost-saving methods like autoscaling, resource quotas, and monitoring tools help but are not enough for complex environments where underutilized clusters are common. Multi-tenancy enables resource sharing, reducing the number of clusters and, in turn, management and operational costs.

A virtual cluster is a fully functional Kubernetes cluster running within a larger host cluster, providing better isolation and flexibility than namespaces. Unlike namespaces, each virtual cluster has its own Kubernetes control plane, so resources like statefulsets and webhooks are isolated within it, while only core resources (like pods and services) are shared with the host cluster. This setup addresses the "noisy neighbor" problem, where workloads in a shared environment interfere with each other due to resource contention.

Virtual clusters offer the isolation benefits of individual physical clusters but are cheaper and easier to manage than deploying separate physical clusters for each tenant or application. They also support "sleep mode," automatically scaling down unused resources to save costs, and allow shared use of central tools (like ingress controllers) installed in the host cluster. By transitioning to virtual clusters, companies can balance security, isolation, and cost-effectiveness, reducing the need for multiple physical clusters and making Kubernetes infrastructure scalable for modern, resource-demanding applications.

In this case, IaC is favored over using EKS directly or manually deploying on EC2

Why use EKS

Cost of EKS

Benefits of using Kubernetes on AWS: - scalability - cost efficiency - high availability

They went through 310 role descriptions and, even though role descriptions may vary significantly, they found 3 core skills that a large percentage of MLOps roles required:

📦 Docker and Kubernetes 🐍 Python 🌥 Cloud

Kubernetes clusters have a maximum IP address limitation

Some of my favourite k8s aliases: * 2. * 3.

Command to check whether an action is allowed

Quarkus framework

Kubernetes-native application

kubectl run a –image alpine –command — /bin/sleep 1d

kubectl get pods -A -o wide

kubectl get nodes -o wide

kubectl get componentstatus

kubectl get events -A

kubectl api-resources -o wide –sort-by name

Alternative to kubectl get --raw '/healthz?verbose'. It does not show scheduler or controller-manager output, but it adds a lot of additional checks that might be valuable if things are broken

8 commands to debug Kubernetes cluster:

kubectl version --short
kubectl cluster-info
kubectl get componentstatus
kubectl api-resources -o wide --sort-by name
kubectl get events -A
kubectl get nodes -o wide
kubectl get pods -A -o wide
kubectl run a --image alpine --command -- /bin/sleep 1d

High level definition of Argo Workflow

Pros of Argo Workflow:

• Resilience
• Autoscaling
• Configurability
• Support for RBAC

Cons of Argo Workflow:

• A lot of YAML files required
• k8s knowledge required

Should you go into Argo, or not?

kind: WorkflowTemplate

At the current moment the best approach is to use minikube with a preferred backend (Docker Engine and Podman are already there), and you can simply run one command to configure Docker CLI to use the engine from the cluster.

• Drivers | minikube
• docker-env | minikube

使用以下命令可以查看结果：

sudo iptables -t nat -nvL OUTPUT

这里应该要创建 Deployment:

kubectl create deployment nginx-web-1 --image=nginx

k3d <--- k3s that allows to run mult-node clusters on a local machine

KinD (Kubernetes in Docker) <--- sounds like the most recommended solution to learn k8s locally

k0s <--- similar to k3s, but not as lightweight

k3s <--- lightweight solution

7 tools for learning k8s locally:

1. k3s
2. k0s
3. Microk8s
4. DinD
5. minikube
6. KinD
7. k3d

We can run Kubernetes locally as a:

1. binary package
2. container using dind

With arkade, we can quickly set up different k8s tools, while using a single command:

e.g. arkade get k9s

• docker-compose runs on a single node (hard to scale)
• docker-swarm or kubernetes run on multiple nodes

Reason why one should not be worried about k8s depreciating Docker

What about between clusters?

中文读者可以在完成了本文学习后，参考这篇文章了解。文章对本文的源码分析做了一些补充。

不是应该每个结点上都有 kube-proxy 吗？

How Vertex AI solves the problem/need of deploying on a Kubernetes Cluster

With Spark 3.1 k8s becomes the right option to replace YARN

You might not need all the Kubernetes clusters and run well on a single Linode VPS.

Twitter thread: https://twitter.com/levelsio/status/1101581928489078784

How Prometheus and Grafana can be used to collect information from running ML on K8s

The way OpenAI runs large ML jobs on K8s

Kubernetes autoscaler

Tip for high availability:

• have at least 2 masters
• set --apiserver-count flag to the number of running apiservers

If we've more than 1k nodes, etcd's hard storage limit might stop accepting writes

Another trick apart from tweaking default settings of Fluentd & Datadog

Default settings of Fluentd and Datadog might not be suited for running many nodes

One of the solutions for etcd using only about 10% of the available IOPS. It was working till about 1k nodes

But it's provided as a plugin

Kubernetes (simple explanation)

Maybe Kubernetes has the advantage of being cloud neutral, but: you pay the cost of a cloud migration:

• maintaining abstractions
• isolating your way from useful vendor specific features

You can set up yourself in production in minutes to only a few hours

Kubernetes:

• require a full time ops team to operate
• the learning curve is far steeper than the alternatives

Use the closest thing to a pure-managed platform as you possibly can. It will be easier to operate in production, and more explicable and supported:

• Azure App Service
• Google App Engine
• AWS Lambda

Kubernetes aren't always required unless you work on huge problems

Other alternatives to Kubernetes:

• Docker Compose on a single machine
• Heroku and similar systems
• Snakemake for computational pipelines

Caring about downtime

Kubernetes' health checks can be replaced with nginx on worker processes + docker-autoheal to automatically restart those processes

Kubernetes alternatives:

• cloud VMs with up to 416 vCPUs and 8 TiB RAM
• scale many web apps with Heroku

Microservices stay as a hard nut to crack.

They are fine for an organisational scaling technique: when you have 500 developers working on one live website (so they can work independently). For example, each team of 5 developers can be given one microservice

You need a complete K8s to run your code, or you can use Telepresence to code locally against a remote Kubernetes cluster

Deployment of Kubernetes is non-trivial

Before running the simplest Kubernetes app, you need at least this architecture:

As of March 2020, the Kubernetes code base has more than 580 000 lines of Go code

Kubernetes might be not the best solution in a smaller team

apt-get install -y docker.io

Install Docker container runtime first.

apt-get install -y docker.io

Install runtime.

sudo -i
apt-get update && apt-get upgrade -y
apt-get install -y docker.io

Install kubeadm, kubelet and kubectl.

https://kubernetes.io/docs/setup/independent/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl

apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

Uau! Muitos assuntos da prova LPI DevOps são explorados nessa palestra. Fica de olho no tópico: 702 Container Management.

95%成功率，需要 25,000,000个参数！