Volcano and YuniKorn: Kubernetes Batch Scheduling for AI Workloads Beyond the Default Scheduler

I have watched the default Kubernetes scheduler fail in fascinating ways over twenty years of building distributed infrastructure. None of those failures were more systematic, or more expensive in terms of wasted GPU-hours, than the ones that happen when you point a standard kube-scheduler at an AI training workload.

The scheduler was designed for always-on services. Start a pod, keep it running, restart it if it crashes. It is exceptionally good at that job. But distributed machine learning is a fundamentally different animal. A PyTorch training run with 64 workers is not 64 independent pods. It is a single computation that happens to require 64 coordinated execution units. If 63 workers start and the 64th stays pending because the cluster is busy, you have not made 98% progress. You have made zero progress, consumed GPU-hours on 63 nodes, and the job will eventually time out or deadlock.

This is the gang scheduling problem, and it is why the teams actually running AI training at scale use specialized batch schedulers rather than the default kube-scheduler.

Two projects dominate this space: Volcano and Apache YuniKorn. If you have read our guide to Kueue, you know there is a third option built into the Kubernetes ecosystem. This article covers the cases where Volcano and YuniKorn are the better fit, what they each do well, and how to choose between them.

Why the Default Kubernetes Scheduler Falls Short for Batch AI

The standard Kubernetes scheduler makes decisions one pod at a time. It looks at each pending pod in priority order, finds a node that fits based on resource requests, and binds the pod. That model has no concept of a job as a coordinated unit.

Consider what happens with a 16-GPU training job in a cluster running at 80% GPU utilization. The scheduler might successfully place 12 of your 16 workers before the remaining GPUs are claimed by other workloads. Your 12 workers sit idle, holding those GPUs, waiting for the other 4. The other workloads that grabbed those 4 GPUs might themselves be waiting for other resources. You have a cluster-level deadlock, or at best a starvation scenario where your job burns money doing nothing useful.

Beyond gang scheduling, there are fairness problems. In a multi-tenant cluster shared by multiple data science teams, nothing in the default scheduler prevents one team from flooding the queue and starving every other team’s workloads. You need hierarchical quotas, fair share policies, and priority lanes with preemption.

The standard Kubernetes resource management model handles requests and limits per container but has no concept of job-level fairness or team-level quotas. That gap is exactly what Volcano and YuniKorn exist to fill.

Volcano: The Gang Scheduling Workhorse

Volcano is a CNCF incubating project that originated as a Huawei contribution and lists adopters across major cloud providers including Baidu, Tencent Cloud, and Alibaba Cloud on its project page. I have seen it deployed by teams running hundreds of concurrent training jobs, and its design philosophy shows that heritage: it solves the gang scheduling problem aggressively, with a rich plugin architecture that extends to preemption, backfill, and elastic job scaling.

The Core Abstraction: PodGroup

Volcano introduces a PodGroup CRD that represents the minimum set of pods required for a job to be useful. You annotate your pods with a scheduling.volcano.sh/pod-group label, and Volcano’s scheduler only places the pods when it can satisfy the minMember requirement atomically.

apiVersion: scheduling.volcano.sh/v1beta1
kind: PodGroup
metadata:
  name: pytorch-training-job
spec:
  minMember: 8           # must start all 8 or none
  priorityClassName: high-priority
  queue: ml-team-a
  minResources:
    nvidia.com/gpu: "8"

The scheduler holds all pods for a PodGroup in a waiting state until it can atomically bind all of them. If the resources are not available, it does not start partial execution. This is the fundamental guarantee that prevents the wasted GPU-hours scenario I described earlier.

The VolcanoJob API

For tighter integration, Volcano provides a VolcanoJob CRD that manages the entire lifecycle of a distributed training job, including restarts, checkpointing coordination, and role-based task specifications (worker, parameter server, chief).

apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: pytorch-distributed
spec:
  minAvailable: 8
  schedulerName: volcano
  queue: ml-team-a
  policies:
    - event: PodEvicted
      action: RestartJob
  tasks:
    - replicas: 8
      name: worker
      template:
        spec:
          containers:
          - name: pytorch
            resources:
              limits:
                nvidia.com/gpu: "1"

The policies section gives you event-driven job lifecycle management. When a pod gets evicted due to node pressure, Volcano can restart the entire job from the beginning rather than leaving orphaned workers running. For frameworks with checkpoint support like PyTorch, you can configure a checkpoint action before restart. These semantics matter enormously when you are dealing with the node failures that are a daily reality in large GPU clusters.

Volcano’s Plugin Architecture

Volcano’s scheduler is built around a chain of action plugins. Each plugin implements one or more scheduling actions:

• Gang: atomic all-or-nothing allocation for PodGroups
• Proportion: distributes resources across queues in proportion to their weights
• Preempt: evicts lower-priority jobs to make room for higher-priority ones
• Reclaim: retrieves resources borrowed by lower-priority queues
• Backfill: places small jobs in gaps that would otherwise remain unused
• Priority: respects job and pod priority classes
• SLA: enforces job deadline and timeout policies

The queue model is flat in Volcano: you define queues with weights and optional capacity limits, and the scheduler distributes resources across those queues according to the proportion plugin. If a queue is idle, its resources are available for other queues to borrow. When the queue’s own jobs need those resources back, the reclaim plugin evicts the borrowers.

For GPU scheduling specifically, Volcano integrates with NVIDIA’s device plugin and supports the same GPU sharing modes (MIG, time-slicing, MPS) that you would use with any Kubernetes-native GPU scheduler.

Framework Integrations

One of Volcano’s genuine strengths is the breadth of its framework integrations. It ships with native support for:

• PyTorch / TorchElastic via the pytorch plugin
• TensorFlow (parameter server and all-reduce modes)
• Ray (as an alternative scheduler for KubeRay) - see our Ray infrastructure guide
• Apache Spark and Apache Flink
• MPI workloads
• Kubeflow operators (MPIJob, PyTorchJob, TFJob)

This breadth matters in shops that run mixed workloads: AI training alongside Spark data processing alongside MPI simulation jobs. Volcano handles all of them under a single queue model.

Apache YuniKorn: Fair Share and Hierarchical Quotas

YuniKorn came out of Cloudera, is now an Apache Top-Level Project (not CNCF; it lives in the Apache Software Foundation), and takes a fundamentally different architectural approach. Where Volcano extends the default kube-scheduler through a plugin framework, YuniKorn replaces the default scheduler entirely.

The replacement approach has consequences: you cannot run YuniKorn alongside the default scheduler for different workload classes, and admission webhooks that assume the default scheduler behavior may need attention. But it also gives YuniKorn complete control over the scheduling path, which enables some of its most powerful features.

The latest stable version as of this writing is 1.8.0, released February 2026.

Hierarchical Queues

YuniKorn’s defining feature is its hierarchical queue model. Instead of Volcano’s flat queue structure with weights, YuniKorn builds a tree of queues where each node carries guaranteed and maximum resource limits.

partitions:
  - name: default
    queues:
      - name: root
        submitACL: '*'
        queues:
          - name: ml-platform
            resources:
              guaranteed:
                vcore: 20
                memory: 200G
              max:
                vcore: 80
                memory: 800G
              nvidia.com/gpu: "20"
            queues:
              - name: team-a
                resources:
                  guaranteed:
                    vcore: 10
                    memory: 100G
              - name: team-b
                resources:
                  guaranteed:
                    vcore: 10
                    memory: 100G

The guaranteed setting defines the minimum resources the queue is always entitled to, even if the parent queue is being pressured by siblings. The max setting caps how much the queue can grow when slack is available. The difference between guaranteed and max is the elastic borrowing range.

This model maps cleanly onto organizational hierarchies: you have a root queue for the whole cluster, department-level queues with guaranteed allocations, and team-level queues underneath. A team’s job can burst beyond its guaranteed allocation if the department has slack, but it cannot burst beyond the department’s max. Preemption works at the queue boundary: a higher-guaranteed queue reclaims resources from lower-priority queues that borrowed beyond their guarantee.

I have seen this model work extremely well in shared AI platform environments where finance teams, data science teams, and inference teams all share the same GPU fleet. Each gets guaranteed headroom for their critical jobs while still being able to burst into each other’s slack when neighbors are idle.

Application-Aware Scheduling

YuniKorn groups pods into logical applications and tracks application state across the entire lifecycle. An application moves through states: Accepted, Starting, Running, Completing, Completed, Failing, Failed.

This application-level tracking enables smarter preemption: rather than evicting individual pods, YuniKorn can preempt entire applications and signal them to checkpoint before termination. It also gives you accurate fairness accounting at the application level rather than just the resource level.

Gang Scheduling in YuniKorn

YuniKorn supports gang scheduling through the TaskGroup scheduling approach. You annotate pods with their minimum completion size:

metadata:
  annotations:
    yunikorn.apache.org/task-group-name: "training-workers"
    yunikorn.apache.org/task-groups: |
      [{
        "name": "training-workers",
        "minMember": 8,
        "minResource": {"cpu": "4", "memory": "8G", "nvidia.com/gpu": "1"}
      }]      
    yunikorn.apache.org/schedulingPolicyParameters: "placeholderTimeoutSeconds=300"

YuniKorn uses a placeholder mechanism: when a gang-scheduled application is submitted, it creates placeholder pods (also called “gang-host” pods) to reserve the required resources before starting real pods. This reservation approach ensures the full allocation is available before any real compute starts. When real pods are placed, the placeholders are replaced.

The placeholderTimeoutSeconds setting is important: if the full gang cannot be satisfied within 300 seconds, YuniKorn releases the placeholders and moves the application back to waiting. This prevents reservation starvation where an application holds resources indefinitely without being able to run.

Integration with Apache Ecosystem

YuniKorn has particularly deep integration with the Apache ecosystem, including Apache Spark on Kubernetes. The Spark Kubernetes operator can plug directly into YuniKorn, enabling fair-share scheduling across Spark applications and GPU training jobs in the same cluster. This is its home turf, and it shows: the integration is mature and production-tested across major cloud providers.

AWS supports YuniKorn as the batch scheduler for Amazon EMR on EKS, which means if you are already running managed Spark on AWS, you have a supported path to YuniKorn without self-managing the scheduler binary. That is a meaningful operational advantage in enterprise environments where unsupported OSS schedulers face procurement and compliance friction.

Kueue: The Kubernetes-Native Third Option

If Volcano and YuniKorn both feel like they carry more complexity than your situation warrants, Kueue is worth considering. We have a dedicated Kueue guide that covers it in depth, so I will keep this brief.

Kueue takes a fundamentally different approach: instead of replacing or extending the scheduler itself, it sits above the scheduler as a job queuing layer. Kueue manages when jobs are admitted to the cluster; once admitted, they are scheduled by the default kube-scheduler (or whichever scheduler you configure). Kueue adds Kubernetes-native CRDs for ClusterQueues and LocalQueues, and it integrates natively with Kubernetes Job, JobSet, MPIJob, PyTorchJob, TFJob, and RayJob.

Kueue’s gang scheduling (through the PodSetTopologyRequest) is less sophisticated than Volcano’s, and its queue model is flat rather than hierarchical. But it requires zero scheduler replacement, has strong CNCF backing from Google, and its simpler operational model makes it the right call for teams that are not running hundreds of concurrent batch jobs.

Choosing Between Volcano, YuniKorn, and Kueue

Here is the decision framework I use when teams ask me which batch scheduler to run.

Choose Volcano if:

• Your primary workloads are AI/ML training jobs and you need mature, battle-tested gang scheduling
• You run mixed workloads (PyTorch, Spark, MPI, Flink) and want a single scheduler to handle all of them
• Your organization already uses Huawei Cloud or a Chinese cloud provider where Volcano has strong ecosystem support
• You need fine-grained job lifecycle policies (checkpoint-on-evict, restart-on-failure)
• You want a flat queue model with simple weight-based fair share

Choose YuniKorn if:

• You run a multi-tenant shared platform for many teams and departments
• Hierarchical resource guarantees across organizational levels are a hard requirement
• You have significant Apache Spark workloads alongside AI training
• You are running EMR on EKS and want AWS-supported batch scheduling
• Application-level tracking and preemption semantics matter to your use case

Choose Kueue if:

• You have fewer than a few dozen concurrent batch jobs and do not need sophisticated fair share
• You want to avoid replacing or extending the default scheduler
• Your workloads are primarily Kubernetes-native (Job, JobSet, Kubeflow operators)
• Your team has limited bandwidth to operate a dedicated batch scheduler
• You are prototyping an AI platform and want to grow into more sophisticated scheduling later

The practical overlap between Volcano and YuniKorn is significant, and the choice often comes down to organizational factors. If your data engineering team runs Spark and your ML team runs PyTorch on the same cluster, YuniKorn’s Apache ecosystem depth is a meaningful advantage. If your team’s workload is pure PyTorch training and you need production-proven gang scheduling with framework-specific job policies, Volcano has more operating history.

Production Deployment Considerations

Regardless of which scheduler you choose, there are a few operational realities I have learned from running batch schedulers in production.

Scheduler anti-affinity for the scheduler itself. Both Volcano and YuniKorn run as Kubernetes deployments. Pin them to dedicated control-plane nodes with taints, or at minimum use pod anti-affinity to keep them off your GPU nodes. A scheduler evicted from its own node while it is mid-decision is a bad day.

Queue saturation monitoring. The most common operational problem I see is unmonitored queue depth. A job queue that fills up silently because of a misconfigured minimum member count, or a queue that grows because a team is submitting jobs faster than they complete, creates a degraded experience for everyone. Export queue depth and pending job count from both schedulers’ metrics endpoints and alert on sustained saturation.

Placeholder timeout tuning (YuniKorn). The default placeholder timeout in YuniKorn is relatively generous. In clusters with heavy churn, long placeholder timeouts hold resources that could be used by other jobs. Tune this conservatively and observe whether gang satisfaction rates change.

PodGroup minimum vs desired (Volcano). Volcano lets you set minMember below the actual desired replica count. This is useful for elastic training frameworks like TorchElastic that can run with fewer workers, but it requires careful thought: if your job starts with 60% of its desired workers because the rest are queued, the effective throughput may not justify the GPU-hours. For most production training, set minMember equal to your desired replica count unless you have verified elastic training semantics.

Cost visibility with GPU batch scheduling. When jobs are gang-scheduled, you need your Kubernetes cost visibility tooling to account for idle-but-reserved pods. A PodGroup that has reserved GPUs but is waiting for the final member to be placed will show up as GPU spend even though no useful work is happening. Tag your jobs with team and project labels from the start; retrofitting cost attribution in a running queue system is painful.

Integration with node autoscaling. Both Volcano and YuniKorn integrate with Karpenter and the Cluster Autoscaler, but gang scheduling and autoscaling interact in non-obvious ways. If Volcano creates a PodGroup and the cluster does not have enough nodes to satisfy the gang, it needs to signal the autoscaler to provision the right nodes before the gang timeout expires. With Karpenter, this works well through the pending pod mechanism. With the Cluster Autoscaler, you need to ensure your NodeGroups are configured to scale up quickly enough for the gang timeout.

For deeper coverage of GPU cloud infrastructure and how it interacts with your scheduling layer, read our dedicated guide. And for the cost side of AI infrastructure, batch job scheduling is one of the highest-leverage places to reduce waste: a well-configured scheduler that prevents partial gang starts can recover a meaningful percentage of GPU budget that would otherwise be burned on idle reservations.

A War Story

In my second year running a shared ML platform (somewhere around 2008, in what was then called a “high-performance computing cluster” before Kubernetes existed), we had a gang scheduling failure that cost an entire weekend of compute. A distributed simulation job requiring 128 nodes started on 127 because one node had a bad memory DIMM and its heartbeat was delayed. The job ran for 52 hours before the team noticed it was making no progress. No checkpoint, no gang enforcement, no budget ceiling.

Kubernetes and modern batch schedulers are a different world. But the failure mode is the same: partial gang execution, or worse, deadlocked gang reservation, will drain your GPU budget silently while everyone assumes the jobs are making progress. The tools exist to prevent this. Using them correctly is an operational discipline, not just a deployment checklist.

Volcano and YuniKorn are not glamorous additions to your stack. They do not make demos well. But in a shared AI platform with multiple teams and hundreds of daily training jobs, the difference between a properly configured batch scheduler and a bare kube-scheduler is the difference between a platform that people trust and one that leads to shadow queues and wasted compute.

Summary

The default Kubernetes scheduler is not the right tool for AI batch workloads. Gang scheduling, hierarchical fairness, and multi-tenant resource isolation require purpose-built components.

Volcano, a CNCF incubating project with active releases in 2026 (v1.14.x and newer), is the choice for teams that need battle-tested gang scheduling across mixed AI and big data workloads, with mature job lifecycle policies and a large ecosystem of framework integrations.

Apache YuniKorn 1.8.0, an Apache Top-Level Project with particular strength in the Apache ecosystem and AWS EMR on EKS support, is the choice for multi-tenant platforms where hierarchical resource guarantees and department-level fairness policies are central requirements.

For teams at earlier stages or with simpler requirements, Kueue offers a lighter-weight alternative that does not require scheduler replacement. Understanding what each of these tools does, and where the boundaries of each design live, is increasingly core infrastructure knowledge for anyone building production AI platforms on Kubernetes.