How to Pull Multi-Gigabyte Container Images in Seconds on Amazon EKS

A standard Docker pull on a 10 GB AI inference image on Amazon EKS can take six to seven minutes before the pod starts. With SOCI (Seekable OCI) and lazy loading, the same pod starts in under 30 seconds. The image is still transferring — but the container doesn’t need it all upfront.

The Framework

Seekable OCI is a lazy-loading snapshotter for container images developed by AWS and integrated natively into Amazon EKS via containerd. The principle: a container typically only needs a subset of the image layers to start. SOCI indexes those layers and serves them on demand via range requests, allowing the runtime to begin execution while the remainder of the image pulls in the background.

The performance gains compound with image size. On a 1.3 GB Python web service, SOCI reduces cold-start pull time from 20 seconds to approximately 2.8 seconds — a 7.4x speedup. On a 2.5 GB image, the speedup reaches 9.3x. For the 10–20 GB images common in AI inference deployments (model weights bundled into the image layer), the reduction from minutes to seconds is the difference between a usable autoscaling system and one that can’t respond to demand spikes fast enough to matter.

Step 1 — Index Your Images with SOCI

SOCI requires a SOCI index alongside the container image in ECR. The index is a separate artifact in the same repository that maps the image’s layer structure for on-demand range requests. Build and push the SOCI index using the soci CLI:

soci push --existing-image <ecr-repo>/<image>:<tag>

This adds the index to ECR without modifying the image itself. Existing images work; rebuilding is not required.

Step 2 — Enable the SOCI Snapshotter on EKS Node Groups

The SOCI snapshotter integrates with containerd as a plugin. On EKS managed node groups, enable it via a DaemonSet or as part of the AMI bootstrap configuration. AWS provides a reference DaemonSet that installs the snapshotter and configures containerd to route image pulls through it when a SOCI index is present.

Nodes without the snapshotter installed fall back to standard image pulling — the SOCI index is ignored rather than causing errors. Gradual rollout across node groups is safe.

Step 3 — Validate Cold-Start Improvement

Measure the time from pod scheduling to container ready. Compare a node without SOCI to a node with SOCI against the same image. The speedup is most pronounced on large images and on nodes where network bandwidth is shared across concurrent pulls — a common condition during horizontal scaling events.

Production deployments at AppsFlyer using SOCI with parallel pull mode reported 82% reductions in GitHub Actions runner cold-start times on EKS — the compound effect of lazy loading plus parallel layer fetching.

Where It Doesn’t Apply

SOCI delivers the largest benefit on images where startup requires only a fraction of the total image content. Tightly packed images where nearly all layers are needed to start the container see smaller speedups. For images under 1 GB, standard pull times are already fast enough that SOCI adds operational complexity without proportional return.

SOCI also requires ECR or an OCI-compatible registry that supports range requests on image layer blobs. Registries that serve layers as monolithic objects need modification to support the protocol.

Source: The New Stack

Content created with AI assistance and reviewed for accuracy.