`docker exec` is flaky in Docker in gVisor

[EtiennePerot]

Description

Sequential docker execs in gVisor fail at ~0.3% rate, with:

time="2026-05-12T19:11:49.104637739Z" level=error msg="Error running exec a7fb1832daf4e4ebd8eadbe8451036160b5ba58a82463c482e79f65727f109c3 in container: OCI runtime exec failed: exec failed: unable to start container process: reading from parent failed: fetch packet length from socket: interrupted system call: unknown"

LLM-generated potential root cause blames both containerd's EINTR handling and gVisor's bubbling up of it. Can be worked around by setting GODEBUG=asyncpreemptoff=1.

The Failure: If Go's runtime wraps the EINTR error (e.g., in a syscall.Errno type vs. unix.Errno type across package boundaries, or wraps it in os.SyscallError), the direct comparison err != unix.EINTR evaluates to true. This causes the loop to break immediately, returning EINTR as a fatal process startup failure.

The Transparency Defect in gVisor (Sentry)

• The Cause: Since Go 1.14, the Go runtime uses SIGURG signals for asynchronous goroutine preemption (forcing threads to yield).
• Syscall Interruption: When a Sentry thread is blocked emulating the container's blocking recvfrom call during process bootstrap, a SIGURG signal sent to that thread interrupts the syscall, returning a host-level EINTR.
• Transparency Leak: gVisor's Sentry is designed to be a transparent guest kernel. It must transparently catch and restart (ERESTARTSYS) all internal scheduler preemption interrupts.
• The Bug: Under high-throughput sequential execution, the Sentry occasionally fails to restart/hide these preemption signals, leaking EINTR directly back to the guest process (runc).
• The Collision: Once EINTR leaks to the guest, it collides with runc's direct-comparison bug, causing the docker exec process to abort instantly. This explains why the error is heavily observed under gVisor sandboxes (~0.2%) but completely immune under standard Linux kernels (0 failures).

🛡️ Comprehensive Go Preemption Mitigation: GODEBUG=asyncpreemptoff=1
Following deep-dive research and multi-dimensional stress testing across standard nodes and powerful compute-optimized sandboxed nodes (c4-standard-48), we have successfully engineered and validated a highly robust, application-level, zero-code mitigation that structurally eliminates the transient EINTR socket bootstrap timing race:

By setting the environment variable GODEBUG=asyncpreemptoff=1 inside the Docker-in-Docker container/pod.

[nicolasalarconrapela]

@EtiennePerot Markdown's writing fails. If you can review it, you are grateful.

[Amaindex]

I opened a draft regression-test-only PR for a reduced syscall-level behavior that appears relevant to this issue: #13211

I was not able to reproduce the full Docker-in-gVisor docker exec flake locally: 2000 sequential inner docker exec calls passed, and another 1200 calls under CPU stress also passed before I stopped the run.

However, a narrower reproducer was stable. On Linux/runc, a blocking recvfrom(fd, NULL, 0, MSG_TRUNC | MSG_PEEK, NULL, NULL) on a Unix SOCK_SEQPACKET socket is restarted when interrupted by a signal whose handler has SA_RESTART, and then returns the packet length once data is sent. Under runsc, the same syscall shape returns -1/EINTR.

The new test is expected to fail under runsc before a fix. It captures that reduced Linux compatibility difference, but does not prove that the full docker exec flake always comes from this path. It covers the same recvfrom packet-length probe shape used by runc's sync socket. Does this match the syscall behavior you expected from the original report?

[milantracy]

thanks @Amaindex #13211 looks valid to me.

I sent the fix at #13212, once that merged, can you send your PR?