Diagnosing Argument List Overflows in Cloud Mac CI

After a large iOS repository has been running on a cloud Mac for several weeks, one branch may suddenly cause a script to fail with argument list too long, while adjacent branches continue to archive successfully. Do not immediately clear DerivedData or dismiss the issue as an intermittent Xcode failure. This error usually comes from execve returning E2BIG: command arguments, the terminators for each argument, and the inherited environment together exhaust the space available to the process.

First identify which layer is failing

The first step is to preserve the complete command and its exit status. If the log only reports that a script phase failed, temporarily enable set -x in the corresponding Run Script to determine whether the failure occurs in find, rm, an archiving tool, a code generator, or the compiler driver. Do not print the entire environment to a public log, because it may contain tokens. Record only variable names, byte counts, and a redacted representation of the command structure.

There are three common triggers: a wildcard expands into tens of thousands of paths at once; a script concatenates every source file into one variable; or CI injects large JSON documents, certificate contents, or multiline configuration as environment variables. Long workspace paths and repeated -I, -F, and -D arguments also steadily consume the remaining headroom.

A command that works on a small branch is not necessarily structured correctly. If it starts failing only as the file count grows, the input-passing mechanism usually lacks a designed upper bound.

Measure argument and environment headroom

On macOS, the usable limit cannot be determined from the command text alone. Start by collecting measurements under the same user account and launch method as the build job:

getconf ARG_MAX

python3 - <<'PY'
import os
limit = os.sysconf("SC_ARG_MAX")
env_bytes = sum(len(k) + len(v) + 2 for k, v in os.environ.items())
largest = sorted(
    ((len(k) + len(v) + 2, k) for k, v in os.environ.items()),
    reverse=True
)[:10]
print("arg_max", limit)
print("environment_bytes", env_bytes)
for size, key in largest:
    print(size, key)
PY

Use these results for comparison, not as a budget that can be consumed in full. The system also needs space for pointers, terminators, and process startup overhead. In practice, leave substantial headroom and watch for sudden increases in environment size relative to the historical baseline.

Find oversized variables

Check PATH, search paths, temporary directories, dependency-manager arguments, and CI-injected variables first. If a JSON configuration reaches tens of KB, write it to a permission-controlled temporary file and pass only the file path to the child process. When extending PATH repeatedly, deduplicate it first instead of concatenating it again at every step.

Reduce the inherited environment without breaking the toolchain

Do not launch a complete Xcode build with an empty environment. Missing values for HOME, PATH, temporary directories, or the developer directory will create new failures. A safer approach is to define an allowlist for an individual tool:

env -i \
  HOME="$HOME" \
  PATH="/usr/bin:/bin:/usr/sbin:/sbin" \
  TMPDIR="$TMPDIR" \
  DEVELOPER_DIR="$DEVELOPER_DIR" \
  /bin/zsh -lc 'xcrun --find xcodebuild'

Production jobs should also include locale settings, cache directories, and proxy configuration according to their actual dependencies. Pass sensitive content through short-lived files and delete them after use. Ordinary Boolean flags and short identifiers can remain as environment variables. Automation jobs on SetMini should likewise centralize environment initialization in a single entry script, preventing the interactive shell, CI runner, and Xcode scripts from each appending the same configuration.

Move large file sets out of the command line

Use null-delimited batches in general-purpose scripts

File names may contain spaces or newlines, so do not use for f in $(find ...). Tools that support batch processing can be combined with find -print0 and xargs -0:

find "$PWD/Artifacts" -type f -name '*.dSYM' -print0 |
  xargs -0 -n 50 /usr/bin/file

The -n 50 option sets an explicit upper bound for each batch. If the tool can read a file list from standard input, prefer that approach to avoid repeatedly starting new processes.

Use file lists in Xcode phases

Run Script inputs and outputs should be written to .xcfilelist files and configured under Input File Lists and Output File Lists. This allows Xcode to track dependencies without requiring the script to expand thousands of paths into a single command. When a compiler or linker explicitly supports response files, stable arguments can be moved into one. Do not assume that every third-party tool understands the @file syntax.

For large sets of options supplied through build settings, shared values should go into an .xcconfig file. Scripts should receive only a small number of required arguments, path collections should be passed through file lists, and structured configuration should be supplied through temporary files. Separating these three input categories also makes logs easier to audit.

Add growth gates and regression checks

After applying the fix, run at least one clean build using the branch with the most files and the longest paths, followed by an incremental build. Verify that the script handles file names containing spaces, that batch failures return a nonzero status, and that temporary files are removed along every exit path.

At the start of a CI job, you can record ARG_MAX, the environment’s total byte count, and the names of the largest variables, but not their values. Establish a team baseline for environment size and issue a warning when it exceeds a threshold instead of waiting for the system to reject process creation. For file collections that grow over time, record both the file count and the length of the longest path.

The goal is not to keep the current command barely below the limit. Its length should no longer grow linearly with the size of the repository. When the environment stays small, file collections travel through dedicated channels, and batch processing has fixed upper bounds, E2BIG changes from an intermittent failure into a configuration issue that can be detected in advance.

Frequently asked questions