Reuse SSH Connections Safely in Cloud Mac CI

When a CI orchestrator repeatedly checks versions, pulls code, starts builds, and collects artifacts from a cloud Mac, opening a new SSH connection for every command repeats key exchange, authentication, and session initialization each time. The latency of one connection may not be noticeable, but across dozens of short commands, handshake time can become a significant part of the total runtime. SSH's built-in ControlMaster feature lets later commands reuse an authenticated TCP connection, provided that socket isolation, stale-connection detection, and cleanup on exit are handled correctly.

Identify the optimization target

Connection reuse is best suited to cases where the same CI job accesses the same Mac repeatedly over several minutes. If a job runs only one long build, reducing handshake overhead offers limited benefit. If the orchestrator runs a dozen or more short commands in sequence, reuse is usually much more valuable.

Start by recording the connection process without reuse:

time ssh -o BatchMode=yes build-mac 'sw_vers -productVersion'
ssh -vv -o BatchMode=yes build-mac 'true' 2>ssh-debug.log

Check the debug log for repeated key exchange and authentication stages. Do not rely on the incidental timing of a single command. Run the test several times in succession, and separate connection setup time from remote command execution time.

ControlMaster reduces connection setup overhead. It does not make Xcode compilation, dependency resolution, or file transfers themselves any faster.

Configure minimal connection reuse

Create a dedicated alias for the build host in ~/.ssh/config on the CI orchestrator. %C hashes the connection parameters, preventing Unix Socket path failures caused by long user names or host names.

Host build-mac
    HostName mac.example.internal
    User ci-runner
    BatchMode yes
    ControlMaster auto
    ControlPersist 10m
    ControlPath ~/.ssh/control/%C
    ConnectTimeout 15
    ServerAliveInterval 30
    ServerAliveCountMax 3
    StrictHostKeyChecking yes
    UserKnownHostsFile ~/.ssh/known_hosts_ci

Restrict permissions when preparing the directory:

install -d -m 700 "$HOME/.ssh/control"
chmod 600 "$HOME/.ssh/known_hosts_ci"
ssh build-mac 'printf "%s\n" ready'
ssh -O check build-mac

The first command creates the master connection, and ssh -O check should return the master process status. ControlPersist 10m means the master connection can remain open for up to ten minutes after the last session ends. It does not mean that a job can depend on the connection indefinitely.

Pin the host identity

Do not set StrictHostKeyChecking=no for convenience. Obtain the host's public key through a controlled channel, add it to the CI-specific known_hosts file, and use a clearly defined update process when rebuilding a node or rotating keys. Connection reuse only eliminates subsequent handshakes; it does not change the trust boundary of the first connection.

Isolate sockets for parallel jobs

One of the most common problems on a shared Runner is allowing every job to use ~/.ssh/control/%C. If two pipelines connect to the same host at the same time, one may reuse the master connection created by the other. If either job then closes that connection, it will interrupt the other job as well.

A safer approach is to create a separate directory for each job and override ControlPath on the command line:

set -euo pipefail

job_id="${CI_JOB_ID:-local-$$}"
control_dir="${TMPDIR:-/tmp}/ssh-control-${job_id}"
install -d -m 700 "$control_dir"

ssh_opts=(
  -o ControlMaster=auto
  -o ControlPersist=10m
  -o "ControlPath=${control_dir}/%C"
  -o BatchMode=yes
)

ssh "${ssh_opts[@]}" build-mac 'xcodebuild -version'
ssh "${ssh_opts[@]}" build-mac 'git --version'

The job identifier must come from a trusted pipeline context and be limited to a short string. Do not insert a branch name directly into the path, because slashes, spaces, and excessively long names can prevent socket creation.

Detect stale connections and rebuild them safely

After a network change, remote restart, or unexpected exit of the control process, the socket file may remain even though its connection no longer works. Check the connection before invoking the actual command. If the check fails, remove only the socket directory for the current job, then establish a new connection.

if ! ssh "${ssh_opts[@]}" -O check build-mac >/dev/null 2>&1; then
  rm -rf "$control_dir"
  install -d -m 700 "$control_dir"
  ssh "${ssh_opts[@]}" build-mac 'true'
fi

Only delete the directory created by the current job. Never clean a shared directory with rm -rf ~/.ssh/control/*. Avoid retrying build commands unconditionally as well: if the connection drops, the remote command may already have started. The safe approach is to retry only idempotent checks automatically, then determine the original job's status from its process, lock file, or build number.

Watch the server-side session limit

A master connection can carry multiple logical sessions, but not an unlimited number. If too many commands run concurrently, the server may reject new channels. First limit SSH concurrency within each job, then assess whether the server-side policy needs adjustment. Do not treat a higher session limit as the default fix.

Clean up deterministically on exit

On a normal exit, close the master connection with the control command. If the job exits unexpectedly, its directory must still be removed. A shell script can handle both cases consistently with trap:

cleanup() {
  ssh "${ssh_opts[@]}" -O exit build-mac >/dev/null 2>&1 || true
  rm -rf "$control_dir"
}
trap cleanup EXIT INT TERM

Final validation should cover four points: consecutive commands actually use the same master connection; two parallel jobs use separate directories; the connection can be rebuilt after the remote host restarts; and no sockets remain after a job is canceled. Automation jobs on SetMini should follow the same principles: confirm the currently available configurations in the console, then treat the connection layer as infrastructure that can be checked, rebuilt, and cleaned up—not as hidden state that remains valid indefinitely.

Frequently asked questions