Animations Off and Parallel Simulators: Measured XCUITest Speed-Ups

Two speed-ups appear in almost every guide to XCUITest performance: turn animations off, and run tests in parallel. Both work. This part explains what each one actually changes, how to implement it so that it does not leak into the product or the developer's machine, how to measure a saving that is close to the size of run-to-run noise, and where the ceiling on parallelism comes from. It begins with a correction to part 4.

Where wait-for-idle time comes from

Before every synthesised event, XCUITest waits for the app to become idle. On a surface that is animating, that wait can run to its 60 second limit while the app remains fully responsive. Part 4 attributed this cost to app.activate() when called inside a polling loop. The advice there (keep activate next to the tap it serves) still holds, but the attribution was wrong. Measured across 168 samples, activate() on an idle app costs about 0.01 seconds. The idle wait attaches to whichever event comes next, whatever helper issues it: on the method that had shown the stall, skipping the activate moved the cost to the following click, which paid 51.3 seconds where the activate had paid 23.6.

Two consequences follow. A fast path that skips activate when the app is already frontmost yields nothing measurable (in this suite the app was frontmost at all 167 calls), although it is useful as accounting, since it can record how many activations each test paid. And the only thing that removes the idle cost is the app not animating, which is what the next section is about.

Disabling animations from inside the app

The test runner cannot switch off an app's animations; only the app can. The arm therefore lives in the app, behind the same UI-testing launch argument the harness already passes, and is compiled only into debug builds. On iOS two in-memory settings cover most of SwiftUI and UIKit: a root transaction as the outermost modifier on the window group's content, and UIView.setAnimationsEnabled(false), set from a stored-property sentinel so that it runs before the first scene is built.

#if DEBUG
extension View {
    /// Outermost modifier on the WindowGroup's content root. Process-scoped:
    /// nothing is written to UserDefaults, so nothing outlives the test process.
    @ViewBuilder func uiTestingAnimationsOff() -> some View {
        if UITestingAnimationsGate.isArmed {
            self.transaction { $0.animation = nil; $0.disablesAnimations = true }
        } else {
            self
        }
    }
}

enum UITestingAnimationsGate {
    /// A pure function, so it can have a unit-test truth table.
    static func shouldArm(isUITesting: Bool, idiom: UIUserInterfaceIdiom) -> Bool {
        isUITesting && (idiom == .phone || idiom == .pad)
    }
    static let isArmed: Bool = {
        let armed = shouldArm(
            isUITesting: ProcessInfo.processInfo.arguments.contains("--uitesting"),
            idiom: UIDevice.current.userInterfaceIdiom)
        if armed { UIView.setAnimationsEnabled(false) }
        return armed
    }()
}
#endif

On macOS three settings are needed: the SwiftUI root transaction on every content root, NSWindow.animationBehavior set to none from an initialiser hook, and three AppKit animation defaults supplied through the volatile argument domain, which means as launch arguments rather than as stored preferences.

What the arm does not reach

• Modal presentation and dismissal. In this app that is 326 sheet sites and 9 full-screen covers.
• The keyboard's own window, system alerts and permission prompts, and UIScrollView deceleration.
• Timers. A product timer is not an animation and should not be touched.

Do

• Gate the arm three ways: debug build, the UI-testing launch argument, and the device idioms it has been measured on.
• Give the gate a pure function and a unit-test truth table. Neither UI lane can distinguish an arm that armed on every idiom from one that armed only on its own.
• Verify the arm is present in a build with nm and a symbol count, since a wrongly gated arm produces the same binary size and md5 story as a correct one.

Don't

• Don't emulate Reduce Motion. It is product behaviour that users can enable; in this app the onboarding carousel changes under it, so every onboarding capture would change its expected state.
• Don't persist anything, and don't remove a product animation to satisfy the harness.

Measuring a small saving: the ABA run

An animation arm saves a few percent, which is about the size of run-to-run noise on a UI-test suite. A single before/after pair cannot separate the two. The approach that can is to fix the acceptance rule before the first run and use three runs of the heaviest class: arm off (A), arm on (Aft), arm off again from a rebuilt tree (B). The saving is the mean of A and B minus Aft; the noise band is the larger of the A-to-B difference and 2 percent of that mean; the change lands only if the saving exceeds the band and the outcome sets are identical across all three runs.

heavy class, 12 methods, summed per-test seconds
  S_A    1,076.468   arm off
  S_Aft    994.668   arm on
  S_B    1,078.901   arm off, rebuilt
mean(A, B) = 1,077.685
saving     =    83.017 s  (-7.70 %)
band       = max(|A - B| = 2.433, 2 % of mean = 21.554) = 21.554
saving / band = 3.85   outcome sets identical on all three  ->  land
negative control (no login, no typing): +2.1 %, no saving, as designed

The mechanism is checked in the activity tree, not in the wall clock, and there is a trap in how to read it. On the method that moved most, the number of Wait for app to idle nodes was 133 on all three runs; the arm removes no waits. Their summed duration went 29.08, 8.93 and 28.87 seconds across A, Aft and B, which accounts for 98.6 percent of that method's change, while the launch span that should not move read 2.97, 2.96 and 2.97 seconds. An acceptance keyed on node count would have reported no change.

Where the saving lands is also worth knowing for estimating: not on the keyboard, which contributed about 2 percent, but on navigation and commit steps such as pushes, sheet transitions and save confirmations. To estimate what the arm is worth for a different suite, count taps and navigations rather than typeText calls. On the iPad, the same ABA method landed a smaller saving, 10.169 seconds against an 8.682 second band, 2.34 percent of the class. On macOS the measured yield was the window-transition waits only: idle-wait time on a pooled control fell from 6.9 to 2.3 seconds, about a second per method, and waits that were real work (a configuration window taking 7.5 seconds) did not move.

Do

• Write the acceptance rule and the band before the first run.
• Measure the arm by idle-wait duration, costed to the event that follows each wait, and keep a span that should not move as a within-run control.
• Include a negative control class with no login and no typing; it should show no saving.
• Discard a triple in which any run overlapped foreign load or an unrelated intermittent failure, and say so. Two triples were discarded on the iPad for those reasons.

Don't

• Don't accept a few-percent change on a single before/after pair.
• Don't count idle-wait nodes as the mechanism check.

Parallel simulator clones

xcodebuild can clone a simulator and distribute test classes across the clones with -parallel-testing-enabled YES and a worker count. The limiting factor is rarely the simulator; it is shared mutable state. If several classes sign in to the same test tenant, two workers on that tenant interfere with each other. Where per-worker tenants do not exist, the suite has to be partitioned by shared state: a set of classes that is safe in parallel, run in one invocation with clones, followed by the classes that share state, run serially. The two invocations run one after the other, never at the same time.

# A: the parallel-safe set, three clones of the pinned simulator
xcodebuild test-without-building -xctestrun "$RUN" \
  -destination "platform=iOS Simulator,id=$UDID" \
  -parallel-testing-enabled YES -parallel-testing-worker-count 3 \
  $(printf -- '-only-testing:UITests/%s ' "${PARALLEL_SAFE[@]}")

# S: the shared-state set, serially, after A has finished
xcodebuild test-without-building -xctestrun "$RUN" \
  -destination "platform=iOS Simulator,id=$UDID" \
  $(printf -- '-only-testing:UITests/%s ' "${SHARED_STATE[@]}")

In this suite the iPhone target of 88 classes split into 47 parallel-safe and 41 shared-state classes. With the adoption threshold fixed in advance at 0.85 of a fresh serial run, the pair finished in 7,195.364 seconds against 8,989.468, which is 0.800, minus 20.0 percent, with three workers. Two workers also qualified at 0.846. The pass, skip and fail sets were identical to the serial run by test id: 167, 60 and 1.

The ceiling is set by the serial half. Here it was 85 percent of the pair's wall clock, so the pair cannot go below 0.788 of serial at any worker count. The next gain is per-worker tenants, not a fourth worker.

Do

• Partition by each class's resolved login identity, walked through the superclass chain to the actual account. Two classes spelling the same expression can resolve to different tenants, and a label in a manifest is not the identity.
• Install the app once, onto the pinned base simulator. A clone carries the app that is installed on its source, so there is no per-worker install.
• Read the worker count back from the run log. On Xcode 26.6 the result bundle's device list names only the base simulator even on a genuinely parallel run.
• Boot the base simulator again before the next run; a parallel run leaves it shut down when it tears its clones down.

Don't

• Don't run the parallel and serial invocations concurrently.
• Don't put a class that changes global device state (language, orientation) in the parallel set. In this suite the one class that flips the app language sits in the serial set, and a run before the split had lost twelve tests to that leak.
• Don't compare the wrapper's wall clock across runs if it includes the test-bundle build; compare the testing-elapsed figure.

Checklist

1. Attribute idle-wait cost to the event that follows it, not to activate().
2. Put the animation arm in the app: debug-only, launch-argument gated, idiom gated, process-scoped, never persisted.
3. Measure it with an ABA run and a pre-registered band, by idle-wait duration.
4. Partition parallel work by shared state, run the two halves sequentially, and accept on identical outcome sets.
5. Look at the serial half for the next gain; it sets the ceiling.

Previously (Part 9): Negative Waits: The Seconds Every Green XCUITest Run Burns

Next up (Part 11): The XCUITest Speed-Ups We Measured and Dropped