Running tests in parallel
Run many Selenium tests at once, keep the results together, and know what happens when you ask for more than you have.
Running tests one after another is the slowest thing you can do with a grid. Going parallel needs nothing special from us — your test runner already knows how, and each worker opens its own session.
Two things to get right: turning workers on, and keeping the results together.
1. Turn on workers
TestNG, in testng.xml:
<suite name="smoke" parallel="methods" thread-count="5">
<test name="checkout">
<classes>
<class name="tests.CheckoutTest"/>
</classes>
</test>
</suite>Mocha:
npx mocha --parallel --jobs 5 "test/**/*.spec.js"pytest, with pytest-xdist:
pip install pytest-xdist
pytest -n 5NUnit, in AssemblyInfo.cs:
[assembly: Parallelizable(ParallelScope.Fixtures)]
[assembly: LevelOfParallelism(5)]RSpec, with the parallel_tests gem:
gem install parallel_tests
bundle exec parallel_rspec spec/ -n 5PHPUnit, with paratest:
composer require --dev brianium/paratest
vendor/bin/paratest -p5Give every worker its own driver. Workers are separate threads or processes. One driver shared between them will not survive — build it inside the test, or in a fixture that runs per worker.
2. Give every worker the same runId
Five workers means five sessions. Without a runId that is five separate runs in your dashboard, which tells you nothing about whether your tests passed.
Use a value your CI already has — GITHUB_RUN_ID, BUILD_NUMBER, CI_PIPELINE_ID. Locally, a timestamp does:
| Language | The line |
|---|---|
| Java | tk.put("runId", System.getenv("GITHUB_RUN_ID")); |
| JavaScript | runId: process.env.GITHUB_RUN_ID |
| Python | "runId": os.environ["GITHUB_RUN_ID"] |
| C# | ["runId"] = Environment.GetEnvironmentVariable("GITHUB_RUN_ID") |
| Ruby | "runId" => ENV["GITHUB_RUN_ID"] |
| PHP | "runId" => getenv("GITHUB_RUN_ID") |
build does not group anything. It is a label. Sessions that share a build but no runId are still separate runs.3. How many at once
| Limit | |
|---|---|
| Paid plans | One session per thread, 1 to 20 threads |
| Free tier | No thread limit — your remaining free minutes are the only ceiling |
| Over your limit | Waits for one of your threads to free up, then refuses if none does |
Going over your limit no longer fails straight away. The session waits for one of your own threads, and only gives up if none comes free in time:
Waited, but all 5 of your parallel sessions were busy the whole time.
Close one, or raise your plan's parallel limit.So a suite that starts more workers than you have threads will mostly still finish — the extra ones wait their turn. It is slower than sizing the run correctly, though, so keep your worker count at or below your threads.
How long it waits is the same maxQueueWaitSeconds described below, and the
two waits share it: time spent waiting for a thread is time not left for
waiting for a machine.
4. Why a session still waits sometimes
Being under your own limit does not mean a machine is free. Our machines have their own limit, shared with everyone, and you wait when every machine matching your request is busy. That wait is maxQueueWaitSeconds — 2 minutes 50 seconds by default on Selenium, up to 9 minutes — and it is not billed.
Two things help more than raising that number:
Ask for a mix. Five workers all asking for one macOS version and Safari draw on a handful of machines. The same five across Chrome, Edge and Firefox on two operating systems draw on many more.
Leave room for cleaning up. A session that is still ending still holds its machine. Fire a second run immediately at the same narrow target and it queues behind the run before it.
The full picture — thread bands, per-machine ceilings, free minutes — is in Reference.
Do you also need to mark tests?
Usually not. With workers, each session runs one test, so the session name is enough to read the run at a glance.
Marking tests is for the other shape — one driver running several tests in a row.