How we get a CI pipeline under ten minutes

A slow pipeline rarely gets fixed because nobody owns the problem. Each engineer loses twenty minutes here and there, switches context while waiting, and comes back to a failure they no longer remember causing. Multiply that by a team of eight and a pipeline that takes forty minutes, and you are spending more than a full engineer's week, every week, staring at progress bars. Getting CI under ten minutes is one of the highest-return infrastructure jobs we take on, and it rarely requires anything exotic.

Measure before you optimize

Start by pulling the last month of pipeline runs and breaking each one into stages: checkout, dependency install, build, lint, unit tests, integration tests, image build, deploy. Most CI providers expose this through an API or export. You are looking for three things: the stage that dominates wall time, the variance between fast and slow runs, and the flaky jobs that trigger reruns.

On a recent project for a SaaS client, the breakdown for a typical 38-minute run looked like this: 9 minutes installing dependencies, 4 minutes building front-end assets, 21 minutes of tests running serially, and 4 minutes building and pushing a container image. Nothing surprising, but seeing the numbers told us exactly where to spend effort.

Cache dependencies properly

Dependency installation is the cheapest win. Every package manager supports a lockfile-keyed cache, but pipelines often cache the wrong directory or key the cache on something that changes every commit.

• Key the cache on the hash of the lockfile, not the branch name.
• Cache the package manager's download store, not just the installed folder, so partial hits still help.
• Provide a fallback key so a changed lockfile still restores most packages.
• For Composer, npm and pip alike, use the non-interactive, frozen-lockfile install mode so CI never resolves versions on its own.

That alone took the client's install step from nine minutes to under one on a warm cache. Front-end asset builds benefit from the same treatment: tools like Vite and webpack have persistent caches that can be stored between runs.

Split tests across parallel runners

Serial test execution is where most of the time goes. Almost every mature test runner can shard a suite across multiple machines, and CI platforms can spin up several identical jobs from a single definition. The trick is splitting by historical timing rather than by file count, so that one shard does not get all the slow integration tests.

test:
  parallel: 6
  script:
    - ./ci/split-tests --index $CI_NODE_INDEX --total $CI_NODE_TOTAL --timings timings.json
    - ./vendor/bin/pest $(cat shard.txt) --log-junit report.xml

Six shards turned 21 minutes into about four. Parallelism costs more compute minutes in total, but compute is cheap compared to engineering time. For that client, the extra runner minutes added roughly 90 dollars a month.

Compute minutes are the cheapest resource in the building. Engineers waiting on them are the most expensive.

Parallel tests also expose hidden coupling. If tests share a database and depend on insertion order, they will fail intermittently once split. Fix those by giving each shard its own database or wrapping each test in a transaction. It is tedious work, but it pays back in reliability.

Build images in layers that stay cached

Container builds are slow when the Dockerfile copies the whole repository before installing dependencies, which invalidates every layer on every commit. Reorder so that the lockfiles are copied and dependencies installed first, then the application code. Use multi-stage builds so the final image contains only runtime files, and enable the registry-backed build cache so ephemeral runners can reuse layers from previous builds.

We also stopped building an image on every push to every branch. Images are built for the main branch and for pull requests that touch deployment configuration. Feature branches run tests against the code directly, which is faster and catches the same bugs.

Decide what belongs before merge

Not every check needs to block a pull request. We sort checks into three tiers:

1. Blocking, fast. Lint, type checks, unit tests, and a focused set of integration tests covering checkout, sign-in and billing. These must finish in under ten minutes.
2. Blocking on main. The full integration and end-to-end suite runs after merge and before deploy to production. A failure stops the release, not the developer.
3. Scheduled. Dependency audits, visual regression across many browsers and load tests run nightly and open tickets rather than blocking anyone.

This tiering is a judgment call and should be written down. Teams that skip it end up with a pipeline where every new check lands in the blocking tier and the ten-minute budget erodes within six months.

Kill the flaky tests

A test that fails one run in twenty costs more than its runtime. Engineers rerun the job, lose trust in red builds and start merging over failures. We track flake rate per test, quarantine anything above two percent into a non-blocking job, and give the owning team a week to fix or delete it. A smaller suite that everyone trusts beats a larger one that nobody believes.

Where the client ended up

After three weeks of work, the median pull request pipeline dropped from 38 minutes to 8, and the ninety-fifth percentile from 55 to 12. Deploy frequency roughly doubled over the following quarter, not because anyone asked for it, but because shipping became cheap. If you want to see how this fits a wider release process, our CI/CD pipelines service covers the full path from commit to production, and we often pair it with staging environments that let the slow suites run against realistic data.

Get your pipeline back under ten minutes

Send us a recent pipeline run and a rough picture of your stack. We will tell you where the time is going and quote a fixed price to fix it, usually within 24 hours. Request your quote.