Where CI/CD Pipelines Actually Lose Time (It Is Rarely the Build Step)

Ask most engineers where their pipeline is slow and they will point at the build step. It is the part they watch scroll by, so it feels like the bottleneck. When we actually measure pipelines end to end, the build step is frequently not the biggest cost at all. Queue time, flaky test retries and serialized stages that did not need to run in sequence usually add up to more.

Queue time is invisible until you measure it separately

Most CI dashboards show how long a job took to run, not how long it waited before starting. If your runners are frequently saturated during peak hours, a pipeline that runs in six minutes can still take twenty minutes wall-clock time to actually finish, and that twenty minutes is what an engineer experiences while waiting for feedback. This is one of the simplest things to measure and one of the most commonly skipped, because the number does not show up unless you specifically instrument for it.

• Track time-to-start separately from time-to-complete for every job
• Watch for patterns tied to specific hours or days, which usually point at runner capacity, not job design
• Consider autoscaling runners or a larger fixed pool before optimizing job internals if queue time dominates

Serial stages that have no real dependency between them

Pipelines grow incrementally, and stages often get added in whatever order made sense at the time, then never get revisited. Lint, unit tests, and a security scan frequently run sequentially even though none of them depend on the others' output. Running independent stages in parallel is one of the highest leverage changes available, and it usually requires no infrastructure change at all, only a reorganized pipeline definition.

Flaky tests cost more than their retry time suggests

A test that fails intermittently and gets automatically retried looks like a minor inconvenience in a dashboard. In practice, flaky tests train engineers to ignore failures, which is far more expensive than the extra minutes spent retrying. Once a team develops the habit of re-running a failed pipeline without investigating why, a genuine failure hiding among the noise takes much longer to notice.

Artifacts get rebuilt instead of promoted

A common but costly pattern is rebuilding the application separately for each environment: build once for staging, build again for production. Beyond the wasted time, this means the artifact tested in staging is not strictly the same one that reaches production, which undermines the entire point of staging as a verification step. Building once and promoting the same artifact through environments is both faster and closer to what teams actually believe they are testing.

Dependency installation happens from scratch on every run

Package installation is one of the most cache-friendly parts of a pipeline and one of the most frequently left uncached. Whether it is a language package manager, container base layers, or build tool dependencies, a properly keyed cache can turn a multi-minute install step into a matter of seconds on most runs. The catch is that the cache key has to be specific enough to invalidate correctly when dependencies actually change, or the cache becomes a source of confusing, hard-to-reproduce failures instead of a time saver.

Approval gates with no clear owner

A manual approval step is a reasonable control for a production deployment. It becomes a bottleneck when the approver is not clearly defined, is often unavailable, or the approval step exists out of habit rather than because it catches anything meaningful. Pipelines with vague or slow approval gates tend to accumulate a queue of releases waiting on a single person, which quietly becomes the real constraint on how often the team can ship.

A short measurement exercise worth doing

• Record time-to-start and time-to-complete separately for the last fifty pipeline runs
• List every stage and mark which ones have a genuine dependency on another stage's output
• Count how often the same test fails and gets retried without being investigated
• Check whether staging and production use the same built artifact or two separate builds
• Confirm dependency and layer caching is actually being hit, not silently missing on every run

Most pipelines we review have at least two or three of these issues at once, and none of them require a new platform to fix. The build step usually gets optimized first simply because it is the part everyone watches. The time that is actually recoverable is often somewhere else entirely.