Back to blog
· Darwin Team

Why A/B testing tools slow down your website (and what to do about it)

You installed an A/B testing tool six months ago. Since then, your team has run 30 experiments. Twelve of them won. Your conversion rate is up.

But your Lighthouse score is down. Your Largest Contentful Paint has crept from 1.2 seconds to 2.8 seconds. Your CLS score has gone from zero to “needs improvement”. And your developers are spending hours debugging layout shifts they can not reproduce locally.

The culprit is something nobody warned you about: mutation stacking.

How A/B testing tools actually work

Every traditional A/B testing tool (VWO, Optimizely, AB Tasty, Convert, and the late Google Optimize) follows the same basic pattern:

  1. Their JavaScript SDK loads on your page
  2. The SDK fetches the list of active experiments from their servers
  3. For each experiment, the SDK patches the DOM: changing text, swapping images, hiding elements, rearranging layouts
  4. The visitor sees the modified page

This is clever engineering. It means you can run experiments without touching your source code. A marketer can change a headline without filing a Jira ticket. The visual editor makes it feel effortless.

The problem starts when an experiment concludes.

What happens to winning mutations?

When a variant wins, most teams celebrate and move on to the next test. The winning mutation stays in the tool. It becomes “the new default” within the SDK configuration.

But it is not the new default in your source code. Your actual HTML still says the old headline. The A/B testing SDK still loads, fetches that mutation, and patches the DOM to show the winning version.

This is fine for one experiment. It is a disaster at scale.

The mutation stacking problem

After 12 months of active testing, a typical team has accumulated 30 to 50 concluded experiments. Each winning variant is a permanent runtime mutation. Every single page load now follows this sequence:

  1. Browser downloads your HTML
  2. Browser starts rendering
  3. A/B testing SDK loads (50 to 200KB of JavaScript)
  4. SDK fetches experiment configuration from a remote server (network round trip)
  5. SDK evaluates 30+ mutations and applies them to the DOM
  6. Page finishes rendering with the “correct” content

Steps 3 through 6 happen on every page view, for every visitor, forever. The mutations never retire. They just accumulate.

Performance impact

Each DOM mutation triggers a browser reflow. After 30 mutations, you are forcing the browser to recalculate layout dozens of times before the page stabilises. This directly impacts:

  • Largest Contentful Paint (LCP): The SDK must load and execute before the final content appears. If the SDK loads asynchronously (which most do to avoid blocking), visitors see a flash of the original content before it gets patched. If it loads synchronously, nothing renders until all mutations are applied.

  • Cumulative Layout Shift (CLS): Each text change, image swap, or element removal causes a layout shift. Thirty mutations means thirty potential shifts. Google measures this, and it affects your search rankings.

  • Total Blocking Time (TBT): The SDK’s JavaScript evaluates selectors, queries the DOM, and applies changes. This is main-thread work that blocks interaction. The more mutations, the longer the block.

  • First Input Delay (FID): If a visitor tries to click something while the SDK is still patching the DOM, they wait. The page looks ready but is not.

We have seen sites lose 40 to 60 points on their Lighthouse performance score after a year of A/B testing. The irony: the experiments improved conversion, but the accumulated overhead started pushing it back down.

Selector fragility

Every mutation targets a DOM element using a CSS selector. Something like #hero-section h1 or .cta-button.primary. These selectors are stored in the A/B testing tool’s configuration.

Now imagine a developer updates the page layout. They rename a class, restructure a component, or swap a div for a section tag. The selectors in 15 of your 30 mutations just broke.

What happens? It depends on the tool. Some fail silently (the mutation does not apply, visitors see the old content). Some throw JavaScript errors. Some apply the mutation to the wrong element entirely, creating bizarre layout bugs that only appear in production and only for visitors who are in the affected experiment cohort.

Debugging this is a nightmare. The developer’s local environment does not run the A/B testing SDK with production experiment data. The bug is invisible until someone reports it.

SDK dependency

Here is the part that should worry you most: your “optimised” page depends entirely on a third-party script loading successfully.

If the A/B testing SDK fails to load (CDN outage, ad blocker, network timeout, JavaScript error), every single mutation vanishes. Your visitors see the original, un-optimised page from 12 months ago. All 30 improvements disappear simultaneously.

This is not hypothetical. CDN outages happen. Ad blockers increasingly target A/B testing scripts. Corporate firewalls block unknown third-party domains. A meaningful percentage of your traffic sees zero optimisations because the SDK never loaded.

The debugging nightmare

Your QA team reports a bug: “The hero section looks wrong on mobile.” Your developer inspects the page. The HTML looks fine. The CSS looks fine. They can not reproduce it.

That is because the bug is not in your code. It is in mutation #23, which changes the hero padding, interacting with mutation #31, which swaps the headline text (which is now longer and wraps differently on small screens). Neither mutation knows about the other. The A/B testing tool has no concept of mutation conflicts.

Now multiply this by every page, every device, every experiment cohort. You have created a combinatorial explosion of potential states that no amount of manual QA can cover.

The root cause

The fundamental problem is architectural: traditional A/B testing tools treat the SDK as a permanent content delivery layer.

It was designed to be a temporary test harness, but the industry never built a mechanism for graduating winning changes out of the SDK and into the actual source code. The SDK became load-bearing. Remove it, and your page breaks.

This is backwards. Your source code should be the source of truth. The A/B testing tool should be a temporary instrument, not a permanent dependency.

A different approach: test temporarily, commit permanently

What if winning mutations did not stay in the SDK forever? What if they graduated into your real codebase?

The workflow would look like this:

  1. Run an experiment the usual way: split traffic, show variants, measure results
  2. When a variant wins, commit the change to your source code (via a Git pull request, or a direct CMS update)
  3. Retire the mutation from the SDK
  4. The page now shows the winning version natively, with zero runtime overhead

After 50 experiments, your page has had 50 genuine improvements. Zero runtime mutations. No SDK dependency for concluded tests. Clean DOM, fast performance, and every change visible in your version history.

This is the approach we built Darwin around. The SDK runs experiments temporarily. Winners get committed to code. The mutation is retired. Your page gets permanently better without accumulating technical debt.

What you can do today

If you are running a traditional A/B testing tool and worried about mutation stacking, here are some immediate steps:

  1. Audit your active mutations. How many concluded experiments still live in your SDK configuration? If it is more than 10, you have a problem.

  2. Manually commit winners. Go through each winning mutation and apply the change to your actual source code. Then remove the mutation from the SDK. This is tedious but eliminates the overhead.

  3. Set a mutation budget. Decide on a maximum number of concurrent runtime mutations (we suggest 5 or fewer) and enforce it. When you hit the limit, commit or discard before adding more.

  4. Monitor performance alongside experiments. Track your Core Web Vitals over time. If they degrade as you run more experiments, mutation stacking is the likely cause.

  5. Consider tools that commit winners automatically. This is what Darwin does, but the principle applies regardless of the tool. The winning change should live in your code, not in a third-party configuration.

The future of A/B testing is temporary

The A/B testing industry has spent 15 years building increasingly sophisticated tools for applying runtime mutations. What it has not built is a way to stop applying them.

The next generation of experimentation tools will treat mutations as genuinely temporary. Test, measure, commit, move on. No stacking, no dependency, no performance debt.

Your landing page should get better over time. The tool that improves it should not simultaneously make it slower.

Darwin is an AI-powered A/B testing tool that tests temporarily and commits permanently. Every winning mutation graduates into your source code. See it in action on our own site.