How to run an A/B test on your framer site

What makes Framer A/B testing special

• Designers and marketers can set up and launch tests in under a minute—no code required.

• A/B testing in Framer doesn’t slow down your site or cause elements to jump or flicker during load.

• You’ll get real-time results that everyone can work with.

• A/B tests work across multiple locales.

When to run an A/B test

A/B tests work best on pages with high traffic. The more visitors you have, the faster you’ll get meaningful results. They’re ideal for testing new ideas on pages designed to drive conversions. That means your page should have a clear conversion event, such as a link click or a completed form.

How to set up a test

Creating a new A/B test

Before you start an A/B test, you need two things: a page and a conversion event. This event can be a link click, form submission, or a subsequent page view. An A/B test always starts with a page. Right-click your page and select New A/B Test. (Note: CMS pages are not supported yet.)

A new variant of the selected page is created, and you can apply your changes. You can add up to five variants per page and rename them to easily identify them later when viewing your results. You have full design freedom when making changes to your variants. Just make sure you’re optimizing for a specific call to action by adding a tracking ID to a link or form that is consistent across all variants.

Configuring an A/B test

Once you’ve added your changes, it’s time to configure your test. Click the Configure button on the banner shown while viewing your page. The Analytics tab will open, and you’ll be prompted to select a step to count as the conversion step. In the right panel, click the + icon next to Steps, and choose the type of event you want to use as the conversion. You can add multiple steps, but the last step will be used to calculate the conversion rate and measure results.

Starting an A/B test

After configuration, click the Start button and publish your site. If you’re using staging, make sure to deploy the site as well. Framer will begin recording page views and events for the test. You can view the results anytime in your analytics.

Once started, Framer will distribute visitors evenly across your variants. For example, if you have two variants, each will be shown to 50% of your visitors.

Analyzing results

With enough traffic, you’ll start seeing results almost immediately. However, you need to wait until there is enough data to make a reliable decision. In the results summary, Framer will show key metrics and how many days are left before a winner can be declared.

In the analytics view, you’ll see a conversion funnel, a table of the different variants, and a graph over time. The most important metrics are:

• Conversion Rate: The percentage of unique visitors who completed the conversion step divided by the total unique visitors.

• Lift: The percentage increase or decrease in the conversion rate compared to the control. A positive lift means the variant is likely performing better than the control.

• Best: The “Probability of Being Best” percentage, which indicates how likely it is that the variant is winning. If this exceeds 90% and enough data has been collected, Framer can declare a winner.

Framer will display a graph once there is sufficient data. This graph shows test performance over time. The bands around the lines represent the credibility interval—the narrower it gets, the more reliable your results. Ideally, you’ll see the lines stabilize and show clear separation between variants to confidently determine a winner.

Stopping an A/B test

When a test becomes conclusive, it’s time to review the results and make a decision. If a winner is declared and the results are significantly different, you can safely stop the test by clicking the Stop Test button. You’ll be prompted to replace the control version with one of the variants. If a variant is clearly outperforming the control, you can select it to become the new control. Afterward, publish your site again to ensure all visitors see the winning variant. You’re free to delete the other variants.

Sometimes you may have enough data, but no clear winner. In that case, you can still replace the control variant, but it’s possible that this change won’t significantly impact your analytics.

FAQs

Does Framer use cookies for A/B testing?

Framer A/B tests work without cookies. When a new visitor arrives, we assign a unique, anonymous identifier based on their IP address and browser info. This identifier can’t be used to personally identify individuals, but ensures the visitor sees the same page variant throughout the day.

The identifier resets daily and can’t be used to track someone across multiple days or devices. The benefit of not relying on cookies is increased reliability and potentially no need for a cookie banner (depending on your local laws). The trade-off is that we can only measure conversions within the same day.

While this is standard for advanced A/B testing tools, we’re planning to add a cookie-based system in the future to allow for longer, more flexible conversion windows.

What happens if I change a variant while a test is running?

It’s not recommended to make changes to variants while a test is running, as this may impact your results. For reliable outcomes, stop the A/B test, apply your changes, and start a new test.

What scientific method does Framer use to determine a winner?

Framer uses a Bayesian approach to determine a winner. This method continuously updates the probability that one variant is better than another as new data comes in. Unlike traditional frequentist methods that rely on fixed sample sizes or p-values, the Bayesian model calculates the Posterior Probability of Being Best (PBB). Once a variant’s PBB exceeds 90%, Framer considers it a statistically confident winner.

Pricing

A/B testing is part of Advanced Analytics, an add-on for sites on a Launch or Scale plan. You can start with a trial for 30 days. After that you will be billed based on the number of events that your site has. An event is either a page view, a click on a tracked link or a submit on a tracked form. Usage is metered per month and you always get 25,000 events for free. Your add-on is billed as part of the billing cycle of the associated site plan.

Prices are localized and subject to change. The table above shows yearly pricing in USD.