The novelty effect is a temporary change in how people behave after something on a website, app or ad changes, simply because it is new and catches their attention. In A/B testing, it shows up as a variant that wins strongly in the first days and then fades back towards the original as the newness wears off.
How the novelty effect works
People who visit a site often form habits. They know where the menu is, which button to press and which part of the page to skip. When something changes, regular visitors notice it and some interact with it out of curiosity. That extra attention can lift clicks, sign-ups or engagement for a while, even if the change is no better in the long run.
Take a London events venue that redesigns its “Check availability” button in a bright new colour with a short animation. Regular visitors, such as event planners who use the site every month, start clicking it more. After two weeks the variant looks clearly better. By week five the lift has almost disappeared, because the button is no longer new to them.
The effect mainly applies to returning users. A new user has never seen the old version, so the change is not novel to them. That difference is the most reliable way to detect it.
There is an opposite pattern too, sometimes called change aversion or the primacy effect. Regular users initially resist a change because it breaks their habits, so a better design can look worse at first and improve later. Both patterns come from the same cause: early results reflect reaction to change, not the long-term value of the design.
Why it matters
If you stop a test early and roll out a winner that was driven by novelty, you take on the cost of the change and get little lasting benefit. Worse, the team learns the wrong lesson and repeats it. This is especially relevant to UK businesses with a loyal returning audience, such as membership sites, trade portals used by regular buyers or online shops with repeat customers.
Timing adds a UK twist. Tests that run across Black Friday, the run-up to Christmas, the January sales or the end of the tax year will see behaviour shift for reasons unrelated to the test, which makes novelty harder to separate from seasonality.
Common mistakes
- Ending a test as soon as it reaches significance in the first week, a habit known as peeking.
- Looking only at the overall result and never splitting by new and returning visitors.
- Running tests for less than a full weekly cycle, so weekday and weekend behaviour are not both captured.
- Testing purely visual changes on a mostly returning audience and expecting the early lift to last.
- Never checking the metric again after rollout to see whether the gain held.
How to act on it
- Plan the test duration in advance, covering at least two full weeks, and do not stop early because the result looks good.
- Plot the difference between control and variant day by day. A gap that shrinks steadily over time is a warning sign.
- Segment results by new and returning users. If the lift exists only among returning users and is fading, suspect novelty.
- After rollout, keep watching the conversion rate for several weeks and compare it with the test result.
When I test landing page changes for paid traffic, I build the test length and segment checks into the plan from the start, as part of PPC landing page design, so a short-lived spike is not mistaken for a better page.