Tamaris has built a strong experimentation culture, running multiple A/B tests at the same time to continuously improve the customer experience. But as the number of experiments increased, the weekly monitoring process became more time-consuming.
Each experiment review took around 15 minutes and involved repetitive manual checks: opening the experiment, reviewing key goals, checking performance data and spotting unusual patterns. When multiple experiments were live at once, this quickly added up and took valuable time away from more strategic optimization work.
Because of these time constraints, the team could only check a limited set of metrics each week. This meant that important performance indicators, such as anomalies, shifts in behaviour or unexpected performance changes, could sometimes be missed. Tamaris needed a more scalable way to monitor experiments without compromising on the quality and accuracy of the insights.
To solve this, we developed an automated experiment monitoring tool for Tamaris.
The tool gives the Tamaris and optimization teams an at-a-glance overview of all active experiments. Instead of manually reviewing each test one by one, it automatically monitors selected goals, metrics and key data points that may indicate performance issues or unusual patterns.
Users can customize the monitoring setup based on the metrics that matter most for each experiment. The tool reviews a much broader set of performance indicators than would be feasible manually and automatically highlights which experiments need attention.
In addition, the tool generates clear conclusions, helping the team quickly understand whether an experiment is running as expected or whether further investigation is needed.
By automating the monitoring process, Tamaris has significantly reduced the time spent on weekly experiment reviews. What used to be a repetitive manual task has become a faster, more consistent and more scalable process.
The tool allows the team to review more experiments, more metrics and more potential warning signs in less time. This helps detect performance issues earlier and ensures that fewer important insights are overlooked.
As a result, the Tamaris and optimization teams can spend more time on higher-value work: generating new test ideas, interpreting results, learning from experiments and making meaningful improvements to the customer experience.
The automated monitoring approach has improved the speed, consistency and quality of experiment monitoring, helping Tamaris stay ahead in its optimization efforts.