Pricing Results Good Ideas Ingredients For Success
Discuss

Uncover high-probability A/B test ideas.
Run them as high-powered experiments.

Jakub Linowski

With Jakub Linowski as your A/B test & experiment design consultant

Jakub Linowski
  • Expert Experiment design & analysis
  • Expert UI design
  • Strong Experiment strategy
  • Strong UI prototyping & front-end dev

1,600+ experiments designed· 600+ published· 15 years experience· worked with Microsoft, Booking, LinkedIn

Choose a conversation starter:

5-10% opportunities

to uncover 5-10% tests worth retesting.

Discuss This

Why? As we run experiments, we forget that some of our inconclusive, winning and/or losing results have a lot more potential as reruns.

Rerunning past experiments

Powered by Rerun, our open-source tool for highlighting what is worth replicating. Having at least 50+ past experiments is a great starting point.

15-30 opportunities

to uncover 15-30 high-probability test ideas.

Discuss This

Why? As we come up with test ideas, we rarely tap into priors from experiments other companies have already run, which raise our win rates and impact.

GoodUI experiments

Powered by GoodUI, our library of 600+ A/B tests from other companies. Best suited for teams already running experiments.

run more tests to multiply your program.

Discuss This

Ideas: Concurrency · Rapid Prototype Testing · Iteration · Democratization · Sharing Results Across Company · Automate Reruns

increase trust and reduce noise of your results.

Discuss This

Ideas: Power Checks · Stopping & Ship Rules · SRM & Guardrail Checks · Business-Aligned Metrics · Crowd-Based Prioritization · Reporting

Have Something Else On Your Mind? Discuss A Custom Project

Ongoing retainer · Monthly advisory · Custom projects · Experiment reviews & training

Historical Experiment Results

Relative median effects from past experiments you can expect on projects
for metrics such as: LEADS SALES SIGNUPS REVENUE

(Medians only show the midpoint. Some projects are higher. Others are lower.)

+2.9%

on smaller experiments

1 IN 3 WIN RATE

Source: GoodUI data; N = ~600

+13.5%

on larger “leap” experiments

4 IN 5 WIN RATE

Source: Linowski "Leap" projects; N = 50

Worth noting: 1/3 of experiments are negative — testing protects you from shipping changes that hurt your metrics, and some can even be inverted into wins with future iterations.

SCREEN TYPES WE HAVE THE MOST EXPERIENCE DESIGNING & OPTIMIZING FOR

Landing &
Home Pages
Product
Pages
Listing
Pages
Signup
Funnels
Checkout
Funnels
Pricing
Pages

HI, I’M JAKUB

Jakub Linowski

OVER

15yrs

experience designing front-end UI A/B tests

COACHED

~100

experimentation & growth teams

PUBLISHED

~600

experiments on GoodUI

HIRED BY

Microsoft
LinkedIn
Booking.com
And others

WITH

2

design degrees

AND A

Love

for stats & numbers

WHERE DO HIGH-PROBABILITY
IDEAS COME FROM?

1

Companies I Worked With

Over 15 years, I've designed ~1,600 A/B tests for teams like these.

Our Client Logos
2

Companies I Coached / Learned From

I've coached ~100 growth teams, learning together what works and what doesn't.

MicrosoftReverbFluke MetroBaremetricsUmbracoYummlyVivaRealVoldersMettler ToledoBackstageThomasnetDrip AgencySnocksKenhubExpert InstituteExamine.com3DHubsBomgarDigital MarketerLovehoneyJared Kayoutlet.comNorman Records 686.comShmoodySvsound.comiBood.comFINN ElevateGetNinjasDesignlabPhorestUpliftsHomer LearningChaos GroupASICS ...
Learning From Growth Teams

Teams I've coached and learned from include: Microsoft, Reverb, Fluke, Metro, Baremetrics, Umbraco, Yummly, VivaReal, Volders, Mettler Toledo, Backstage, Thomasnet, Drip Agency, Snocks, Kenhub, Expert Institute, Examine.com, 3DHubs, Bomgar, Digital Marketer, Lovehoney, Jared, etc.

INGREDIENTS FOR SUCCESS

1. Upfront ROI, Sensitivity Check & Clear Business Metrics
Before we start, we'll check the return on investment based on historical data and optimization potential. We'll also assess your testing sensitivity to determine what types of experiments can be run and where. Together we'll agree on business metrics that matter most to you (less so: clicks, page visits, time on site; more of: acquisition, referrals, average order value, transactions, leads) along with realistic targets needed for success.
2. All Optimizations Are A/B Tested
All optimization ideas that we propose will be designed as A/B tests. Controlled experimentation is one of the most reliable methods for learning about the effects of particular changes.
3. Compound If Possible
When we see opportunities to combine multiple changes into a larger leap experiment, we'll let you know. This way, bigger leap experiments open up the potential for larger effects (which are also easier to detect, faster, with less traffic - one of many possible strategies).
4. 80% Statistical Power With Sequential Stops
Before we run each test, we'll do a sample size calculation based on probable effects from similar experiments run in the past. Generally, this hovers around ~2.9%, but will be adjusted lower or higher depending on the test design. We will also use sequential statistics to determine if we can stop the test earlier.
5. Room For Iteration
Iteration is an ingredient present in all of our projects as it increases the odds of success (including our ability to turn a first failed attempt around).
6. Variation Over Compromise
When we find ourselves in situations with a difference in opinion (normal), we'll typically steer you away from compromise and toward handling this through experiment design. This is done by adding new variations or turning them into new test ideas.
7. Democratic Ideation + Autonomy
Although compromise is to be avoided, democratization of the experimentation process is highly encouraged. Multiple and testable ideas should come from diverse people on the team.
8. Modern Experimentation Platform + Peek Protection with Early Aborts
Implement a reliable experimentation platform such as: Eppo, ABSmartly, Convert, Growthbook with modern stats (SRM checks, sequential testing, custom metrics, triggering, post-hoc / exploratory analysis, guardrail metrics, etc).
9. Testing Velocity + Concurrency
Since roughly 1/3 of experiments succeed, experimentation becomes more successful with multiple attempts and with iteration. Since individual experiments require variable time frames (ex: 1 to 12 weeks) to detect a predefined effect, industry standard practice is to run experiments concurrently where possible.
10. Adaptable Metrics
It's not just about conversion rate. It's also about lead quality. Over time we'll be adjusting what we measure as we learn what matters more.
Discuss Customize

Book a 25 min intro call with Jakub

See Add-Ons