Ben Skrainka is an accomplished professional who began his career in physics before transitioning to economics and the software industry. He holds a PhD in microeconometrics from University College London and has extensive experience in computational methods and applied econometrics. Skrainka has worked for notable companies such as Amazon and eBay, where he played a significant role in enhancing experimentation platforms. He is set to return to Amazon, joining their campaign measurement and optimization team.
Some Key Takeaways:
1. To make experimentation work in a corporate setting, it's not just about having the technical skills—cultural adaptation is key. Scientists need to get comfortable with the company's culture to make sure their work actually impacts decisions.
2. At eBay, Ben spearheaded the use of generative AI in their experimentation process, making it easier for non-experts to design and run experiments without needing deep statistical know-how.
3. Implementing fully automated experimentation systems in large companies comes with its challenges. While automation can improve the process, ensuring these systems are practical and widely adopted requires addressing both cultural and technical issues.
5. A maturity model was introduced at eBay to evaluate and enhance the company's experimentation culture. This model, comprised of three pillars—culture, infrastructure, and methodology—helped identify key areas for improvement and motivated teams to elevate their experimentation practices.
In this episode, we cover:
(00:15) Introduction and Ben Skrainka's Background
(03:02) Cultural Challenges in Experimentation
(06:25) Generative AI in Experimentation
(08:56) Building Automated Experimentation Systems
(12:15) Comparing Cultures at eBay and Amazon
(18:23) The Experimentation Culture at eBay
(25:35) Where Experimentation Programs Fail
(31:12) Lessons from Career Failures
Referenced:
Benjamin Skrainka on LinkedIn: https://www.linkedin.com/in/benjamin-skrainka/
Experimentation Growth Survey: https://www.exp-growth.com/
Stefan Thomke’s book Experimentation Works on Amazon: https://www.amazon.com/Experimentation-Works-Surprising-Business-Experiments/dp/163369710X
Thomas Blake, Chris Nosko and Steven Tadelis’s paper on the effects of paid search at eBay: https://onlinelibrary.wiley.com/doi/abs/10.3982/ECTA12423
Free preprint download of Thomas Blake’s paper: https://www.nber.org/papers/w20171
A Decisions Theoretical Approach to A/B Testing: https://arxiv.org/abs/1710.03410
Causal Inference the Mixtape: https://arxiv.org/abs/1710.03410
Empowering AI with Causal Reasoning: https://causalens.com/
ExP Experimentation Platform: https://exp-platform.com/
🎙️ Interested in elevating your personal brand and thought leadership?
Join one of three available formats: 1:1 interviews with top leaders in experimentation, micro-interviews designed for those just starting their thought leadership journey, and intimate roundtable discussions for 2-3 established in-house brands seeking peer-to-peer insights. Ready to take the next step? Send me a message on LinkedIn to explore how you can get involved.
Whenever you're ready, there are 3 ways I can help you:
1. Full-Service Experimentation: Looking for an all-in-one solution? Our experimentation service covers everything from crafting the strategy to executing the tests. We’re tool-agnostic, so there’s no need to change your setup. With all team members with 10+ years of experience working with companies like Best Buy and Verizon, we guarantee fast and effective results.
2. Conversion Research: Stop guessing and start converting. Our CRO service uses a mix of quantitative and qualitative UX research, including digital analytics and usability tests, to find what works best for your audience. I’ve helped brands like Capella University and Verizon Wireless achieve more with their existing traffic.
3. Brand Booster Roadmap: Future proof your brand experience. Work directly with Brian, Chief Optimization Officer, and Jennie, Head of Brand and former Head of Brand Strategy & Management at Wayfair. You’ll receive a comprehensive, tailored audit and brand strategy guide with data-driven recommendations. This includes brand positioning, online experience analysis, omni-channel content analysis, and a detailed brand experience roadmap. Contact us for a free consultation and to learn more.
About the Host
I’m Brian Poe, Founder and Chief Optimization Officer of Mammoth Insights, where we specialize in helping companies enhance their digital strategies through Conversion Rate Optimization (CRO) and experimentation. With over 15 years of personal experience in the marketing industry, we have a deep passion for uncovering hidden opportunities in data to drive significant improvements in overall business performance. At Mammoth Insights, we offer affordable CRO services, including quant/qual research, market analysis, UX and copy research, A/B testing, and more, tailored specifically for mid-sized to enterprise companies. Our mission is to make complex CRO concepts easy to understand and implement at an affordable price without sacrificing expertise or quality.
Share this post