The CRO Shift Is Here: Future-Proof Your Career at Conversion Live (Sponsor)
AI is transforming marketing faster than ever, and 2025 is the year we either use it or lose it. At Conversion Live 2025, we’re bringing together the brightest minds in CRO, AI, and digital strategy to unpack the tools, tactics, and trends that will define the future. How will you optimize in a world where search traffic is shrinking, ad costs are rising, and AI is rewriting the rules? Find out at this one-of-a-kind virtual conference.
3 days of expert-led sessions
Live networking in our virtual lounge
Access to ConversionMind, an AI trained on the event’s content
Use coupon code EARLYBIRDEXTRA before February 13th to get $299 off.
Seats are limited so Save your spot today.
Tim Mehta recently led the growth product teams at Lucid Software as Director of Growth, where he oversaw user acquisition, onboarding, monetization, expansion, and retention. With over a decade of experience in marketing and product experimentation, he has worked both in-house and on the agency side with brands like Microsoft, AT&T, Xbox, and Expedia. His expertise lies in conversion rate optimization (CRO) and building experimentation frameworks within organizations.
In this episode, Tim shares his thoughts on how to build a strong experimentation culture by contrasting his experience at Microsoft, where experimentation was highly structured and deeply embedded, with Lucid’s decentralized model, where teams operate independently with varying processes. Tim explains the challenges of this approach—siloed learnings, inefficiencies, and inconsistent methodologies—and why Tim wanted to shifting Lucid toward a Center of Excellence (COE) to drive alignment, scalability, and better decision-making.
We also discussed the trade-offs between speed and rigor in experimentation. Tim explains how product teams must balance test velocity with the complexity of working within a live product environment, where small changes can have significant ripple effects. We also discussed the need for a strong operational framework, cross-team collaboration, and the right tooling to ensure experimentation leads to effective change. Finally, we touch on the evolving role of AI in experimentation, particularly in surfacing insights and streamlining workflows.
In this episode, we cover:
(00:00) Introduction to Tim Mehta and Lucid Software
(05:07) Comparing Experimentation Culture at Microsoft and Lucid
(11:25) Decentralized vs. Center of Excellence Models
(25:21) Balancing Experimentation: Acquisition vs. Retention
(30:01) Principles of Experimentation: Integrity, Velocity, and Documentation
(39:53) The Role of AI in Experimentation and Documentation
(50:21) Resources and Final Thoughts
Referenced
Episode: Experimentation at Microsoft with Prerit Saxena - In this episode, you'll learn how Microsoft's Edge team uses data and experiments to make web browsing better, why testing new features is so important, and how AI helps make smarter decisions faster.
Episode: Decentralized Experimentation at Delivery Hero with Javier García Latorre - In this episode, you'll learn how Delivery Hero runs a decentralized experimentation program, why testing is built into their product culture, and how AI is helping teams speed up testing and decision-making.
Book: Experimentation Works: The Surprising Power of Business Experiments - Stefan Thomke teaches that businesses can become more successful by constantly testing new ideas, learning from the results, and using that knowledge to improve their products and services.
Book: Inspired: How to Create Tech Products Customers Love - Marty Cagan explains that to make tech products people enjoy, companies should understand what users need, work together as a team, and keep testing and improving their ideas.
Post: Why Create a Team Charter For Experimentation / CRO Programs - Team charters help the team work together better by clearly stating their goals, roles, and how they will communicate, ensuring everyone is aligned and accountable.
Blog: Balancing Velocity and Confidence in Experimentation - While not the entire point of the article, DoorDash describes two types of startup leaders: one who believes in moving quickly by testing many ideas to see what works, and another who prefers to carefully plan and check each idea to ensure it's correct before acting.
Book: Never Split the Difference - Former FBI negotiator Chris Voss shares stories from his career to teach readers how to use empathy, active listening, and effective communication to become better at negotiating in everyday situations.
Book: Influence: The Psychology of Persuasion - Dr. Robert Cialdini explains that people can be persuaded to say "yes" by using six key principles, helping readers understand how to influence others and recognize when they're being influenced.
Book: Crucial Conversations: Tools for Talking When Stakes Are High - Joseph Grenny teach that when discussing important topics where people have different opinions and strong feelings, it's essential to stay calm, listen carefully, and speak honestly to resolve disagreements and strengthen relationships.
Book: Thinking, Fast and Slow - Daniel Kahneman explains that our brains use two systems to make decisions: one that thinks quickly and automatically, and another that thinks slowly and carefully, helping us understand how we can make better choices.
Websites
Interested in sponsoring the newsletter?
Interested in sponsoring or advertising on this newsletter? I’m always looking to partner with brands and businesses that align with my audience. Reach out to me at brian@mammothinsights.io to inquire about sponsorship opportunities!
Upcoming Editions
Price Testing with Intelligems
Experimentation at Comcast
Experimentation at Meta
📚 Worth Exploring
Drop-Down Usability: When You Should (and Shouldn’t) Use Them
Baymard’s article explains that drop-down menus work best with 5 to 10 options. Too many options can confuse users, and sometimes using text fields is a better choice. It shows how to design drop-downs that are easier for people to use. Link to article
TBM 340: Path to $
John Cutler’s article explains how different types of teams work and how they demonstrate their value to the company. For an experimentation team, it emphasizes the importance of collaborating with other teams, especially those directly connected to customer outcomes or business goals. Which kind of teams do you support? Link to article
Six Mind-Benders that will change how you optimize
This Convert.com post shares six CRO lessons, like valuing relationships over data, focusing on custom features, and testing product finders. And by the way, #1 is the best mind-bender—it’ll make you rethink everything! 😉 Link to post
Deliveroo experimentation principles
Deliveroo’s article talks about how to run good experiments, like having clear ideas and sharing results. Principle #4, which is often missed, says it’s important to balance how much experiments cost with the value they bring, and to make choices based on both data and the bigger picture. Link to article
👋 Hi, I’m Brian Poe, Founder and Chief Optimization Officer of Mammoth Insights, a boutique agency with 15+ years of experience helping mid-sized to enterprise companies improve digital strategies through affordable, high-quality Conversion Rate Optimization (CRO) research and experimentation. I’ve had the pleasure of working with leading global brands such as Best Buy, Verizon, Target, Ulta Beauty, Kia Motors of America and more.
Whenever you're ready, there are a few ways I can help you:
1. EXO Field Notes Participation: Want to elevate your personal brand on EXO Field Notes? Here’s how: 1:1 Interviews spotlight top experimentation and product leaders (see Nike example), Micro-Interviews are perfect for newcomers (view archive here), and Roundtable Discussions offer peer-to-peer discussions for 2-3 in-house brands (see roundtable example). Click here to message me on LinkedIn to inquire.
2. Full-Service Experimentation: Looking for an all-in-one solution? Our experimentation service at Mammoth Insights covers everything from crafting the strategy to executing the tests. We’re tool-agnostic, so there’s no need to change your setup. With all team members with 10+ years of experience working with companies like Best Buy and Verizon, we guarantee fast and effective results. Click here to learn more.
3. CRO Kickstarter Workshop: Ready to take your CRO program to the next level? Join me for an exclusive 90-minute workshop, tailored for you and your team. Whether you're just starting out or looking to advance, we’ll cover CRO fundamentals, prioritization frameworks, behavioral design, and more. I’ll show you everything I’ve learned over 15+ years in CRO and how to maximize your program for just $499 USD. Don’t miss out—reach out today!
4. Conversion Research: Stop guessing and start converting. Our CRO service uses a mix of quantitative and qualitative UX research, including digital analytics and usability tests, to find what works best for your audience. I’ve helped brands like Capella University and Verizon Wireless achieve more with their existing traffic. Click here to learn more.
5. Brand Booster Roadmap: Future proof your brand experience. Work directly with Brian, Chief Optimization Officer, and Jennie, Head of Brand and former Head of Brand Strategy & Management at Wayfair. You’ll receive a comprehensive, tailored audit and brand strategy guide with data-driven recommendations. This includes brand positioning, online experience analysis, omni-channel content analysis, and a detailed brand experience roadmap. Contact us for a free consultation and to learn more.
Share this post