Prerit Saxena is an experienced data scientist with nine years in the field of data science and analytics. He holds a bachelor's degree in manufacturing and project management, as well as a master's in analytics from the University of Cincinnati. Prerit has worked across various domains including healthcare, software, and tech, and has been with Microsoft for three years. His expertise lies in data analysis, modeling, and experimentation, particularly within the Edge browser team at Microsoft.
Key Takeaways:
1. Role and Scope at Microsoft: Prerit works on the Edge browser team, focusing on experimentation to improve product features. He collaborates with multiple feature teams to ensure seamless product progression through data-driven decisions.
2. Understanding Metrics: Prerit and I discuss the importance of differentiating between North Star, guardrail, and feature metrics. North Star metrics guide the overall product direction, guardrail metrics ensure the product’s stability and user satisfaction, and feature metrics assess specific feature performance.
3. Experimentation Culture: Microsoft promotes a culture of testing everything before shipping. This includes leveraging user research and data-driven signals to ensure that product features align with consumer preferences and maintain product health.
4. Challenges in Large Organizations: Managing experimentation in a large company like Microsoft involves maintaining focus amidst numerous potential distractions. Prerit highlights the importance of team charters and aligned objectives to ensure consistency and clarity in their experimentation goals.
5. AI in Experimentation: Prerit sees AI as a valuable tool for idea generation, documentation management, and summarizing experiment results. AI can enhance efficiency by automating routine tasks, allowing teams to focus on strategic decision-making.
6. Future of Experimentation Tools: While large organizations have an advantage due to their vast data repositories, Prerit believes that experimentation platforms like Optimizely and Kameleoon can benefit from AI to provide more tailored and automated experimentation tools to a broader range of companies.
In this episode, we cover:
(01:34) – Prerit's background and journey in data science and analytics.
(03:33) – Discussion on the scope of Microsoft's Edge browser team and the experimentation involved.
(04:29) – Explanation of different metrics used in experimentation: North Star, guardrail, and feature metrics.
(14:09) – Challenges and approaches in monitoring guardrail metrics.
(18:45) – Importance of team charters and focused objectives in large organizations.
(23:09) – Experimentation culture and avoiding the "Highest Paid Person's Opinion" (HIPPO) effect.
(24:54) – Discussion on AI's role in experimentation and data science.
(32:54) – Potential for AI-driven experimentation tools to be accessible to various companies.
(38:18) – Recommendations for resources and blogs on experimentation.
🎙️ Interested in elevating your personal brand and thought leadership?
Join one of three available formats: 1:1 interviews with top leaders in experimentation, micro-interviews designed for those just starting their thought leadership journey, and intimate roundtable discussions for 2-3 established in-house brands seeking peer-to-peer insights. Ready to take the next step? Send me a message on LinkedIn to explore how you can get involved.
Whenever you're ready, there are 3 ways I can help you:
1. Full-Service Experimentation: Looking for an all-in-one solution? Our experimentation service covers everything from crafting the strategy to executing the tests. We’re tool-agnostic, so there’s no need to change your setup. With all team members with 10+ years of experience working with companies like Best Buy and Verizon, we guarantee fast and effective results.
2. Conversion Research: Stop guessing and start converting. Our CRO service uses a mix of quantitative and qualitative UX research, including digital analytics and usability tests, to find what works best for your audience. I’ve helped brands like Capella University and Verizon Wireless achieve more with their existing traffic.
3. Brand Booster Roadmap: Future proof your brand experience. Work directly with Brian, Chief Optimization Officer, and Jennie, Head of Brand and former Head of Brand Strategy & Management at Wayfair. You’ll receive a comprehensive, tailored audit and brand strategy guide with data-driven recommendations. This includes brand positioning, online experience analysis, omni-channel content analysis, and a detailed brand experience roadmap. Contact us for a free consultation and to learn more.
About the Host
I’m Brian Poe, Founder and Chief Optimization Officer of Mammoth Insights, where we specialize in helping companies enhance their digital strategies through Conversion Rate Optimization (CRO) and experimentation. With over 15 years of personal experience in the marketing industry, we have a deep passion for uncovering hidden opportunities in data to drive significant improvements in overall business performance. At Mammoth Insights, we offer affordable CRO services, including quant/qual research, market analysis, UX and copy research, A/B testing, and more, tailored specifically for mid-sized to enterprise companies. Our mission is to make complex CRO concepts easy to understand and implement at an affordable price without sacrificing expertise or quality.
