Product Insights Testing Lead, Business Analytics
Full Time - Austin, TX
We are looking for a Product Insights Testing Lead to join our Business Analytics team. Data is at the center of everything Favor does. The appetite for testing, reporting and analysis grows each day as we expand our geographic footprint throughout Texas and introduce new programs, platforms, products and features to our community of customers, runners and merchant partners.
As the Product Insights Testing Lead on the Business Analytics team you will work cross-functionally to design and build a centralized testing program that adheres to best-practices to ensure we produce clean, accurate and reliable results from which we can make high-confidence decisions. In addition to designing the centralized testing program, you will be on the front lines to help stakeholders determine testing hypotheses, primary KPIs, and parameters such as significance level, power, sample size, and expected duration. Once tests are complete, you will lead the analysis effort to determine how to proceed – e.g. did the test pass or fail, and if it failed, how can we iterate on the product to increase the odds of a successful future test? You will also own the presentation and sharing of test results and recommendations to ensure the company is aligned on test performance and next-steps.
- Work cross-functionally to design and build a best-practices testing program for our front/backend products and user-facing programs while fostering a culture of statistical rigor.
- Create a standardized process for test proposal, acceptance, execution and ongoing measurement and management.
- Provide end-to-end test support including hypothesis creation, KPI selection, sample sizing, in-flight monitoring, significance measurement, post-test analysis and recommendations.
- Advocate for data-backed interpretation of test results and decisions which lead to improvements in key areas of the business, such as acquisition, engagement, conversion, retention, frequency and monetization.
- Advise on alternative testing frameworks for atypical and network-effect scenarios, such as difference-in-difference and switchback testing.
- Share new test results and make accessible previous test results to increase transparency into the testing pipeline.
- Make results readily available. Ensure we know what hypotheses we’ve tested and the outcome, in order to make the most of our testing bandwidth and avoid burning money on duplicate testing ideas.
- Grow the collective Business Analytics team knowledge base through process documentation and technical mentorship.
- 3-5 years experience working in an analytical role with a deep understanding of statistical analysis and testing concepts, techniques and design.
- Demonstrated track record of leading projects and processes from inception to v1, then maintaining v1 while evaluating potential improvements for v2.
- Demonstrated track record of working cross-functionally while driving alignment on high-priority, high-visibility initiatives.
- Strong written and verbal communication skills, in particular in a presentation setting with leadership- and executive-level teams.
- Analysis ninja, with proficiency in multiple tools, scripting languages and statistical packages - e.g. Matlab, R, SAS, Python
- Intermediate to advanced SQL skills, preferably with large datasets.
- Experience with business intelligence software, such as Looker, Tableau, or Domo.
- Experience with client-side analytics, like Google Analytics and Firebase.
- Detail-oriented. You ensure accuracy and consistency of reporting and analysis through data cleansing, transformation and ongoing stewardship. You think 2 steps ahead to identify potential roadblocks and have contingency plans to address them.
- Narrative-oriented. You connect changes in top-level metrics to the underlying drivers and create compelling visualizations to drive home key findings. You communicate results effectively to audiences at all levels of the organization.
- Action-oriented. You are always up for a challenge and view reporting and analysis as an opportunity to recommend tactics and strategies that make the business better. You are open to feedback in the interest of delivering continually better output and growing as a professional.
- Teamwork-oriented. You know how and when to effectively engage with your stakeholders and your teammates. You view yourself as a partner in the decision-making process and build trust through goal alignment, process transparency and world-class output.