Experimentation Methodology Lead, Siri Data Organization

Santa Clara Valley (Cupertino), California, United States
Machine Learning and AI


Posted: Jul 23, 2019
Role Number: 200075041
Would you like to play a critical part in the next revolution in human-computer interaction? Contribute to the advancement of a product that is redefining human-computer interaction, and work with the people who build the intelligent assistant that helps hundreds of millions of people get things done — just by asking? The vision for the Siri Data organization is to improve Siri by using data as the voice of our customers. Within this organization, as the leader of the Experimentation methodology team, you will build a team of experimentation methods experts, to guide a rigorous cycle of learning-based product development. You will partner with product engineering teams developing Siri to guide development, so Siri can be the easier, faster, and safer way to get things done across iPhone, iPad, HomePod, Mac, Apple Watch, Apple TV, CarPlay and AirPods, across dozens of languages. Learn more about Siri: https://www.apple.com/siri/ We are looking for a creative problem solver with a passion for delivering data- informed insights, deep knowledge of testing and experimentation, and passion for education on experimentation. You will partner with many teams, and drive experimentation process improvement and automation to dramatically transform the way in which we develop and utilize experiments that drive decision-making.

Key Qualifications

  • Demonstrated knowledge and passion for experimentation methods, data analysis, data-informed product innovation, building and shaping data culture
  • Have successfully hired and mentored top talent team members
  • 8+ years of experience in decision support, ideally in online tech company
  • Confident applying appropriate analysis methods of causal inference in both experimental and non-experimental situations
  • Excellent problem solving skills and end to end quantitative thinking
  • Outstanding communications skills with both technical and non-technical
  • colleagues
  • Excellent relationship-building skills; ability to manage complex cross- functional relationships and establish strong partnerships
  • Ability to tell stories with data, educate effectively, and instill confidence, motivating stakeholders to act on recommendations
  • Flexibility to adapt and learn quickly about complex machine-learning- based products across hardware and software, and experimentation in these settings
  • Understanding of complex customer ecosystems, best practices and ability to put this knowledge into action


Build a team of experiment methodology experts (statisticians, data scientists) Partner closely with Experimentation Platform team to guide and ensure the platform is designed and enhanced such that it accurately enables experimentation science and supports visualization and readout of experiments. Collaborate with Analytics embedded data scientists, to ensure that as they support experiments in their domain, they use sound methodology, test setup and analysis. Work with Product Engineering teams to develop experimentation learning plans to set up, maintain and optimize experiments. Recommend analytic approaches to address questions or validate hypotheses (especially in cases where there is no dedicated Analytics partner). Establish and iterate on processes such as Experiment Review Committee to ensure solid and robust. Establish and provide training on experimentation standard methodologies Provide test requesters with testing methodology and frameworks to monitor tests over time Define and standardize dashboards to read out experiment results Collaborate with internal and external partners to assist with data collection and reporting

Education & Experience

PhD or MS in a quantitative discipline such as but not limited to statistics, applied mathematics, informatics, operations research, computer science.

Additional Requirements

  • Recruit, hire and nurture people who are not only great technically but also demonstrate success in the highly collaborative, functional environment of innovation at Apple.
  • Lead a team while keeping them connected, collaborating, inspired; whereby they understand how their work ladders into the bigger picture, mission, and vision
  • Give crisp feedback to your team that balances opportunities for growth with vision to inspire.
  • Work in service of something larger without creating a service org. Your team can deliver what is needed but also redefine what is possible.
  • You are humble and lead by example.
  • You have compassion for your colleagues and the challenges they face. You partner to drive pragmatic progress, but can drive a controversial direction when needed.
  • You model detailed thinking and crisp communication.
  • Move up and down the ladder of abstraction and help your team do so as well. You can talk and design on technical matters but are always able to connect that work to the impact it is in service of.