Siri - Data Human Annotation Assets Lead, Data organization

Santa Clara Valley (Cupertino), California, United States
Software and Services


Posted: Feb 13, 2019
Weekly Hours: 40
Role Number: 200007330
Siri is a powerful product family that powers intelligence on many Apple hardware device categories, across many languages, and many use cases. As a machine learning based product, Siri relies on human annotation dataset creation as an asset to enable ongoing development and improvement to the Siri product experience. We’re looking for someone to lead business and program management for Human Annotation at Siri. In this role, you will be responsible for establishing a process for requirement gathering, prioritization, budget allocation and management, budget management, and forecasting. You will also be responsible for managing the Human Annotation data assets built for use by engineering teams. You will collect requirements from all engineering groups developing the Siri product, set up a process to vet those asks, improve them, establish leverage across dataset requirements, then establish a prioritization of the development of these data assets by the global human annotator workforce, and manage the execution of the creation of them. You will ensure that the data assets built are utilized by the product engineering teams, and that timelines are efficient and optimized. You will build a future roadmap for this data asset management.

Key Qualifications

  • 10+ years as an Engineering or Product Manager or Business Operations manager, or similar role leading cross functional teams
  • Proven capability to set direction and work across a large number of partners to establish operating procedures, enforce process and drive efficient outcomes
  • Self-motivated and dedicated with demonstrated process optimization capabilities required to set a path to success
  • Great attention to detail and organized
  • Outstanding communication and presentation skills, written and verbal, to all levels of an organization
  • Experience with:
  • working across teams to collect requirements, document, prioritize
  • managing execution of complex projects
  • reporting on progress, priorities, timelines, and risks, at different levels (Executive, Product teams)
  • capital planning, budgeting and finance management
  • setting up training/evangelization
  • building processes and tools to track and visualize progress against goals, and utilization of assets, accessible to engineers and executives.
  • building a new program/process from the ground up including defining requirements, driving agreements from multiple teams and executing towards a successful completion
  • Preferred experience with:
  • working with engineering teams
  • working in a technology company, especially with machine learning based products
  • leading large scale human annotation efforts or working on ML projects that involve large scale data collection of this type
  • working with vendors or in vendor management


Establish process for requirement gathering, prioritization, budget allocation, budget management, and forecasting Lead roadmap, detailed execution of future human annotation programs as well as annual plus quarterly human annotation budget planning for Siri Manage allocation of human annotation resources across Siri teams, and define metrics to monitor our performance against those goals Partner with procurement and vendors to meet human annotation resource demand Liaise between Siri teams and annotation teams to ensure quality results Streamline a process for training engineering teams on how to ask for, set up and execute human annotation projects to build datasets needed for machine learning modeling work Partner closely with Siri engineering teams to gather requirements and understand priorities, to ensure we collect data that will inform and drive development of the product and user experience Communicate with and advocate to a wide audience of engineers, managers and executives Organize data sharing initiatives for Siri teams to improve utilization and value generation of our human grading data assets Establish efficacy and efficiency standard methodologies for human annotation at Apple Drive Apple wide initiatives to scale human annotation process (tooling and process)

Education & Experience

BA/BS, MA/MS or PhD in Computer Science, Computer Engineering, Statistics, Economics, Information Science, Linguistics, Communication Studies, or related fields/experiences Preferred: Masters in Business Administration

Additional Requirements