Siri - Data Pipeline Engineer
Santa Clara Valley (Cupertino), California, United States
Software and Services
Want to push the envelope on in human-computer interaction at Apple? Contribute to a product that is redefining mobile and desktop computing, and work with the people who built the intelligent assistant that helps millions of people get things done — just by asking? The vision for the Siri Data organization is to improve Siri by using data as the voice of our customers. The Siri Team at Apple is seeking a talented data pipeline engineer to build complex & scalable data pipelines. As part of this group, you will work with one of the most exciting high performance computing environments, with petabytes of data, millions of queries per second, and have an opportunity to imagine and build products that delight our customers every single day.
- * 3+ years experience working with Spark or other big data architectures (Hadoop, Mapreduce) in high-volume environments
- * Experience building and managing ETL pipelines from inception to production rollout
- * Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- * Experience with workflow management tools: Airflow, Oozie, Azkaban, etc
- * Experience with configuration management & Monitoring : Splunk, Grafana, Prometheus, Nagios, puppet, etc
- * Experience ensuring a high degree of reliability and data integrity for mission critical systems
- * Experience with SQL and basic database knowledge for modifying queries and tables.
- * Experience with CI / CD : Teamcity, Jenkins etc
The Siri Metrics Platform team is in a unique position to align our quality initiatives to a singular platform. Help us architect highly scalable distributed data pipelines. A part of the job is to ensure the Operational SLA for data generation & availability across the data org in Siri. In order to achieve these things a successful candidate would have experience in : * Writing Tools / Dashboard for operational excellence * Contribute to our monitoring & Alerting framework * Develop & contribute to Open source projects ( Apache Spark, Apache Druid ) * Constantly evolve our pipelines & question the status quo * Ensure the platform can handle all types of robust data exploration in real-time * Partner with all Siri teams and build features to enhance data analysis
Education & Experience
Bachelor’s Degree or foreign equivalent in Computer Science, or related field, or equivalent experience