Senior Data Engineer - Analytics Engineering
Apple Services Engineering (ASE) is the organisation responsible for products such as Apple Music, Podcasts, TV+, tvOS, App Store, iCloud, and many others.
We, at ASE Analytics and Data Engineering, are responsible for collecting, analysing and reporting on insights derived from user and device generated data from across all Apple media services.
Reporting plays a crucial role in this process, enabling teams at Apple to gain valuable insights and make informed decisions about their daily activities. Reporting involves integrating data from multiple data pipelines managed by different teams, which presents challenges such as achieving clear visibility into the dependencies and SLAs of the contributing flows. To address these challenges, we are establishing a new team in Dublin to develop internal tools that will enhance our ability to manage these complexities.
We are looking for a Senior Data Engineer to design and develop data pipelines that will be the basis of our Knowledge Graph. The Knowledge Graph aims to unify insights on data processing and lineage across the entire Services division, pulling in metadata on workflows, execution jobs, schemas and more to generate a rich view of our data pipelines, exposed in an application that will be used daily by hundreds of internal data engineers. The graph will also power operational excellence, capturing production issues, driving analysis and guiding resolution of issues and enabling proactive avoidance of future incidents.
You’ll work alongside a Dublin based team of software and other data engineers committed to bringing the knowledge graph to life, focusing on designing the data layer. You will have significant individual responsibility and influence over the direction of this critical service, and the opportunity to learn and interact with a department of global teams each with unique skill sets and operating in different time zones.
- Bachelor’s degree in Computer Science, Computer Engineering, Mathematics, or equivalent professional experience.
- Industry experience in a Data Engineering/Data Architecture role
- Experience working with Spark and other distributed data technologies (e.g. Hadoop, Presto, Druid) for building efficient & large scale data pipelines.
- Experience in designing streaming data pipelines (eg Kafka, Flink, Spark-Streaming)
- Growth mindset and ability to learn new technologies
- Experience in backend API development and management
- Experience with a graph database (eg Neo4j, TigerGraph)
- Familiar with Data Lake and Data Warehouse technologies
- Good understanding of software development life cycle, version control, code reviews, testing, data quality tools and frameworks