SW Big Data Systems Engineer
Santa Clara Valley (Cupertino), California, United States
Software and Services
The SWE Data Analytics team at Apple collects, processes, and analyzes diagnostics and usage data from Apple devices across the world. Our data is used to generate insights that informs and drives product strategies across all of software and hardware development. We develop batch and streaming analytics solutions using Kafka, Flume, Hadoop, Spark, Jenkins, and other state of the art technologies in a large scale infrastructure. We are looking for a passionate and results-oriented senior big data engineer to join our team and work on some of the highly visible data projects in software engineering organization. You will be collaborating with data analysts, device engineers and engineering teams. You will drive the development of data pipelines and services with high degree of ownership.
- Experience developing large scale distributed computing systems
- In-depth knowledge and experience in one or more of the following technologies: Hadoop ecosystem, Kafka, Samza, Flume, HBase, Cassandra, Redshift, Vertica, Spark.
- Deep understanding of key algorithms and tools for developing high efficiency data processing systems
- Validated software engineering experience and discipline in design, test, source code management and CI/CD practices
- Experience in data modeling and developing SQL database solutions
- Proficient in working with Linux or other Posix operating systems, shell scripting, and networking technologies
- Strong software development, problem-solving and debugging skills with experience in one or more of the following languages: Java, Python, Scala, or Ruby
- Ambitious, passionate about software development, especially in data technologies, you love working in a fast-paced and dynamic environment
- You are deeply organized, detail oriented, and thorough in every undertaking. You are able to multi-task and change focus quickly
- Excellent interpersonal skills.
As part of a small team of highly skilled data engineers, you will own significant responsibility in crafting, developing and maintaining our large-scale ETL pipelines, storage, and processing services. You will build self-service analytics tools to help engineering teams derive actionable metrics out of large volumes of raw data. You will partner with data science and engineering teams and develop algorithms to answer complex questions on usage of our products You will work closely with the DevOps team and develop monitoring and alerting scripts on various data pipelines and jobs You will have the opportunity to learn and work on the latest Big Data technologies, lead POCs to demonstrate new ideas and influence the future direction of our technology stack
Education & Experience
Bachelors in Computer Science or equivalent experience.
- Experience using data storage technologies such as Apache Parquet or Avro
- Experience in machine learning algorithms is a plus
- Testing tools and methodologies to test large scale distributed computing systems