Hadoop Operation Lead Engineer
Hyderabad, Telangana, India
Software and Services
The people here at Apple don’t just build products— they craft the kind of wonder that has revolutionized entire industries. It’s the diversity of those people and their ideas that inspires the innovation that runs through everything we do, from amazing technology to industry-leading environmental efforts. Imagine what you could do here. Join Apple, and help us leave the world better than we found it. A job at Apple is unlike any other you’ve had. You will be challenged. You will inspired. And you’ll be proud. At Apple, great ideas have a way of becoming phenomenal products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you could accomplish. We are looking to hire extraordinary individuals with strong communication and interpersonal skills and ability to work effectively across multiple business and technical teams who should also be a senior Hadoop lead focused on operations to administer/scale our multi-petabyte Hadoop clusters and related services that go with it. This role focuses primarily on provisioning, ongoing planning for capacity, monitoring, management of Hadoop platform and performance enhancement of application/middleware that runs on Hadoop. You should be a self starter, motivated, with exceptional interpersonal skills as well as a positive and flexible approach to problems. The applicant should have the ability to thrive in a fast-paced, tight deadline delivery timeline. As a part of this dynamic team, the applicant will help us build the Apple experience on a global scale.
- Hands on experience with handling production clusters (Hadoop, Spark) as well as strong development/automation skills and being very comfortable with reading and writing Python and Java codes
- Overall 10+ years with at least 5+ years of Hadoop experience in large scale, multi-tenant production clusters (1000+ nodes with Queues) with a tools-first mindset -'You build tools for yourself and others to increase efficiency and to make hard or repetitive tasks easy and quick.'
- Experience with Configuration Management and automation along with an Organized, focused on building, improving, resolving and delivering.
- Good communicator in and across teams along with strong team management and leadership skills
The role would primarily involve - - Taking responsibility for maintaining and scaling large scale (1000+ nodes) production Hadoop and Spark clusters as well as managing a team of Hadoop Engineers providing mentorship and allocation of tasks along with technical help required by engineering team - Strong knowledge of kerberos for operation and debugging kerberos issues in Hadoop clusters along with being the sole point of contact for Hadoop related critical issues coming from Application teams and internal clusters - Being responsible for the implementation and ongoing administration of Hadoop infrastructure including monitoring, tuning and troubleshooting and providing hardware architectural mentorship, plan and estimate cluster capacity, and craft roadmaps for the Hadoop cluster deployment - Improving scalability, service reliability, capacity, and performance of the cluster and applications running in the cluster - Triaging production issues when they occur with other operational teams, conducting ongoing maintenance across our large scale deployments across the world and writing automation code for running large BigData clusters - Working with development and QA teams to design Ingestion Pipelines, Integration APIs, and provide Hadoop ecosystem services and participating in on-call rotation supporting the Hadoop Eco system - Detailed experience in solving incidents, formulate theories and test hypothesis, and narrow down possibilities to find the root cause
Education & Experience
Bachelors or Master Degree in Computer Science or similar discipline is required
- Familiarity with geo spatial data would be an added advantage
- Apple is an Equal Opportunity Employer that is committed to inclusion and diversity. We also take affirmative action to offer employment and advancement opportunities to all applicants, including minorities, women, protected veterans, and individuals with disabilities.