Multimodal ML Engineer
Cupertino, California, United States
Hardware
The Health Sensing team builds outstanding technologies to support our users in living their healthiest, happiest lives by providing them with objective, accurate, and timely information about their health and well-being. As part of the larger Sensor SW & Prototyping team, we take a multimodal approach, using a variety of sensors across HW platforms, such as camera, PPG, and natural languages.
Key Qualifications
- Expertise in multimodal ML
- Deep understanding in one of the following domains: computer vision, NLP, LLM, generative models
- Proficiency in Python and ML frameworks e.g. PyTorch, Tensorflow
- Familiar with software development standard methods/collaborations
- Sufficient SW skills to keep large ML training jobs going efficiently on a distributed backend with large volume of data (e.g. image, video)
- Excellent interpersonal skills; comfortable in a collaborative and ground breaking research environments
Description
In this role, you will be at the forefront of inventing AI/ML algorithms with real-world applications and ensuring the efficient evaluation of these models to be in production at scale. You will be interacting closely with a variety of ML researchers, software engineers, hardware and designers. You will address open-ended business objectives, from the ML problem and navigate through ambiguities to focus on technology development. You will be delivering solutions on time and with high quality standing up to the standards considering a customer facing product.
Responsibilities:
Research, develop and improve multimodal capabilities (e.g. vision-language). Examples include, but not limited to, aligning language guidance to the space of visual perception, aligning image recommendations to the space of conversation understanding and behavioral modeling
Adapt pre-trained models for downstream tasks; building evals for new modal capabilities
Work across the entire ML development cycle, from setting up data collection pipelines to model evaluation
Analyzing model behavior and finding weaknesses; drive design decisions with in-depth failure analysis
Team up with algorithm engineers to build end-to-end pipelines that prioritize rapid iterations in support for reliability of a complex multi-year project
Work cross-functionally to bring algorithms to real-world applications; this can span a wide range of teamworks with designers, clinical authorities, engineering specialists across HW and SW
Education & Experience
BS and a minimum of 3 years relevant industry experience