Perception Engineer - Multimodal & Embodied AI (R6022)
About this role
The day-to-day work for this role involves developing perception capabilities for autonomous systems within Shield AI's Hivemind Detection, Tracking, and Mapping (DTM) team. The engineer will translate research and partner-developed technology into reusable, configurable product capabilities by building algorithms and production C++ libraries integrated into the Hivemind SDK. This is a full-stack perception product-development position requiring a deep understanding of core computer vision or machine learning areas. Candidates will evaluate emerging models against real sensor data, build prototypes to understand system limitations, and combine theoretical knowledge with classical methods to enhance detection, tracking, and sensor fusion.
This role exists to bridge the gap between advanced AI research and deployable, robust product features for Shield AI's defense technology. The successful candidate will be responsible for optimizing these capabilities for edge deployment, ensuring they meet the operational needs of autonomous systems across air, maritime, and ground domains. The ideal candidate possesses a strong foundation in deep learning research and the engineering discipline to deliver production-quality software.
This role is suited for individuals who possess significant research depth in perception domains and a strong drive to transition that knowledge into tangible product features. Those who enjoy iterating between research, prototyping, and production engineering, and who are curious about how AI impacts complex operational environments, will thrive in this position.
What you will do
- Evaluate emerging models and methods against representative sensor data and autonomy needs.
- Build prototypes and experiments to determine the strengths, limitations, and practical deployment opportunities of new technologies.
- Combine learned and classical approaches to improve object detection, tracking, geometric vision, or sensor fusion.
- Optimize and integrate capabilities for edge deployment using both Python and C++ software.
- Contribute software to the Hivemind SDK while collaborating with experienced product engineers.
- Deliver tested, documented capabilities and maintain involvement as other teams adopt these features.
What you need
- A PhD in computer vision, machine learning (ML/DL), robotics, or a related field, or an equivalent record of advanced research and hands-on engineering.
- Demonstrated research depth in an area such as multimodal vision, visual foundation models, video understanding, open-vocabulary perception, or embodied learning.
- Strong proficiency in Python coupled with experience using PyTorch or a comparable machine learning framework.
- A thoughtful methodology for conducting experiments, evaluating data, and diagnosing reasons for model failures.
- Experience building substantial research software and a genuine interest in developing production-quality C++ code.
- A collaborative approach necessary for solving complex, cross-disciplinary engineering problems.
Nice to have
- Experience bringing a research prototype into sustained use.
- Experience with robotics, real sensors, temporal data, or multi-sensor perception.
- Familiarity with vision-language models, vision-language-action models, or embodied AI.
- Experience with C++, edge inference, or model optimization using tools such as ONNX, TensorRT, or CUDA.
Benefits
- Pay within range listed + Bonus + Benefits + Equity
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
The posting does not explicitly detail the remote work policy. The role is listed as a Full Time Employee position located in San Diego, California.
What is the compensation structure?
The compensation package includes pay within a listed range, a bonus, benefits, and equity. Salary compensation is influenced by factors like skill set, level of experience, licenses, and work location.
What is the primary technology stack used?
The core technologies mentioned include Python, C++, and PyTorch for machine learning frameworks. Preferred experience includes tools like ONNX, TensorRT, or CUDA for edge inference optimization.
What is the team structure like?
The role is within the Hivemind Detection, Tracking, and Mapping (DTM) team, which focuses on developing perception capabilities for autonomous systems across various domains.