Operating Systems Engineer, On-Device Inference | Consumer Devices
About this role
OpenAI Consumer Devices is building the next generation of products that bring powerful AI into people's everyday lives, working across silicon, embedded systems, operating systems, and cloud services to create reliable consumer devices and novel platforms. The person in this role will design, develop, and ship the operating system stack that makes advanced artificial intelligence capabilities reliable, responsive, and energy efficient on consumer devices, spanning OS services, frameworks, inference runtime integration, model fitting, scheduling, and power management. Operating systems engineers who enjoy tackling zero-to-one challenges at the intersection of hardware, software, and AI while collaborating closely with research teams to adapt models to physical device constraints will find a strong fit here.
What you will do
- Design and implement maintainable operating system services, frameworks, and clear interfaces for inference execution, model lifecycle management, and resource allocation.
- Partner with research teams on quantization, runtime integration, and memory optimization to meet memory, compute, and energy budgets while maintaining model quality.
- Develop scheduling and resource policies that balance inference tasks with other device activities to preserve responsiveness under strict battery, thermal, and latency constraints.
- Create and validate execution strategies that dynamically adapt to workload needs, available device resources, and changing environmental conditions.
- Investigate correctness, concurrency, performance, and reliability issues across models, inference runtimes, and operating system components using structured profiling and tracing tools.
- Build diagnostic tools, instrumentation, and automated tests to demonstrate repeatable performance gains on physical devices and prevent regressions.
- Collaborate across research, hardware, firmware, platform, and product engineering to turn emerging model capabilities into maintainable systems and shipped features.
What you need
- Demonstrated hands-on experience designing, developing, and debugging operating system components, system services, or performance-critical platform software.
- Proficiency in C++ for systems development, encompassing concurrent programming, memory ownership, and resource lifetime management.
- Practical background in integrating or optimizing inference runtimes and machine learning workloads within resource-constrained environments.
- Solid understanding of scheduling mechanics, memory management, and how concurrent workloads compete for shared system resources.
- Track record of diagnosing complex system behaviors and delivering measurable performance or power improvements supported by repeatable data.
- Ability to bridge disciplines by translating research and product needs into clear system requirements while explaining technical tradeoffs.
Nice to have
- Delivered end-to-end on-device inference in shipped consumer products from model adaptation through production debugging.
- Adapted models for deployment via quantization or compression while balancing quality, memory footprint, and execution cost.
- Optimized inference workloads across CPUs, GPUs, or neural accelerators while accounting for data movement and synchronization.
- Proficiency in Rust for systems programming.
- Experience taking ownership of complete solutions across model, runtime, and operating system boundaries.
Benefits
- Offers Equity
Skills and tech stack
Questions candidates ask
What kind of projects will I work on?
You will design, develop, and ship the OS stack that makes advanced AI capabilities reliable, responsive, and energy efficient on consumer devices. Your work will span OS services and frameworks, inference runtime integration, model fitting, scheduling, and performance and power management.
What is the compensation for this position?
The salary range for this role is $230K – $385K and includes offers of equity.
Who will I be collaborating with?
You will partner closely with research to adapt models to device constraints and collaborate with research, hardware, firmware, platform, and product engineering teams to turn emerging model capabilities into maintainable systems.
What technical background is required for this role?
Candidates need substantial hands-on experience designing and debugging operating system components, proficiency in C++ for systems development, and practical experience integrating or optimizing inference runtimes in resource-constrained environments.