About this role
Scaling machine learning infrastructure efficiently remains a constant bottleneck for marketing tech platforms operating globally, a challenge this role directly addresses at Braze. As part of the Predictive and Generative AI group, the incoming engineer takes charge of the underlying platform responsible for training hundreds of customer-specific models and serving high-throughput predictions across multiple regions. This hands-on delivery position requires driving transformative architectural changes, such as replatforming orchestration queues and optimizing cloud identity, while ensuring production reliability and cost-effectiveness at scale.
Success in this capacity demands a seasoned technical leader who balances high personal output with cross-functional mentorship and collaboration. The ideal candidate brings extensive background in distributed systems and production-grade machine learning workloads, paired with deep familiarity with Kubernetes and cloud infrastructure. Someone who thrives on autonomy, acts with a strong bias toward action, and enjoys connecting complex infrastructure decisions directly to customer outcomes will find a natural home in this environment.
What you will do
- Identify and drive transformative initiatives that change how the team runs machine learning in production, including replatforming orchestration and overhauling cloud infrastructure.
- Build and ship complex infrastructure initiatives hands-on from design through production, such as multi-region model serving fleets and customer-specific training pipelines.
- Own the technical vision and production quality bar for the platform, establishing direction for model training, deployment, serving, and observation.
- Drive initiatives that span multiple teams, maintaining technical relationships for shared infrastructure, deployment tooling, and data systems.
- Raise overall engineering quality through design reviews, code reviews, and production readiness checks while mentoring senior engineers and data scientists.
- Connect technical decisions to broader customer and business outcomes, representing the technical perspective to product and engineering leadership.
What you need
- Bring 8 or more years of experience building and operating distributed systems in production with a strong focus on deployment and operations.
- Possess hands-on experience running machine learning workloads in production, including training pipelines, model serving, or platform tooling.
- Demonstrate a background as a technical leader who has directed team strategy, led multi-quarter cross-functional initiatives, and grown senior engineers.
- Maintain deep working knowledge of Kubernetes and cloud infrastructure, encompassing identity and access management, networking, and cost management.
- Communicate effectively through verbal and written mediums to build consensus and drive forward technical decision-making.
Nice to have
- Queueing and orchestration systems such as Celery, RabbitMQ, Kafka, or Ray
- ML platform tooling including MLflow, model registries, feature stores, or ML observability
- Experience in the Braze tech stack featuring Python, Ruby on Rails, MongoDB, Redis, and Kubernetes
- Operating under compliance regimes like SOX or HIPAA
- Customer engagement, personalization, or marketing technology domain experience
Benefits
- Competitive compensation that may include equity
- Retirement and Employee Stock Purchase Plans
- Flexible paid time off
- Comprehensive benefit plans covering medical, dental, vision, life, and disability
- Family services that include fertility benefits and equal paid parental leave
- Professional development supported by formal career pathing, learning platforms, and a yearly learning stipend
Skills and tech stack
Questions candidates ask
What is the hybrid work policy?
The position features a curated in-office employee experience designed to foster community, team connections, and innovation alongside flexible ways of working.
What is the salary range for this role?
For candidates based in the United States, the pay range at the start of employment is expected to be between $184,000 and $314,000 per year, with expected On Target Earnings between $204,000 and $348,000 per year including bonus or commission.
What technologies will I be working with?
The tech stack includes Python, Ruby on Rails, MongoDB, Redis, and Kubernetes, along with potential exposure to queueing systems like Celery, RabbitMQ, Kafka, or Ray, and platform tooling such as MLflow.
What kind of team will I join?
You will join the Predictive and Generative AI team, which delivers engaging and personalized customer experiences through ML and AI-enhanced marketing solutions operated at global scale.