Role Overview
Grafana Labs is seeking a remote Staff AI Engineer to own the AI agent infrastructure and automation platform powering the Marketing Operations organization in Canada. This high-autonomy role suits an experienced engineer who wants to define technical direction, build multi-agent architectures, and deploy production systems that teams depend on daily.
Key Responsibilities
Own the end-to-end development of multi-agent AI systems, covering architecture, implementation, testing, deployment, and ongoing operation.
Build modular, composable agentic systems using orchestration frameworks like LangChain or CrewAI to operate 24/7 across teams.
Develop reusable agentic skills invoked across interfaces such as Slack, dashboards, internal applications, and CLIs.
Implement observability and feedback loops, including performance metrics, logging, prompt iteration, model evaluation, and cost management.
Establish governance and compliance standards for AI workflows, encompassing access controls, audit trails, PII handling, and human-in-the-loop escalation paths.
Build Model Context Protocol (MCP) servers, APIs, CLIs, and microservices connecting AI models to external and internal business systems like BigQuery, CRMs, and Slack.
Architect data flows for retrieval-augmented generation (RAG) to connect large language models with internal knowledge bases and real-time business context.
Build serverless or containerized services using GCP Cloud Functions and Cloud Run that scale with usage.
Partner with RevOps, demand generation, regional marketing, and SDR teams to scope automation problems and deploy workflows using tools like n8n or Workato.
Requirements
8+ years of software engineering experience with a strong background in backend development, systems integration, or data and analytics engineering.
2+ years of hands-on experience applying LLMs and AI to production workflows rather than just prototypes.
Strong proficiency in Python and JavaScript or Node.js, alongside Git-based workflows, code review practices, and testing discipline.
Hands-on experience with LLM frameworks and patterns, including RAG, prompt engineering, structured output parsing, function calling, and evaluation.
Experience building and operating multi-agent systems at scale, including state management, agent decomposition, production monitoring, and orchestration patterns.
Deep familiarity with Google Cloud Platform, BigQuery, and serverless or containerized services such as Cloud Run and Cloud Functions.
Understanding of LLM failure modes and production mitigations, including confidence thresholds, fallback logic, human escalation, and latency management.
Comfort with AI-assisted development tools like GitHub Copilot, Cursor, and Claude Code.
Located in Canada (Remote), excluding residents of Quebec.
Nice to Have
Experience with vector databases or retrieval pipelines like Pinecone, Weaviate, ChromaDB, Qdrant, or pgvector.
Familiarity with sales or marketing platforms including Salesforce, HubSpot, Marketo, Customer.io, and Outreach.
Experience with frontend frameworks like React or Slack Block Kit for user-facing AI tool interfaces.
Observability tooling for AI systems such as LangSmith or Weights & Biases.
Experience with workflow orchestration platforms like n8n, Temporal, Prefect, or Airflow.
Familiarity with Model Context Protocol or similar standards for connecting AI systems to data sources.
Prior work automating B2B SaaS marketing, sales, or customer success workflows.
Active involvement in open-source communities.
Skills and Tools
Python
JavaScript
Node.js
GCP (Google Cloud Platform)
BigQuery
Cloud Functions
Cloud Run
LLMs
LangChain
CrewAI
RAG
Airflow
React
Git
Salary and Benefits
Base compensation range of CAD 164,490 to CAD 197,389 per year for candidates in Canada.
Restricted Stock Units (RSUs) provided with every role to offer ownership in company success.
Global annual leave policy of 30 days per annum, which includes 3 days reserved for Grafana Shutdown Days.
100% remote working culture with team members across more than 40 countries.
In-person onboarding program from day one.
About the Company
Grafana Labs is the creator of Grafana Cloud, a fully managed observability platform trusted by over 10,000 organizations, including industry leaders like Anthropic, Bloomberg, NVIDIA, Microsoft, and Salesforce. Built on open source and open standards, the company brings AI to observability and observability to AI to help teams see, understand, and act on all their disparate data. Backed by major investors such as Sequoia Capital, Lightspeed Venture Partners, and Coatue, Grafana Labs operates as a 100% remote company driven by transparency, autonomy, trust, and a strong open-source legacy.
How to Apply
To apply for this role, please visit the official Grafana Labs careers page and submit your application online.
