About this role
Embedding directly with enterprise clients in Tokyo, day-to-day work involves architecting and shipping production systems that leverage global connectivity, voice, messaging, AI, and wireless infrastructure. Collaborating closely with an enterprise account executive within a localized sales pod, the engineer leads technical discovery, designs architectures, builds proofs of concept, and drives implementations from initial whiteboard sessions to stable production go-lives. Technical ownership spans custom AI voice assistants, API integrations, and the deployment of open-weight large language models alongside robust observability and model routing strategies.
The role exists to establish and expand the enterprise business presence in the Japanese market through outcome-focused technical engagements rather than standard product demonstrations. Success requires an autonomous engineer who thrives in direct customer-facing environments, possesses strong hands-on software development skills, and can independently navigate complex integration challenges without relying on an external support team. Candidates who combine deep infrastructure and AI expertise with effective communication capabilities will perform well in this hybrid position.
What you will do
- Embed with enterprise customers to understand communications workflows, AI use cases, and integration challenges firsthand.
- Build and deploy custom implementations including AI Voice Assistants, Telnyx APIs, and WebRTC solutions.
- Run open-weight LLMs in customer environments through Telnyx Inference or deploy self-hosted stacks behind serving engines.
- Deploy and operate LiteLLM as a model gateway with routing, load balancing, retries, fallbacks, and rate limits.
- Instrument and govern LLM usage through gateways with cost tracking, budgeting, caching, and observability tools.
- Design model routing strategies for real-time voice workloads while balancing latency, cost, and quality.
- Adapt models to customer domains using prompt and RAG pipelines for English and Japanese language use cases.
- Lead proofs of concept, pilots, and production launches while owning customer outcomes until solutions remain stable.
What you need
- Possess a computer science degree or equivalent professional software engineering experience.
- Bring at least three years of experience building and shipping production software with a history of handling on-call duties and debugging complex failures.
- Demonstrate proficiency in multiple programming languages such as Python, Node.js, TypeScript, or Go.
- Show practical understanding of production failure modes including provider rate limits, timeouts, streaming, and concurrency issues.
- Have experience deploying containerized services on Kubernetes with proper secrets management and upgrade procedures.
- Maintain high-concurrency experience working with Kafka, message queues, or event streams at scale.
- Exhibit customer-facing engineering experience capable of running technical discovery and presenting solutions to executives.
- Communicate fluently in English and Japanese for whiteboard sessions, live troubleshooting, and client escalations.
Nice to have
- Experience with AI voice assistants, STT/TTS, or LLM-based conversational systems
- Building eval sets for a specific domain
- Hands-on production experience with an LLM gateway such as LiteLLM, Portkey, Kong AI Gateway, or an in-house proxy
Skills and tech stack
Questions candidates ask
What is the work location and remote policy?
This is a hybrid role based in Tokyo, Japan, or for those willing to relocate, with required travel across Japan and the broader APAC region.
What tools and technologies will I be working with?
The tech stack includes Python, Node.js, TypeScript, Go, Kubernetes, Kafka, LiteLLM, WebRTC, and various observability tools like Prometheus, Grafana, and OpenTelemetry.
What is the team structure for this role?
You will work in a local Enterprise Sales Pod in Tokyo consisting of one Account Executive and one Forward Deployed Engineer to build the enterprise business in the market.