Member of Technical Staff (Search Core, Indexing & Data Pipeline Stream)
About this role
Engineers who thrive on low-level optimization and hardware-adjacent challenges will find a natural home within the Search Core team at Perplexity. This group focuses on building next-generation search products and technologies, specifically driving the indexing and data processing stream that handles billions of pages. Daily work involves designing foundational search engine components, implementing streaming and batch data processing systems, and pushing hardware limits through systems-level tuning and concurrency management.
Perplexity created this position to shape the architecture, design, and implementation of core infrastructure operating at massive scale. Ideal candidates bring extensive systems programming experience in languages like Rust or C++ alongside a strong grasp of Linux internals, multithreading, and distributed services. Anyone joining this team must care deeply about the full project lifecycle, from initial implementation to production deployment and operational scaling in high-load environments.
What you will do
- Design and build core search engine components, including indexing pipelines, retrieval algorithms, and ranking systems that operate at the scale of billions of pages.
- Develop streaming and batch data processing systems for search index construction in a high-load environment.
- Push the limits of hardware performance through low-level optimizations and systems-level tuning.
- Tackle complex challenges in multithreading, concurrency, and system-level optimization.
What you need
- Bring three or more years of hands-on experience in systems programming using Rust, C++, C, or similar languages.
- Demonstrate ownership of the full project lifecycle from building fast inner loops to deploying, operating, and scaling systems in production.
- Possess knowledge of Python or other scripting languages along with data processing systems like YT, Kafka, or Hadoop.
- Showcase a strong command of algorithms, data structures, and multithreading trade-offs.
- Exhibit a solid understanding of Linux internals, including syscalls, the networking stack, memory models, and kernel tuning.
- Apply familiarity with low-level optimization techniques such as memory management, cache efficiency, SIMD, and profiling.
Nice to have
- Experience developing core components of search engines, databases, or information retrieval systems
- Understanding of search fundamentals including indexing, query parsing, ranking, and relevance
- Experience with trading systems or other latency-sensitive real-time systems
- Familiarity with cloud services, Kubernetes, and AWS infrastructure
Skills and tech stack
Questions candidates ask
What team will I be joining?
You will join the Search Core team specifically for the indexing and data processing stream. This role is critical to building next-generation search products and technologies.
What will I work on regarding the tech stack?
You will develop streaming and batch data processing systems for search index construction and build core search engine components that operate at the scale of billions of pages. You will also use languages such as Rust, C++, C, and Python, while interacting with data processing systems like YT, Kafka, and Hadoop.
What level of experience is required?
The position requires three or more years of hands-on experience in systems programming. Candidates need a deep understanding of multithreading and solid knowledge of Linux internals.