Sr. Designated Support Engineer, Apache Spark™
About this role
Troubleshooting complex performance bottlenecks by analyzing thread dumps, Spark UI, and driver logs forms the core daily work in this position. Strategic accounts receive high-touch support as this team manages critical issues and guides customers through architectural best practices on the Lakehouse platform. Databricks seeks an experienced technical professional to build close relationships with key clients and coordinate cross-functional resources toward issue resolution.
Working closely with engineering, product teams, and senior leadership, the incoming engineer removes roadblocks and drives customer self-sufficiency. Success in this role requires a strong background in distributed big data computing combined with executive presence and bilingual fluency in English and Portuguese.
What you will do
- Manage large strategic accounts to deliver a high-touch, proactive support experience.
- Investigate complex data engineering and Apache Spark issues by analyzing driver and executor logs, thread dumps, and the Spark UI.
- Advise customers on architecture best practices regarding data modeling and pipeline optimization.
- Coordinate with internal engineering, product, and account teams to accelerate support ticket resolution.
- Conduct regular case reviews to spot underlying trends and uncover proactive optimization opportunities.
- Advocate for clients by chairing roundtables and fostering communication with key stakeholders.
What you need
- Bring a minimum of three years of dedicated experience building and maintaining large-scale data pipelines.
- Demonstrate at least three years of experience developing and sustaining Python, Java, or Scala-based applications.
- Possess deep architectural knowledge of Apache Spark internals such as the Catalyst Optimizer, memory management, and DAGs.
- Show a proven track record of resolving big data performance issues including out-of-memory errors and data skew.
- Maintain bilingual professional proficiency in both English and Portuguese for daily collaboration and client support.
Nice to have
- Customer-facing experience as a Support Engineer, Technical Account Manager, DSE, or related production support roles
Skills and tech stack
Questions candidates ask
What is the work location policy for this position?
This role is hybrid and requires working at least three times a week in the office based in São Paulo, SP.
What level of language proficiency is required?
Mandatory bilingual proficiency is required with full professional fluency in English for collaboration with US-based teams and in Portuguese to support the Brazilian customer base.
What technical background is needed for the role?
Candidates need architectural knowledge of Apache Spark internals and a minimum of three years of experience in data engineering and Python, Java, or Scala development.