About this role
Capital Technology Group needs a Lead Data Engineer to modernize federal data environments and build secure, scalable, and mission-driven analytics solutions that impact millions of people. In this role, the engineer will design, build, and maintain efficient data pipelines, prototype new tools, and implement AI-enabled data solutions leveraging Amazon Bedrock, RAG pipelines, and vector search technologies.
The person stepping into this position will partner with cross-functional Agile teams to deliver high-performance ingestion, transformation, and orchestration workflows using Python, PySpark, SQL, and AWS-native platforms. Ideal candidates are strategic problem solvers with extensive cloud-based solution design experience who balance business needs with technical constraints while mentoring junior engineers.
What you will do
- Design, build, and maintain scalable data pipelines, ETL/ELT workflows, and data models using Python, Apache Spark, SQL, and AWS Glue.
- Develop and optimize AWS-native data platforms leveraging Amazon EMR, Apache Airflow, S3, RDS, and CloudWatch.
- Build high-performance ingestion, transformation, and orchestration workflows for structured and semi-structured data using Apache Iceberg and Parquet.
- Design and optimize analytical data platforms utilizing Amazon Athena, Trino, OpenSearch, and enterprise data catalog technologies.
- Build AI-enabled data solutions using Amazon Bedrock, RAG pipelines, and vector search technologies.
- Develop cloud infrastructure using Infrastructure as Code tools like CloudFormation alongside GitHub and enterprise CI/CD pipelines.
- Support mission-critical analytics solutions within large-scale federal data environments while complying with security controls.
- Mentor junior engineers through technical guidance, architecture discussions, and code reviews.
What you need
- Hold a Bachelor's degree in Computer Science, Engineering, or a related technical field.
- Bring 8+ years of professional experience in data engineering, data architecture, or related fields.
- Demonstrate strong hands-on experience with Apache Spark, Python, SQL, and relational databases for large-scale data engineering.
- Possess extensive background working with AWS Glue, Amazon EMR, Apache Airflow, AWS Lambda, S3, and RDS.
- Show proven capability in developing scalable data pipelines and workflow orchestration solutions across enterprise environments.
- Have experience working with modern data lake technologies and data formats such as Parquet and Iceberg.
- Applicants must be US Citizens and be able to obtain Public Trust clearance.
Nice to have
- Experience supporting analytics or modernization initiatives for financial regulators or highly regulated environments
- Experience with Apache Iceberg and modern data lakehouse architectures
- Experience working with unstructured data processing, document processing, embeddings, vector search, and LLM-based data solutions
- Exposure to integrating LLMs and generative AI capabilities into enterprise data pipelines
- Experience designing data architectures supporting structured and unstructured data at scale
Benefits
- Remote Work
- Competitive Compensation Package
- Medical, Dental, and Vision
- Life Insurance, Short/Long Term Disability
- 401(k) with 4% matching
- Liberal PTO vacation policy
Skills and tech stack
Questions candidates ask
What is the remote work policy for this role?
The job location is listed as Remote. Hybrid roles will be specified in the job post.
What is the salary range for the Lead Data Engineer position?
The estimated salary range is $165k to $225k annually. The final offer may vary based on experience, skills, and other factors.
What clearance or citizenship is required to apply?
Applicants must be US Citizens and be able to obtain Public Trust clearance. This is a client requirement for the position.
What types of technologies and tools will I be working with?
You will use Python, Apache Spark, SQL, and AWS-native platforms like AWS Glue, Amazon EMR, and Airflow. The stack also includes AI tools like Amazon Bedrock, RAG pipelines, and vector search technologies.
The application is on the official Capital Technology Group careers page.
Apply on company site