What you will be doing

We are looking for an experienced Principal Data Engineer to lead the design and evolution of the data platform that powers our AML/KYC  and Fraud products. Our platform depends on ingesting, transforming and serving billions of signals every day: sanctions and watchlist data, adverse media, corporate registries, transaction events and customer records, all flowing into a real-time financial crime intelligence knowledge graph used by thousands of customers across the world.

As a Principal Data Engineer you will set the medium to long term technical direction for our data infrastructure. You will partner with Engineering, Data Science, Product and SRE leadership on cross-tribe initiatives, coach engineers at every level, and tackle the data problems that no single team can solve alone. You will also represent ComplyAdvantage at engineering and industry events.

Your impact will shape the data foundations on which ComplyAdvantage's AI, screening and monitoring products are built. Your work will directly affect how quickly and accurately our customers can detect money laundering, terrorist financing, sanctions evasion and other financial crime, and make that crime a thing of the past.

Scope of the role

Scope of Principal Engineers at ComplyAdvantage

• Sets the medium to long term technical direction for the data domain, working with the VP of Engineering and senior data science, data governance and engineering leaders.
• Leads the architectural design of complex, business-critical data systems that span multiple tribes and affect the whole company.
• Shapes how engineering teams work across the organisation, including data quality standards, tooling and ways of working.
• Tackles the hardest data problems and ships work with direct impact on company goals.
• Acts as an active interviewer, helps improving the hiring process, and coaches engineers across the engineering organisation.
• Represents ComplyAdvantage at meet-ups, conferences and industry forums.

Data engineering and engineering skills required of the role

• Architect petabyte-scale data platforms across batch, micro-batch and streaming, making explicit trade-offs between latency, throughput, cost and operational complexity.
• Design and own the lineage, quality, freshness and observability of the financial crime knowledge graph and the pipelines that feed it.
• Build and evolve the foundational data infrastructure: ingestion frameworks, the event bus, feature and serving stores, the lakehouse, orchestration and the developer experience around them.
• Set the standard for event-sourced and streaming patterns across the company using Kafka and similar technologies, and drive consistency in how services produce and consume data.
• Design data services with scale and ease of operation in mind. Write maintainable, performant, well-tested Python code (and where appropriate Kotlin or Python), and review the work of others.
• Partner with ML engineers and data scientists so the platform supports feature engineering, training pipelines and online inference at scale.
• Set the data quality, schema evolution and contract-testing standards that other engineering teams adopt.
• Integrate the data platform with new and existing services. Build and consume APIs and event streams, and produce documentation that engineers and analysts can self-serve from.
• Coach staff, senior and mid-level engineers across the tribe and the wider engineering organisation, and build the bench of future technical leaders.

As a Principal Data Engineer at ComplyAdvantage

• You will own the technical architecture of the data platform behind our sanctions, PEP, adverse media, transaction monitoring, fraud and customer risk products.
• You will lead the architecture that supports ML, data science, and product teams ship new detection models and risk signals in days rather than quarters.
• You will design the data foundations that make agentic AI work at scale: retrieval pipelines, grounding sources, tool data and the event histories that let agents reason over our knowledge graph.
• You will work hand in hand with our Customer Risk, Fraud, Knowledge Graph and Screening tribes so the data foundations keep pace with the product and AI roadmap.
• You will set the technical direction for how we ingest, normalise and merge entity, relationship and event data from millions of public and private sources.
• You will be the deciding voice on company-wide data architecture decisions, the make-or-buy choices that follow, and our long-term vendor and tooling strategy for the data estate.

Our Tech Stack:

• Our technology stack is designed to run on public cloud architectures, notably AWS and GCP
• Development is organised around Kotlin and Python for our backend languages and TypeScript/ES6+React for our frontend stack
• We make substantial use of relational database technologies, notably Postgres, Yugabyte
• We also use an event-sourced model powered by Kafka for our communication bus and gRPC for our intra-service communication protocol
• We use modern observability solutions from Grafana Cloud and deploy our code using ArgoCD

We have a strong emphasis on engineering excellence and strive to ship the best possible code and the best possible solutions to our customers

About you

As a Principal Data Engineer with deep impact in a real-time environment, you will bring:

• Substantial experience designing and operating production-grade data platforms at high scale, whether that is large request volumes, large data volumes, or both.
• Deep expertise in distributed data systems: streaming (Kafka or similar), batch and ELT/ETL frameworks (Spark, Flink, dbt, Airflow or Argo Workflows), and modern lakehouse or warehouse technologies.
• Strong production Python experience and awareness of other relevant languages (Java/Kotlin) sufficient to set direction, review code and coach others.
• Experience designing for cloud (AWS and GCP) and containerised infrastructure (Kubernetes, Docker, ArgoCD).
• A track record of treating data quality, observability and data contracts as first-class engineering concerns rather than afterthoughts.
• Strong working understanding of logging, monitoring, alerting and incident management tooling for data systems.
• Excellent written and verbal communication. You can produce technical documentation that senior leaders and engineers can act on.
• Ownership of software and data products from inception through to production and long-term operation.
• A track record of coaching staff, senior and mid-level engineers, and of helping Recruiting improve hiring and onboarding.

Nice to have

• Experience building or operating data systems in financial services, AML, KYC, fraud, regtech or another regulated domain.
• Familiarity with knowledge graph and entity resolution problems: deduplication, linkage, hierarchies and temporal relationships.
• Experience supporting ML, LLM and agentic AI workloads, including feature stores, vector stores, retrieval pipelines, tool data and online/offline parity.
• Experience representing engineering externally at conferences, meet-ups or in technical publications.

Benefits:

• Equity participation in our innovative mission to combat financial crime
• Unlimited Time Off Policy to promote work-life balance and well-being
• We embrace a hybrid approach that requires employees to be in the office for two days a week. We strongly believe that this approach fosters collaboration and enables the building of meaningful relationships
• Opportunities for collaboration and career development with smart, like-minded professionals
• Annual learning budget to support professional growth
• A home office budget to support working from home
• Enhanced parental leave and childcare benefits
• Life insurance and medical coverage through BUPA, including pre-existing conditions
• Pension contribution through The People's Pension