Senior Data Engineer

About the role

We are seeking a Senior Data Engineer to help design and build the next generation of our Data Platform as we continue to scale to larger customers and new jurisdictions. At Alpaca, Data Engineering encompasses financial transactions, customer data, API logs, system metrics, augmented data, and third‑party systems that impact decision making for both internal and external stakeholders.

Key responsibilities

• Design, build, and evolve the core data platform infrastructure, including distributed query engines, orchestration, warehousing, and cataloging.
• Own lakehouse infrastructure as code, managing deployments through Terraform and Ansible on Kubernetes.
• Build and maintain low‑latency streaming and CDC ingestion pipelines, as well as batch ingestion paths landing in Iceberg.
• Develop and scale the BI landscape to provide performant, self‑serve access to lakehouse data.
• Enforce platform reliability best practices such as monitoring, alerting, on‑call rotations, incident response, maintenance windows, runbooks, and SLAs.
• Partner with DevOps, Analytics Engineering, and other stakeholders to close infrastructure gaps and support new data requirements.

Required profile

• 5+ years of experience in Data Engineering, with at least 2 years building and operating scalable, low‑latency data platforms handling >100M events/day.
• Strong hands‑on experience running data infrastructure on Kubernetes, using Docker and Helm.
• Production experience with Infrastructure‑as‑Code tools such as Terraform, Ansible, and ArgoCD (or equivalents).
• Deep knowledge of distributed systems and operation of open‑source query engines like Trino or Presto.
• Expertise with object storage and open table formats, especially Apache Iceberg.
• Experience with streaming and CDC technologies (Kafka, Redpanda, Debezium).
• Proficiency in Python and SQL for building pipelines and platform tooling.
• Experience with Google Cloud Platform data services (GCS, Cloud Build, Cloud SQL, Dataproc, etc.).

Required skills

• Python
• SQL
• Kubernetes
• Docker
• Helm
• Terraform
• Ansible
• ArgoCD
• Trino or Presto
• Apache Iceberg
• Kafka
• Redpanda
• Debezium
• Airflow
• Airbyte
• Google Cloud Platform (GCS, Cloud Build, Cloud SQL, Dataproc)

What we offer

• Competitive salary and stock options
• Health benefits
• One‑time home‑office setup stipend of USD 500
• Monthly stipend of USD 150 via a Brex card