We are looking for a Senior Data Engineer to implement and validate change data capture from AWS Aurora PostgreSQL into the Snowflake analytics environment. Initial focus: build integration metadata-driven frameworks to deliver data in near-real-time from PostgreSQL to Snowflake and adopt the new schema in the existing Medallion architecture. Collaboration with data platform, infrastructure, and architecture teams. Production rollout and full dimensional modeling are follow-on, not the first milestone.
This position is intended for Hyderabad-based candidates who are available to work from the office.
- Configure and operate Snowflake OpenFlow from Aurora PostgreSQL to Snowflake.
- Tune and troubleshoot Aurora PostgreSQL for logical replication.
- Provision infrastructure using Terraform and AWS IAM with least privilege access controls.
- Run structured proof of concept tests for large tables, high volume CDC, concurrency, and outage recovery scenarios.
- Document outcomes of validation and testing activities.
- Monitor replication health using Snowflake native observability and Amazon CloudWatch.
- Define clear success criteria with stakeholders.
- Support Snowflake side setup for managed ingestion in coordination with infrastructure.
- Apply security and governance practices for a regulated financial and member services context.
- Perform schema, DBT, and Snowflake native pipeline renewal while preserving business and data continuity.
- Execute performance and cost improvements.
- Follow agreed teardown procedures to remove long lived replication or access artifacts.
- Hands-on experience with Snowflake OpenFlow or Apache NiFi
- Experience with Python or similar languages for checks, automation, and reporting
- Strong experience with PostgreSQL or Aurora logical replication, WAL, slots, and access control
- Experience using Terraform on AWS with networking, security groups, IAM, and role pass-through for managed services
- Experience with DBT for data transformations
- Strong SQL skills for transformations, validation, and reconciliation
- Working knowledge of Snowflake warehouses, roles, and loading patterns
- Solid understanding of CDC and ELT fundamentals
- Experience applying a data quality mindset in delivery work
- Strong communication skills in Agile cross-team collaboration, including distributed teams
- Experience with GitHub
- Experience with Airflow or equivalent tools for production orchestration
- Experience in financial services or similar regulated domains
- Experience with cloud cost estimation for replication at scale
- Experience with EKS or Kubernetes to monitor and resolve issues in the OpenFlow or NiFi environment
- Experience with Cursor or Claude
- Experience with specification driven development