Senior Data Engineer
Project description
Build data pipelines and data products to enhance investment research capabilities for a top US & global investment firm.
Responsibilities
- - Collaborate closely with business stakeholders, product managers, and engineering teams to translate business objectives into technical solutions. - Design, build, and enhance data pipelines and integration frameworks supporting AI, analytics, and business applications. - Improve data quality, reliability, lineage, and monitoring. - Integrate data solutions with the organization's AI infrastructure (AWS based) and tools with security in mind. - Implement AI first approach: use AI agents as a primary tool to design, build, test, and support solutions
SKILLS
Must have
- - 10+ years of professional software development, including 5+ years of data engineering experience. - Advanced proficiency in Python, including experience building scalable backend services and data processing solutions. - Strong hands-on experience designing and implementing data pipelines, data integration solutions, with a latency and cost in mind - Experience working with relational databases and modern data storage technologies, including graph, vector, or analytical data platforms. - Experience with data quality and data governance approaches, including validations in the pipeline, entitlements, audit, and lineage - Experience working with MCP Gateway, including the security aspect. - Experience working with cloud platform to host AI solutions, such as Amazon Bedrock AgentCore. - Experience operating in agile delivery environments. - Strong communication skills and ability to interact efficiently with various stakeholder groups including business users. Experience working with financial domain users, especially investment business, is a big plus.
Nice to have
- Financial services, especially investment domain. - Experience with Databricks, Spark, Delta Lake, or similar modern data engineering platforms. - Experience with dbt. - Experience designing enterprise data lakes, lakehouses, or large-scale analytical platforms. - Experience with Aurora, Neptune, and other AWS data services. Strong hands-on experience designing and implementing data pipelines, data integration solutions, and large-scale data processing architectures. - Experience architecting, provisioning, and managing production infrastructure on AWS, including services such as ECS Fargate, Lambda, SQS, and related cloud-native technologies. - Experience defining cloud architecture programmatically using Infrastructure-as-Code (IaC) tools such as AWS CDK or Terraform.
Required Skills
Required Languages
🇬🇧 English