Senior Python Developer

Project description

We have a robust, hands-on engineering culture dedicated to continuous learning, knowledge-sharing, technical skill development and networking. We are an essential part of the Bank’s technology platform and develop applications for many important business areas. Your initial project will be a core AI & Data platform, designed to provide a best in class environment for building, running and scaling AI solutions. The platform provides tools, services, frameworks and data so teams can focus on solving business problems, not rebuilding infrastructure. • Data Scientists building analytics, ML and GenAI solutions • Engineers and Developers integrating AI into applications and workflows • Business users consuming AI-enabled insights and services As a Senior Engineer, you will be responsible for managing or performing work across multiple areas of the bank's overall IT Platform/Infrastructure including analysis, development, and administration.

Responsibilities

• Data Pipeline Development & Optimization: Design, develop, and optimize complex data pipelines using Python and PySpark/Apache Spark for large-scale data processing, transformation, and ingestion.
• Workflow Orchestration: Implement and manage data workflows and dependencies using Apache Airflow to ensure efficient and reliable data delivery.
• Containerization & Orchestration: Deploy and manage data engineering workloads and applications within containerized environments using Kubernetes, ensuring scalability, resilience, and efficient resource utilization.
• CI/CD Implementation: Design and implement robust CI/CD pipelines for automated testing, deployment, and monitoring of data engineering solutions, promoting a culture of continuous delivery and quality.
• Data Lake & Storage Management: Work with modern data lake technologies, including Apache Iceberg, for efficient data storage, versioning, and schema evolution.
• Object Storage Integration: Utilize object storage solutions for managing and accessing large volumes of unstructured and semi-structured data.
• Problem Solving: Proactively identify, diagnose, and resolve complex data-related issues, performance bottlenecks, and data quality challenges.
• Collaboration & Mentorship: Collaborate effectively with cross-functional teams including data scientists, analysts, software engineers, and product managers. Potentially mentor junior data engineers, sharing knowledge and best practices.
• Technical Leadership: Contribute to technical design discussions, propose innovative solutions, and drive the adoption of best practices within the data engineering team.
• Documentation: Create and maintain comprehensive technical documentation for data pipelines, architectures, and processes.

SKILLS

Must have

• Strong Python knowledge
• Familiarity with Apache Airflow, Apache Spark, Kubernetes
• Understanding of AI concepts like LLM, Embeddings, Vectors, RAG, MCP, Agents
• Java knowledge is a plus
• Good communication and interpersonal skills

Nice to have

• Java knowledge is a plus