Senior BigQuery Engineer

Project description

Join our Development Center in Bucharest, and become a member of our open-minded, progressive and professional team. In this role you will be working on projects for one our world famous clients. You will have a chance to grow your technical and soft skills, and build a thorough expertise of the industry of our client. On top of attractive salary and benefits package, Luxoft will invest into your professional training, and allow you to grow your professional career.

Responsibilities

• We seek a highly skilled Senior BigQuery Data Engineer with strong Python expertise to design and deliver high-performance data solutions. This pivotal role supports complex reporting functions and leads the migration of on-premise data platforms to Google Cloud Platform (GCP). You will manage large-scale financial datasets, ensuring secure, compliant, and low-latency data solutions for analytics and decision-making. This role demands technical mastery, problem-solving acumen, and a deep understanding of data best practices in the financial sector. Key Responsibilities: Data Platform Architecture & Modernization:
• Design, develop, and maintain scalable BigQuery data warehouse solutions (architecture, schema, data modeling).
• Lead cloud-native data warehouse design and implementation to replace on-premise infrastructure.
• Optimize BigQuery datasets, tables, and views for performance, cost, and governance through partitioning, clustering, query tuning, and workload management. Data Pipeline Engineering:
• Build robust data processing pipelines and workflows feeding BigQuery from diverse sources.
• Develop and maintain efficient ETL/ELT processes for large-volume financial data ingestion, transformation, and loading.
• Implement comprehensive data quality checks, validation, and reconciliation processes. GCP Ecosystem & Python Development:
• Utilize and optimize core GCP services: BigQuery, Cloud Composer (Apache Airflow) for workflow orchestration, Dataflow (Apache Beam) for batch/streaming, and Cloud Storage.
• Develop, test, and deploy high-quality Python code for data processing, automation, API integrations, and custom solutions, adhering to best practices (Git, testing, documentation).
• Explore and integrate other relevant GCP technologies (e.g., GKE, Cloud Functions, Pub/Sub) and implement security, access control, and compliance. Collaboration & Agile Practices:
• Collaborate with product owners, analysts, and engineering teams to align solutions with business requirements.
• Actively participate in agile ceremonies and foster team collaboration. Business Intelligence & Financial Expertise:
• Support data needs for BI tools (Qlik Sense, Tableau, Looker).
• Apply strong understanding of banking/financial sector data, regulations, and reporting for compliance.

SKILLS

Must have

• 5+ years of data engineering experience.
• Strong SQL knowledge, preferably on Google Cloud Platform (GCP).
• Expert-level Python programming skills for data manipulation, scripting, and application development.
• Proven experience with Cloud Computing, demonstrating practical application of cloud services.
• Direct hands-on experience with core GCP services: BigQuery, Cloud Composer (Apache Airflow) for DAG development/orchestration, Dataflow (Apache Beam) for batch/streaming pipelines, and Cloud Storage.
• Good understanding of data warehousing concepts, data flows, and data feeds.
• Experience with Git-based source control, particularly Bitbucket.
• Experience working in an agile development environment with familiar agile ceremonies.
• Solid understanding of the banking and financial sector, including data types and regulatory landscapes.
• Ability to work collaboratively in a dynamic environment.

Nice to have

• Experience with other GCP services (GKE, Cloud Functions, Pub/Sub, Dataproc). • Familiarity with CI/CD pipelines and DevOps practices. • Experience with Infrastructure as Code (Terraform). • Knowledge of data visualization tools (Qlik Sense, Tableau, Looker). • Understanding of data governance, data quality, and metadata management. • Experience with real-time data processing.