Senior Data Engineer - Team *instinctools

*instinctools is a software development company that provides custom software solutions for businesses of all sizes. Our team works closely with clients to understand their specific needs and provide personalized solutions that meet their business requirements.

*instinctools is looking for a  Senior Data Engineer to join a project for one of the TOP-5 global management consulting firms.

The client works with Fortune 500 companies across industries, including financial services, technology, and media, providing platforms and services that support complex business decisions.

The project is a data-driven solution that helps companies optimize promotional activities and maximize return on investment. It collects and validates data, analyzes promotion effectiveness, supports promotional planning, and integrates with existing business systems. By combining data analytics and machine learning, the platform helps businesses make informed decisions and improve commercial outcomes.

As a Senior Data Engineer, you will contribute to building and optimizing the data infrastructure that powers these capabilities, with a focus on scalability, reliability, data quality, and performance.

What you will get hands-on with:

• Modern data engineering: Work with Databricks, Apache Spark, SQL, and Python to build and optimize scalable data pipelines.

• Lakehouse architecture: Apply Delta Lake capabilities, including Delta Live Tables, Unity Catalog, and Delta Sharing.

• Data pipeline orchestration: Develop automated workflows using Airflow and Azure Data Factory.

• Cloud-based data solutions: Work with cloud platforms, with Azure as the preferred environment.

• Engineering best practices: Apply CI/CD pipelines, testing, modularization, refactoring, and automation.

• Analytics engineering: Work with data modeling and analytics frameworks, including dbt.

• End-to-end technical ownership: Participate in solution design, implementation, optimization, and continuous improvement.

You’ll work on a variety of data engineering challenges, from pipeline development and data processing to performance optimization and data quality. The focus is on building reliable production solutions, not just maintaining existing workflows.

What you will do:

• Design, build, and optimize scalable and reusable data pipelines using Databricks, SQL, and Spark.

• Orchestrate automated ETL workflows using Airflow, Azure Data Factory, and CI/CD pipelines.

• Implement and maintain scalable data processing systems, ensuring performance, reliability, and cost-effectiveness.

• Ensure data quality, consistency, and compliance with governance and security standards.

• Collaborate with stakeholders to clarify requirements and translate them into technical solutions.

• Continuously evaluate and implement tools and technologies to improve pipeline automation, monitoring, and efficiency.

• Take ownership of technical solutions and contribute to engineering best practices.

• Share knowledge and support other engineers in solving complex technical challenges.

What we expect from you:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

• 6+ years of experience in Data Engineering, with strong big data expertise.

• Deep expertise in Databricks, Spark, and SQL, including Spark SQL.

• Strong data modeling and ETL experience with large-scale datasets.

• Hands-on experience orchestrating data pipelines with Airflow and CI/CD.

• Proven experience with Delta Live Tables, Unity Catalog, Delta Sharing, and SQL Warehouse (server and serverless).

• Proficiency in Python for data engineering workflows.

• Familiarity with analytics engineering and dbt.

• Experience with data governance, compliance, and security in a cloud environment.

• Strong problem-solving ability, requirement clarification, and communication skills.

• Experience applying software engineering best practices, including testing, modularization, refactoring, and automation.

• Ability to optimize data systems for performance and cost efficiency in cloud environments.

• Exposure to AI/ML frameworks for development efficiency.

• Demonstrated ability to lead projects independently and mentor junior engineers.

Soft skills:

• Strong self-management and the ability to work independently and meet deadlines.

• Proactive, ownership-driven mindset.

• Effective communication, including in distributed teams.

• Collaborative approach, openness to feedback, and focus on shared goals.

• Strong problem-solving skills, especially in ambiguous contexts.

• High attention to detail and code quality.

• Adaptability to changing priorities and new technologies.

English: Upper-Intermediate or higher
Working time zone: IST + EU overlap

What we offer:

• A project for a TOP-5 global management consulting firm. Contribute to a technology platform used by major enterprises and gain experience with a globally recognized client.

• Challenging engineering work. Work with modern data engineering technologies and solve complex problems related to data processing, scalability, performance, and reliability.

• Varied technical tasks. Contribute to pipeline development, data modeling, integrations, automation, and optimization rather than focusing exclusively on repetitive maintenance.

• A dedicated office in Noida. Work from our own office and collaborate with colleagues in a professional, supportive environment.

• A hybrid work model. Four days in the office and one day working remotely each week.

• A real team and structured onboarding. Get support from an onboarding buddy and build working relationships with your colleagues and delivery manager.

• Regular feedback and professional guidance. Have regular one-to-one meetings with your delivery manager and receive feedback from the client’s technical leads.

• Professional development. Access learning opportunities, seminars, conferences, and time to explore new technologies.

• Certification support. Benefit from co-funded certifications in AWS, Azure, and Kubernetes.

Important Notice Regarding Your Personal Data Security

Thank you for your attention and cooperation.

share Margarita Lukyanenka Recruiter We will be glad to see you in our team!

Developers

*instinctools is a developer of IT solutions for Western Europe. We have over 350 projects in our portfolio. The spheres are different – from healthcare and medicine, training and education, to digital advertising, economics and IT. For the software implementation and maintenance across such a number of projects, competent developers are important.

We use a modern technology stack, the key of which are:

Javascript, ReactJS, Angular, Node.js, Golang, Python, Java, iOS, Android, Blockchain, AWS.

Join the *instinctools team!

*instinctools Poland
We have had a representative office in Poland (Warsaw) since 2021. Our Polish team consists of highly qualified and experienced specialists – most of them are Middle to Senior level Developers.
Right now we’re on the lookout for bright and innovative minds to join our professional team!