Principal Architect - Databricks

Own the Technical Side of Databricks Pursuits

• Lead qualification, technical discovery, target architecture design, effort estimation, and risk assessment.

• Develop technical content for proposals and Statements of Work (SOWs).

• Build and maintain technical relationships with customer architects and stakeholders.

• Provide customers with realistic and accurate assessments of migration complexity, implementation effort, AI workload readiness, and associated risks.

Provide Architectural Oversight Through Delivery

• Review architecture decisions throughout project delivery.

• Confirm that the delivered solution aligns with what was originally scoped and proposed.

• Surface technical risks and issues early in the engagement.

• Remain closely involved throughout delivery to ensure accountability for architecture and outcomes.

Shape Databricks Service Offerings

• Define the technical content of market-facing Databricks offerings.

• Establish what is included in scope, how success is measured, and what evidence validates successful completion.

• Create repeatable delivery standards and frameworks that teams can execute consistently.

Partner with Databricks

• Engage directly with Databricks field teams and specialist architects on joint customer accounts.

• Build productive relationships with Databricks technical teams.

• Effectively collaborate and navigate technical challenges, feedback, and differing viewpoints from vendor architects and specialists.

Lakehouse Platform and Architecture

• Account and workspace design

• Unity Catalog and metastore architecture

• Compute strategy across interactive, jobs, and serverless workloads

• Open table format decisions

• Cost architecture and design choices that affect long-term scalability and affordability

The ability to design platforms that remain technically and financially sustainable at enterprise scale is critical.

Migration and Consolidation

• Migrating organizations from Hadoop and Spark environments

• Modernizing legacy analytics and statistical platforms

• Consolidating established warehouses and appliance-based solutions

• Migrating workloads from other cloud platforms

Key areas of expertise include:

• Platform assessments

• Wave planning and migration strategy

• Workload and code conversion

• Reconciliation and parity validation

• Cutover planning and execution

This is expected to represent a significant portion of the work within the practice and requires genuine expertise rather than general familiarity.

Data Engineering and Pipelines

• Batch data ingestion

• Streaming data ingestion

• Declarative pipeline development

• Workflow orchestration

• Testing and validation frameworks

• Data quality instrumentation

• Building and operating reliable, production-ready data engineering solutions

Machine Learning and AI Engineering

• Feature engineering

• Experiment tracking

• Model registry and serving

• Evaluation and validation frameworks

• Retrieval and grounding architectures

• Agent development

This represents one of Databricks’ key differentiators and is an area where many customers require assistance moving from proof-of-concept solutions to production-grade, supportable implementations.

Governance at the Perimeter

• Catalog-based access controls

• Data classification

• Data masking

• Data lineage and governance frameworks

Candidates should understand how Databricks governance integrates with enterprise governance strategies and be able to address challenges that exist across organizational and platform boundaries.

Consumption Economics

• Compute sizing strategies

• Workload placement optimization

• Serverless versus classic compute trade-offs

• Consumption attribution and chargeback models

• Cost management and optimization practices

Candidates must be comfortable discussing platform costs with customers and providing practical guidance on controlling and forecasting consumption.

Technical Depth Expectations

Candidates do not need to be equally strong across all six areas; however, they must possess:

• Deep expertise in lakehouse architecture and platform design

• Deep expertise in migration and consolidation

• Deep expertise in data engineering

• Sufficient proficiency in the remaining areas to recognize when specialist support should be engaged

Databricks Expertise

• Substantial hands-on Databricks experience gained through:

  • A Databricks partner organization

  • Databricks directly

  • Managing a significant Databricks Lakehouse environment internally

• This role requires Databricks-specific expertise rather than general data platform leadership experience.

Enterprise Migration and Consolidation Experience

• Proven success delivering enterprise-scale migration and consolidation initiatives.

• Ability to discuss at least one major migration project in detail, including lessons learned and outcomes.

Production Machine Learning and AI Experience

• Experience operating and supporting production machine learning or AI solutions.

• Expertise beyond proof-of-concept implementations.

• Demonstrated understanding of the operational, governance, and maintenance considerations required for production AI workloads.

Principal-Level Technical Leadership

• Proven success operating at Principal Architect level or equivalent.

• Recognized as the senior technical authority in customer engagements.

• Trusted to commit organizations to scopes, architectures, and technical approaches.

Presales Experience

• Experience in presales environments, or

• Delivery leadership experience demonstrating the ability to:

  • Lead customer discussions

  • Define scope and solution approaches

  • Develop proposals and Statements of Work

  • Stand behind technical commitments made during the sales process

Vendor Collaboration

• Comfortable working with and being challenged by vendor architects and technical specialists.

Certifications

• Databricks certification preferred.

• Ability to obtain certification quickly if not currently certified.

• Certification support and funding are provided.