Lead Data Engineer

Straive · Newark, Delaware

FULL TIME

Job Description

Straive

is a global leader in enterprise-grade Data Analytics and AI solutions, committed to empowering businesses across various industries with cutting-edge technology and expert insights. Backed by EQT, a top private equity firm, we are uniquely positioned to drive innovation through significant investments and an entrepreneurial spirit.

Our core focus is on delivering advanced Data Analytics & AI Solutions. By combining sophisticated technology with subject matter expertise, we deliver material impact on our clients' topline and streamline their operations. We specialize in providing tailored solutions across financial services, CPG, legal, pharma, life sciences, Energy, retail and logistics, helping them build robust data analytics and AI capabilities.

With a client base spanning 30 countries, Straive's strategically located teams operate from eight countries and is headquartered in Singapore. This global presence enables us to offer localized expertise with a worldwide perspective.

Join Straive to be part of a dynamic team at the forefront of data analytics and AI innovation. Here, you'll have the opportunity to contribute to transformative projects, supported by significant investments and an entrepreneurial drive fueled by our partnership with EQT.

Website: https://www.straive.com/ LinkedIn

Job Tittle-

Lead Data Engineer

Location -

Newark,NJ (Hybrid)

Employment Type-

Fulltime

Job Overview

We are seeking a

Lead Data Engineer

to help us architect and implement modern data solutions using the

Microsoft Fabric

ecosystem. You will be responsible for streamlining data workflows, implementing OneLake architectures, and enabling seamless data integration and analytics across the enterprise.

Key

Responsibilities

Unified Storage:

Implement and optimize

OneLake

structures to create a seamless, single source of truth for diverse data assets.

Data Engineering:

Develop scalable ETL/ELT workflows using

Synapse Data Engineering

and

Data Factory .

Performance Tuning:

Optimize Fabric capacity and compute usage to ensure cost-efficient, high-speed data processing.

Pipelines & Ingestion:

Build metadata-driven pipelines using Fabric Pipelines to ingest and process

JSON, ZIP, XLSX, and CSV

files effectively. Implement pattern-based incremental refreshes and

SCD Type 1 & Type 2 logic .

Architecture & Performance:

Design scalable Medallion Architectures (Bronze/Silver/Gold) across Fabric Lakehouses and Data Warehouses. Apply

PySpark optimization

(broadcast joins, partition strategy, data skew, memory tuning),

SQL query tuning , and Delta Lake features (V-Order, Liquid Clustering, Time Travel).

Automation & Scripting:

Write advanced Python and leverage Fabric Utils (notebookutils/mssparkutils) alongside

Fabric REST APIs

to automate inter-notebook workflows, dynamic parameterization, and workspace management.

Governance & Data Quality:

Build automated Data Quality practices ( schema enforcement, dynamic validation, reconciliation logging ). Implement Fabric Governance frameworks, Microsoft Purview integration, data lineage, and security controls

(RLS/CLS) . Managing staged environments Dev. UAT and PROD with CICD Principals.

DevOps, Capacity & Analytics:

Manage Fabric DevOps/ALM (Git integration and deployment pipelines) and monitor compute using the Fabric Capacity Metrics App to optimize CUs. Support basic

Power BI semantic modeling (Direct Lake mode)

and real-time ingestion via Eventhouse/KQL.

Collaboration:

Partner with Data Scientists and BI teams to deliver structured, high-fidelity datasets for GenAI and advanced analytics.

Required Skills and

Qualifications

Total Experience

of 10-15 years

Ecosystem Mastery:

3-5 years in

Azure Data Engineering

with focused experience in

Microsoft Fabric , Synapse Analytics, and OneLake.

Languages:

Advanced proficiency in

Python (PySpark)

and

Complex SQL .

Frameworks:

Hands-on experience with

Delta Lake

formats, Medallion Architecture, and Spark-based processing.

Integrations : Working knowledge of Power BI (Direct Lake connectivity), Git-based CI/CD, and Fabric Capacity Management.

Orchestration:

Proven ability to build and monitor pipelines in

Data Factory

and manage CI/CD via

Azure DevOps .

Data Modeling:

Good understanding of Star/Snowflake schemas and semantic modeling.

Certifications (Preferred):

DP-600

(Fabric Analytics Engi

Company	Straive
Location	Newark, Delaware
Type	FULL TIME
Niche	Full-time

Explore 1M+ Live Opportunities.