Job Description

Senior Data Engineer Job Description Job Title: Senior Data Engineer

Experience: 12+ Years

Job Type: W2 Contract

Job Summary

We are seeking a highly experienced Senior Data Engineer with 12+ years of experience in designing, developing, and maintaining scalable data platforms and data pipelines. The ideal candidate will have strong hands-on expertise in Python, SQL, Spark, Databricks, cloud technologies, ETL/ELT, data warehousing, and data integration .

Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines for enterprise applications.
Develop complex data transformation and processing workflows using Python, SQL, and Apache Spark .
Build and optimize data pipelines using Databricks and PySpark .
Design and implement data solutions on AWS, Azure, or Google Cloud Platform cloud platforms.
Develop data ingestion processes from databases, APIs, files, and other structured/unstructured sources.
Implement data quality, validation, reconciliation, and monitoring processes.
Design and optimize data warehouses, data lakes, and lakehouse architectures .
Work with relational and NoSQL databases to support data engineering requirements.
Perform performance tuning and optimization of SQL queries, Spark jobs, and data pipelines.
Implement orchestration using Apache Airflow, Azure Data Factory, AWS Glue, or similar tools .
Develop reusable frameworks and components for data ingestion and transformation.
Integrate data engineering solutions with CI/CD pipelines and DevOps processes.
Implement data security, access controls, encryption, and governance best practices.
Troubleshoot production data pipelines and resolve data processing issues.
Collaborate with data architects, analysts, data scientists, application developers, and business stakeholders.
Participate in Agile/Scrum ceremonies and contribute to technical design and architecture discussions.
Mentor junior and mid-level data engineers and provide technical guidance.

Required Skills

12+ years of experience in Data Engineering, ETL, or related data technologies.
Strong hands-on experience with Python and SQL .
Extensive experience with Apache Spark / PySpark .
Strong experience with Databricks and Delta Lake.
Experience building enterprise-scale ETL/ELT pipelines .
Strong knowledge of data warehousing concepts, dimensional modeling, and data lake architectures.
Experience with one or more cloud platforms: AWS, Azure, or Google Cloud Platform .
Experience with databases such as SQL Server, Oracle, PostgreSQL, MySQL, Snowflake, or similar .
Experience with workflow orchestration tools such as Airflow, Azure Data Factory, AWS Glue, or similar .
Strong understanding of batch and near-real-time data processing.
Experience with Git, CI/CD, Jenkins, Azure DevOps, or GitHub Actions .
Strong understanding of data quality, data validation, and performance optimization.
Excellent analytical, troubleshooting, and communication skills.

Preferred Skills

Experience with Azure Data Lake, AWS S3, ADLS, or Google Cloud Storage .
Experience with Snowflake or other modern cloud data warehouses.
Knowledge of Kafka, Event Hubs, or other streaming technologies .
Experience with Terraform or Infrastructure as Code .
Knowledge of Docker and Kubernetes.
Experience implementing data governance, metadata management, and lineage .
Experience working with large-scale distributed data environments.
Knowledge of Medallion Architecture (Bronze, Silver, Gold) .
Experience with real-time/streaming data pipelines.
Experience in designing highly available and fault-tolerant data platforms.

For applications and inquiries, contact:hirings@openkyber.com

Job Tags

Contract work

Similar Jobs

Preferred Care at Home of Boca Raton and Delray Beach

Live in caregiver Job at Preferred Care at Home of Boca Raton and Delray Beach

Perferred Care at home is seeking an experienced HHA or CNA for a day live-in position, 7 days a week. The caregiver must be comfortable assisting a client with an ostomy bag and providing medication reminders. A Skype interview is required before starting, and interviews...

Spark by Hilton Fishers

Hotel Houseman Job at Spark by Hilton Fishers

...hands while staying with us. As a Hotel Houseman, your contribution... ...linens, and trash placed near Housekeeper carts. Clean, maintain, and... ...stand and walk for an entire shift and perform repetitive motions... ...Available -Morning/Evening/Night -Weekends -Full-Time/Part...

Nexiva Inc

VTEX Developer - Remote Job at Nexiva Inc

...provide end-to-end technical leadership for the VTEX global rollout, ensuring the solution is scalable, secure, and aligned with Whirlpool's global platform architecture while supporting country-specific business requirements. Own the end-to-end solution architecture...

SWBC PEO

Workers Comp Claims Management Specialist Job at SWBC PEO

...from injury reporting through claim closure. This role works closely with customers, injured employees, insurance carriers, claim adjusters, HR, and payroll to facilitate efficient claim handling, support return-to-work efforts, and provide a high level of customer service...

The Aspenwood Company - The Village on Morehead

Housekeeper Job at The Aspenwood Company - The Village on Morehead

...Village on Morehead, an Independent Living senior community. You will be responsible for cleaning occupied/empty apartments and community areas, laundry as assigned, etc. Housekeeping experience preferred but not necessary. We will train the right candidates. Must desire...

Lead Data Engineer Job at Openkyber, Atlanta, GA