Veriipro
Python Data Engineer with PySpark Expertise
Job Description
Shape scalable data solutions as a Python Data Engineer with expertise in PySpark. Embrace opportunities in designing data pipelines, optimizing workflows, and managing cloud-based systems.
We are seeking a Data Engineer who excels in Python and PySpark. Your role involves building ETL/ELT pipelines and ensuring data consistency while processing large datasets. With responsibilities in data modeling, performance optimization, and cloud services, you will contribute significantly to data-driven decisions.
Key Responsibilities:
• Design and develop ETL/ELT pipelines using Python and PySpark
• Process structured and unstructured data efficiently
• Utilize Apache Spark for distributed data processing
• Optimize Spark jobs for batch and real-time data
• Implement data validations for quality assurance
Requirements:
• Strong programming skills in Python
• Expertise in PySpark and Apache Spark
• Advanced SQ...
We are seeking a Data Engineer who excels in Python and PySpark. Your role involves building ETL/ELT pipelines and ensuring data consistency while processing large datasets. With responsibilities in data modeling, performance optimization, and cloud services, you will contribute significantly to data-driven decisions.
Key Responsibilities:
• Design and develop ETL/ELT pipelines using Python and PySpark
• Process structured and unstructured data efficiently
• Utilize Apache Spark for distributed data processing
• Optimize Spark jobs for batch and real-time data
• Implement data validations for quality assurance
Requirements:
• Strong programming skills in Python
• Expertise in PySpark and Apache Spark
• Advanced SQ...