BeachHead Inc.
Sr. Data Engineer - Databricks
Job Description
Are you a highly skilled Data Engineer with expertise in Databricks ready to lead a major data platform modernization? Apply Now! Working with one of our top financial clients, this role calls for a Sr. Data Engineer - Databricksspecializing in Databricks and Data Platform Migration. This position offers a compelling opportunity to work on a large-scale initiative to transition trading and market data from legacy systems into modern, scalable data frameworks, supporting strategic modernization efforts within Capital Markets. Responsibilities
Design and develop data ingestion pipelines, both batch and real-time, leveraging Databricks platform. Build and optimize Spark/Scala-based data processing workflows for high performance and reliability. Implement streaming ingestion solutions using Kafka, APIs, or similar technologies. Develop medallion architecture models (bronze, silver, gold layers) consistent with best practices. Create and maintain materialized views and other data s...
Design and develop data ingestion pipelines, both batch and real-time, leveraging Databricks platform. Build and optimize Spark/Scala-based data processing workflows for high performance and reliability. Implement streaming ingestion solutions using Kafka, APIs, or similar technologies. Develop medallion architecture models (bronze, silver, gold layers) consistent with best practices. Create and maintain materialized views and other data s...