Job Title: Data Engineer
Job Summary
We are looking for a skilled and motivated Data Engineer with a strong background in Databricks to join our Next Gen MES team based in Limerick, Ireland. This role will focus on designing and optimizing data pipelines, ensuring data integrity, and enabling advanced analytics through the effective use of Databricks and associated technologies. The ideal candidate will have a proven ability to develop robust ETL workflows and collaborate cross-functionally to support business intelligence and data-driven decision-making.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark
- Collaborate with data analysts and data scientists to identify data requirements and deliver tailored solutions
- Enhance performance and efficiency of existing ETL processes
- Implement data validation and testing procedures to maintain high data quality
- Work with Big Data technologies to manage and process large-scale datasets
- Support the design and modeling of data warehouses to meet analytical and reporting needs
- Monitor data pipeline performance and troubleshoot issues proactively
- Participate in code reviews and promote best practices in data engineering
- Document processes, data architecture, and workflows for internal knowledge sharing
- Stay current with industry trends, tools, and technologies in the data engineering space
- Contribute to innovation and continuous improvement within the team
- Communicate and collaborate effectively within a cross-functional team environment
Required Qualifications
- Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field
- 3+ years of hands-on experience in data engineering or a similar role
- Strong proficiency with Databricks and Apache Spark for large-scale data processing
- Proficient in SQL and scripting languages such as Python or Scala
- Familiar with ETL tools, data modeling, and data warehousing concepts
- Experience working with cloud platforms such as Azure (preferred) or AWS
- Understanding of data governance and data quality best practices
- Excellent analytical and problem-solving skills, with strong attention to detail
#LI-MS2
