As a Senior Data Engineer, you will be at the forefront of designing and optimizing advanced data processing pipelines. Collaborating closely with cross-functional teams, you'll transform complex business requirements into efficient, scalable data solutions. Your expertise in SQL, Python, Databricks, and Spark will ensure smooth and effective processing of large-scale data, including imagery and other big data sets. This position offers the chance to work with cutting-edge technologies while making a meaningful impact on our data infrastructure.
location: Akron, Ohio
job type: Permanent
salary: $140,000 - 155,000 per year
work hours: 8am to 5pm
education: Bachelors
responsibilities:
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines powered by Apache Spark, enabling efficient processing of large-scale imagery and data sets.
- Work alongside data scientists and analysts to define data requirements and develop solutions that align with business goals.
- Optimize Spark jobs for enhanced performance and scalability, ensuring fast data processing times and optimal resource utilization.
- Troubleshoot and resolve performance bottlenecks in Spark applications, implementing strategies to improve speed and reliability.
- Leverage the Databricks platform to manage and orchestrate Spark jobs, ensuring smooth operation and high availability.
- Develop custom Python or Scala functions to extend Spark's capabilities and meet specific project needs.
- Keep abreast of the latest trends and advancements in big data and Spark technologies, integrating best practices into our processes.
- Ensure seamless integration of Spark-based solutions with existing infrastructure and systems.
qualifications:
- Skills and Qualifications:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field.
- Extensive experience as a Data Engineer, specializing in building and optimizing Spark-based data pipelines.
- Expertise in SQL and Python, including the ability to craft complex queries and scripts for data analysis and manipulation.
- Deep knowledge of Apache Spark, including its ecosystem (Spark SQL, Spark Streaming, Spark MLlib).
- Proven experience with Databricks, managing Spark clusters, and orchestrating workflows.
- Familiarity with cloud platforms like AWS, Azure, or Google Cloud, with experience in big data services (e.g., AWS EMR, Azure Databricks).
- Experience with large-scale imagery or geospatial data is highly desirable.
- Strong problem-solving abilities, with a focus on optimizing workflows for performance and efficiency.
- Excellent communication and collaboration skills, with a demonstrated ability to work well in a team environment.
skills:
- Data Warehouse
- Building and optimizing data pipelines
Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.
At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact HRsupport@randstadusa.com.
Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including health, an incentive and recognition program, and 401K contribution (all benefits are based on eligibility).
This posting is open for thirty (30) days.
