Job Description

Summary

The Inter-university Consortium for Political and Social Research (ICPSR), based at the University of Michigan, is the world’s premier political and social science data archive. We are building a next-generation platform for delivering digital social science artifacts to a global research community on a cloud-native platform, and we need a Lead Software Engineer, specialized in Data Engineering, to join our team.

You will work with a small, diverse team committed to delivering high-quality products through modern development practices. If you desire to operate in a startup environment but have your nights and weekends as your own, ICPSR could be your organization. If you love creating secure, scalable, resilient data pipelines within a test-driven development framework, and can manage a small team of data engineers, we want to talk to you!

You will report to the Associate Director of Computing and Network Services.

Responsibilities*

Author production-quality data pipelines running in a distributed, containerized, cloud-native environment to support ingestion, transformation, and dissemination of complex social science research data from a variety of heterogenous sources.
Apply best practices in software development and data management to improve metadata-curation workflows in our Secure Data Environment, building a library of reusable scripts and tools orchestrated by an automation framework.
Develop secure, scalable, and resilient APIs to expose Data-as-a-Service frameworks to upstream producers and downstream consumers.
Support various development teams in all aspects of data management, from the conceptual modeling stage through to the application of data governance policies.
Implement metrics and monitoring solutions for data engineering products and provide operational support to pipelines and APIs in production.
Research and identify open source and enterprise tools to solve challenging problems while minimizing our maintenance footprint.
Collaborate with technical leadership on platform architecture and design.
Manage the Data Engineering staff, provide mentorship, set departmental and individual goals, and assure the application of best practices.
Other tasks as assigned.
Required Qualifications*

Bachelor's degree in computer science or a related field, or equivalent data-engineering experience.
Six or more years of experience in software development, data engineering, data architecture, or a combination.
Experience with using Python for data engineering projects.
Experience designing and managing relational and NoSQL databases.
Experience building production-quality data pipelines using modern ETL, data integration, and data orchestration frameworks.
Experience building interfaces for heterogeneous data sources and sinks, including RESTful API development.
Experience with cloud storage and computing.
Experience developing in distributed Microservice architectures and containerized environments.
Desired Qualifications*

Knowledge of data architecture, modeling, and governance fundamentals.
Familiarity with streaming and/or event-driven architectures.
Familiarity with common practices for quality assurance of pipelines and code.