Role: Senior Data Engineer
Location: Bay Area, CA (1 day onsite must)
Duration: 12+ months
Pay Rate; $60 to $65
Responsibilities:
- Develop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation.
- Collaborate with product and technology teams to design and validate the capabilities of the data platform
- Identify, design, and implement process improvements: automating manual processes, optimizing for usability, re-designing for greater scalability
- Provide technical support and usage guidance to the users of our platform’s services.
- Drive the creation and refinement of metrics, monitoring, and alerting mechanisms to give us the visibility we need into our production services.
Qualifications:
- 15+ years’ experience
- Experience in data structures and algorithms
- Strong in Python, Palantir Foundry and PySpark
- Experience building and optimizing data pipelines in a distributed environment
- Experience supporting and working with cross-functional teams
- Proficiency working in Linux environment
- Working knowledge of SQL, Git/Bitbucket, Jenkins/CodeBuild, CodePipeline
