Data Platform Engineer
Egen is a 2020 (3 years running) Great Place to Work company that provides cloud-native solutions to help our customers unlock the power of data to rapidly inform, transform, and outperform their industry. We move at digital light-speed to build and consult on some of the most cutting edge data and software solutions with modern tech stacks for large established enterprises, scaling mid-sized enterprises, and emerging startups.
Our Data Platform Engineering teams build scalable data pipelines using Python and AWS, GCP, or Azure. The pipelines we build typically integrate with technologies such as Kafka, Storm, and Elasticsearch. We are working on a continuous deployment pipeline that leverages rapid on-demand releases. Our developers work in an agile process to efficiently deliver high value applications and product packages.
As a Data Platform Engineer, you will architect and implement cloud-native data pipelines and infrastructure to enable analytics and machine learning on rich datasets.
Required Experience:
- Minimum of Bachelor’s Degree or its equivalent in Computer Science, Computer Information Systems, Information Technology and Management, Electrical Engineering or a related field.
- You know what it takes to build and run resilient data pipelines in production and have experience implementing ETL/ELT to load a multi-terabyte enterprise data warehouse.
- You have implemented analytics applications using multiple database technologies, such as relational, multidimensional (OLAP), key-value, document, or graph.
- You value the importance of defining data contracts, and have experience writing specifications including REST APIs.
- You write code to transform data between data models and formats, preferably in Python (Spark or PySpark is a bonus).
- You've worked in agile environments and are comfortable iterating quickly.
Nice to have's (but not required):
- Experience moving trained machine learning models into production data pipelines.
- Expert knowledge of relational database modeling concepts, SQL skills, proficiency in query performance tuning, and desire to share knowledge with others.
- Experience building cloud-native applications and supporting technologies / patterns / practices including: AWS/GCP/Azure, Docker, CI/CD, DevOps, and microservices.