Data Engineer (PySpark, Hadoop and AWS)
A leading bank with commercial, international, and private banking services is undergoing a major digital transformation and shift towards using public cloud. We are entrusted with building the leadership and tech team to bring in experienced professionals who can help in the transformation.
Responsibilities:
- Work with new technology, focus on using the right tool for the job, rather than any sticky preference for a tool or technology
- Learn and share knowledge across our engineering teams, so we can continue to iterate and improve
- Write reusable, testable, and efficient code
- Creates and manages schema and instances for SQL and non-SQL databases
- Design and implement low-latency, high-availability, and performant applications
- Write data pipelines that access multiple systems to retrieve, validate, process, and store data
- Integrate user-facing elements developed by front-end developers with server-side logic
- Work on implementation of security and data protection projects
- Collaborate and work on the integration of data storage solutions
- Focus on performance tuning, improvement, balancing, usability, and automation
What does a typical day look like:
- Data Analysis (20%)
- Development (40%)
- Testing (20%)
- Deployment activities (20%)