Job ID: 803147
Experience: 7–11 Years
Location: Bengaluru
Type: Permanent
Job Description
We are looking for a Senior Specialist – Architecture with strong expertise in Python, AWS data engineering services, and data pipeline orchestration. The ideal candidate will have a solid background in data analysis, cloud-native architecture, and Agile methodologies, with a passion for leveraging AI throughout the software development lifecycle (SDLC).
Key Responsibilities
- Design, develop, and maintain data pipelines using Python and modern orchestration tools (e.g., Apache Airflow, AWS MWAA, Mage AI).
- Build and optimize data architectures supporting analytics, reporting, and AI-driven applications.
- Work closely with cross-functional teams to deliver scalable, secure, and efficient data solutions on AWS.
- Manage and monitor data ingestion, transformation, and loading (ETL/ELT) processes.
- Support day-to-day engineering operations, issue resolution, and performance optimization.
- Implement and maintain CI/CD pipelines using GitHub or GitLab.
- Collaborate in an Agile / SAFe environment, contributing to sprint planning, reviews, and retrospectives.
- Ensure solutions align with organizational standards and the Red Cross core values: Collaborative, Committed, Compassionate, Creative, and Credible.
Required Technical Skills
- Python – strong hands-on development and scripting experience.
- Data Analysis – ability to interpret, manipulate, and analyze large datasets.
- Git / CI-CD – experience with GitHub or GitLab pipelines.
- Agile Mindset – proven track record working in Agile teams.
- AI Integration – proactive use of AI tools and techniques across the SDLC.
Desired Technical Skills
- Mage AI or similar modern data pipeline frameworks.
- AWS Services: S3, RDS, Aurora, Redshift, Glue, Lambda, EMR, Athena, QuickSight, Step Functions, DMS, IAM.
- DBT (Data Build Tool) experience for data modeling and transformation.
- Dimensional Data Modeling and data warehouse design.
- Programming Languages: Java or Scala (in addition to Python).
- Operational Support: Engineering operations, incident response, and troubleshooting.
- Scaled Agile Framework (SAFe) experience.
Soft Skills & Mindset
- Curious / Growth Mindset – proactive in learning and adopting new technologies.
- Strong communication and collaboration across technical and non-technical teams.
- Commitment to innovation, reliability, and service excellence.
Mandatory Skills
- AWS MWAA
- Apache Airflow
- Athena
- Aurora
- QuickSight
- RDS
- Redshift
- AWS DMS
- AWS EMR
- AWS Glue
- AWS Lambda
- AWS S3
- AWS Step Functions
- Dimensional Data Modeling
- DynamoDB
- Java
- Python
- Scala