Senior Data Engineer ( AWS )

Key Responsibilities:

1. Data Architecture and Design

• Design and implement scalable, secure, and efficient data pipelines and architectures on

AWS

• Define and manage the architecture for real-time and batch data processing.

2. Data Pipeline Development

• Build ETL/ELT processes to ingest, transform, and store structured and unstructured

data from diverse sources.

• Develop solutions for data integration, data quality, and data governance.

3. Optimization and Monitoring

• Optimize data processing workflows for performance and cost-effectiveness.

• Monitor data pipelines and systems, troubleshooting and resolving issues promptly.

4. Collaboration

• Partner with data scientists, analysts, and business teams to understand data needs and

deliver high-quality solutions.

• Collaborate with DevOps teams to implement CI/CD for data workflows.

5. Security and Compliance

• Ensure data solutions comply with organizational policies and industry security, privacy,

and governance standards.

• Implement data anonymization and encryption techniques as needed.

6. Innovation

• Stay updated with emerging AWS technologies and data engineering best practices.

• Drive continuous improvement in processes, tools, and methodologies.

Required Qualifications

1. Education

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.

2. Experience

• 5+ years of experience as a Data Engineer or in a similar role, focusing on AWS cloud solutions.

• Proven experience designing and building data architectures and pipelines.

3. Technical Skills

• Expertise in AWS services, including but not limited to S3, EMR, Lambda, and RDS.

• Proficiency in SQL, Python, and Java for data processing and manipulation.

• Experience with data tool stacks such as: Spark, Flink, Airflow.

• Strong knowledge of data modeling, data warehousing, and data lakes.

• Experience with data lake formats such as Delta or Iceberg.

• Familiarity with CI/CD tools and infrastructure as code (e.g., Terraform).

• Experience with data governance, security, and compliance standards.

• Containerization experience with Docker and Kubernetes (K8s).

• Real-time data integration using Debezium and Kafka.

• Experience with Linux and Bash commands.

• Experience with query engines like Trino and Presto.

4. Soft Skills

• Strong problem-solving and analytical skills.

• Excellent communication and collaboration abilities.

• Ability to work in an agile and fast-paced environment.

• Ability to communicate efficiently in English.

Nice to have

• Knowledge of machine learning workflows and integration with data pipelines.

• Knowledge of monitoring tools like CloudWatch, Datadog, or Prometheus.

• Knowledge of modern data lakehouse platforms like Apache Doris and Pinot.

• Knowledge and experience in Banking / Finance/Securities/ Fintech domains