Data Engineer II
at J.P. Morgan
Posted a month ago
No clicks
- Compensation
- Not specified
- City
- Pune
- Country
- India
Currency: Not specified
JPMorgan Chase is hiring a Data Engineer II for the Connected Commerce Travel Technology team to design, build, and maintain large-scale cloud-based data integration and analytical solutions. The role focuses on developing scalable data pipelines and models, optimizing storage and processing for high-volume datasets, and ensuring data integrity and quality. You will collaborate with cross-functional teams in an Agile environment and apply TDD/CI-CD practices to deliver secure, stable, and scalable solutions. Preferred technologies include Python/PySpark, Spark, cloud lakehouse services (AWS/Databricks), Airflow, Terraform, and Apache Iceberg.
Location: Pune, Maharashtra, India
Job responsibilities
- Design, develop and maintain scalable and large-scale data processing pipelines and infrastructure on the cloud following engineering standards, governance standards and technology best practices
- Develop and optimize data models for large-scale datasets, ensuring efficient storage, retrieval, and analytics while maintaining data integrity and quality.
- Collaborate with cross-functional teams to translate business requirements into scalable and effective data engineering solutions.
- Demonstrate a passion for innovation and continuous improvement in data engineering, proactively identifying opportunities to enhance data infrastructure, data processing and analytics capabilities.
Required qualifications, capabilities, and skills
- Strong analytical problem solving and critical thinking skills
- Proficiency in at least one programming language ( Python, if not Java or Scala)
- Proficiency in at least one distributed data processing framework ( Spark or similar)
- Proficiency in at least one cloud data Lakehouse platforms (AWS Data lake services or Databricks, alternatively Hadoop),
- Proficiency in at least one scheduling/orchestration tools ( Airflow, if not AWS Step Functions or similar)
- Proficiency with relational and NoSQL databases.
- Proficiency in data structures, data serialization formats (JSON, AVRO, Protobuf, or similar), and big-data storage formats (Parquet, Iceberg, or similar),
- Experience working in teams following Agile methodology
- Experience with test-driven development (TDD) or behavior-driven development (BDD) practices, as well as working with continuous integration and continuous deployment (CI/CD) tools.
- Proficiency in Python and Pyspark
- Proficiency in IaC (preferably Terraform, alternatively AWS cloud formation)
- Experience with AWS Glue, AWS S3, AWS Lakehouse, AWS Athena, Airflow, Kinesis and Apache Iceberg
- Experience working with Jenkins
