Data Engineer-Data Platforms-AWS
IBM
Data Engineer-Data Platforms-AWS
Job Description
Data Engineer-Data Platforms-AWS
📋 Job Overview
As a Data Engineer at IBM, you will develop, maintain, evaluate, and test big data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platforms. You will be responsible for building data pipelines to ingest, process, and transform data, and develop efficient software code for various use cases. This role is based in one of IBM's Consulting Client Innovation Centers, delivering technical and industry expertise to a wide range of clients globally.
📍 Location: Pune, IN (Remote/Hybrid)
💼 Career Level: Entry Level
🎯 Key Responsibilities
- Develop, maintain, evaluate and test big data solutions
- Build data pipelines to ingest, process, and transform data from files, streams and databases
- Process data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS
- Develop efficient software code for multiple use cases leveraging Spark Framework using Python or Scala and Big Data technologies
- Develop streaming pipelines
- Work with Hadoop / AWS ecosystem components to implement scalable solutions to meet the ever-increasing data volumes
✅ Required Qualifications
- Total 5 - 7+ years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills
- Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala
- Minimum 3 years of experience on Cloud Data Platforms on AWS
- Exposure to streaming solutions and message brokers like Kafka technologies
- Experience in AWS EMR / AWS Glue / DataBricks, AWS RedShift, DynamoDB
- Good to excellent SQL skills
⭐ Preferred Qualifications
- Certification in AWS and Data Bricks or Cloudera Spark Certified developers
🛠️ Required Skills
- Spark Framework
- Python
- Scala
- Hadoop
- AWS Cloud Data Platform
- PySpark
- Hive
- Hbase
- NoSQL databases
- Big Data technologies
- Streaming pipelines
- Apache Spark
- Kafka
- Cloud computing
- AWS EMR
- AWS Glue
- DataBricks
- AWS RedShift
- DynamoDB
- SQL
🎁 Benefits & Perks
- Opportunity to learn and develop yourself and your career
- Encouragement to be courageous and experiment everyday
- Continuous trust and support in an environment where everyone can thrive
- Growth-minded culture with openness to feedback and learning
- Opportunity to collaborate with colleagues and drive exceptional outcomes for customers
Locations
- Pune, IN, India (Remote)
Salary
Estimated Salary Rangemedium confidence
800,000 - 1,500,000 INR / yearly
Source: ai estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- Spark Frameworkintermediate
- Pythonintermediate
- Scalaintermediate
- Hadoopintermediate
- AWS Cloud Data Platformintermediate
- PySparkintermediate
- Hiveintermediate
- Hbaseintermediate
- NoSQL databasesintermediate
- Big Data technologiesintermediate
- Streaming pipelinesintermediate
- Apache Sparkintermediate
- Kafkaintermediate
- Cloud computingintermediate
- AWS EMRintermediate
- AWS Glueintermediate
- DataBricksintermediate
- AWS RedShiftintermediate
- DynamoDBintermediate
- SQLintermediate
Required Qualifications
- Total 5 - 7+ years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills (experience)
- Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala (experience)
- Minimum 3 years of experience on Cloud Data Platforms on AWS (experience)
- Exposure to streaming solutions and message brokers like Kafka technologies (experience)
- Experience in AWS EMR / AWS Glue / DataBricks, AWS RedShift, DynamoDB (experience)
- Good to excellent SQL skills (experience)
Preferred Qualifications
- Certification in AWS and Data Bricks or Cloudera Spark Certified developers (experience)
Responsibilities
- Develop, maintain, evaluate and test big data solutions
- Build data pipelines to ingest, process, and transform data from files, streams and databases
- Process data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS
- Develop efficient software code for multiple use cases leveraging Spark Framework using Python or Scala and Big Data technologies
- Develop streaming pipelines
- Work with Hadoop / AWS ecosystem components to implement scalable solutions to meet the ever-increasing data volumes
Benefits
- general: Opportunity to learn and develop yourself and your career
- general: Encouragement to be courageous and experiment everyday
- general: Continuous trust and support in an environment where everyone can thrive
- general: Growth-minded culture with openness to feedback and learning
- general: Opportunity to collaborate with colleagues and drive exceptional outcomes for customers
Tags & Categories
Answer 10 quick questions to check your fit for Data Engineer-Data Platforms-AWS @ IBM.
10 Questions
~2 Minutes
Instant Score
Related Books and Jobs
Data Engineer-Data Platforms-AWS
IBM
Data Engineer-Data Platforms-AWS
Job Description
Data Engineer-Data Platforms-AWS
📋 Job Overview
As a Data Engineer at IBM, you will develop, maintain, evaluate, and test big data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platforms. You will be responsible for building data pipelines to ingest, process, and transform data, and develop efficient software code for various use cases. This role is based in one of IBM's Consulting Client Innovation Centers, delivering technical and industry expertise to a wide range of clients globally.
📍 Location: Pune, IN (Remote/Hybrid)
💼 Career Level: Entry Level
🎯 Key Responsibilities
- Develop, maintain, evaluate and test big data solutions
- Build data pipelines to ingest, process, and transform data from files, streams and databases
- Process data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS
- Develop efficient software code for multiple use cases leveraging Spark Framework using Python or Scala and Big Data technologies
- Develop streaming pipelines
- Work with Hadoop / AWS ecosystem components to implement scalable solutions to meet the ever-increasing data volumes
✅ Required Qualifications
- Total 5 - 7+ years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills
- Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala
- Minimum 3 years of experience on Cloud Data Platforms on AWS
- Exposure to streaming solutions and message brokers like Kafka technologies
- Experience in AWS EMR / AWS Glue / DataBricks, AWS RedShift, DynamoDB
- Good to excellent SQL skills
⭐ Preferred Qualifications
- Certification in AWS and Data Bricks or Cloudera Spark Certified developers
🛠️ Required Skills
- Spark Framework
- Python
- Scala
- Hadoop
- AWS Cloud Data Platform
- PySpark
- Hive
- Hbase
- NoSQL databases
- Big Data technologies
- Streaming pipelines
- Apache Spark
- Kafka
- Cloud computing
- AWS EMR
- AWS Glue
- DataBricks
- AWS RedShift
- DynamoDB
- SQL
🎁 Benefits & Perks
- Opportunity to learn and develop yourself and your career
- Encouragement to be courageous and experiment everyday
- Continuous trust and support in an environment where everyone can thrive
- Growth-minded culture with openness to feedback and learning
- Opportunity to collaborate with colleagues and drive exceptional outcomes for customers
Locations
- Pune, IN, India (Remote)
Salary
Estimated Salary Rangemedium confidence
800,000 - 1,500,000 INR / yearly
Source: ai estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- Spark Frameworkintermediate
- Pythonintermediate
- Scalaintermediate
- Hadoopintermediate
- AWS Cloud Data Platformintermediate
- PySparkintermediate
- Hiveintermediate
- Hbaseintermediate
- NoSQL databasesintermediate
- Big Data technologiesintermediate
- Streaming pipelinesintermediate
- Apache Sparkintermediate
- Kafkaintermediate
- Cloud computingintermediate
- AWS EMRintermediate
- AWS Glueintermediate
- DataBricksintermediate
- AWS RedShiftintermediate
- DynamoDBintermediate
- SQLintermediate
Required Qualifications
- Total 5 - 7+ years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills (experience)
- Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala (experience)
- Minimum 3 years of experience on Cloud Data Platforms on AWS (experience)
- Exposure to streaming solutions and message brokers like Kafka technologies (experience)
- Experience in AWS EMR / AWS Glue / DataBricks, AWS RedShift, DynamoDB (experience)
- Good to excellent SQL skills (experience)
Preferred Qualifications
- Certification in AWS and Data Bricks or Cloudera Spark Certified developers (experience)
Responsibilities
- Develop, maintain, evaluate and test big data solutions
- Build data pipelines to ingest, process, and transform data from files, streams and databases
- Process data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS
- Develop efficient software code for multiple use cases leveraging Spark Framework using Python or Scala and Big Data technologies
- Develop streaming pipelines
- Work with Hadoop / AWS ecosystem components to implement scalable solutions to meet the ever-increasing data volumes
Benefits
- general: Opportunity to learn and develop yourself and your career
- general: Encouragement to be courageous and experiment everyday
- general: Continuous trust and support in an environment where everyone can thrive
- general: Growth-minded culture with openness to feedback and learning
- general: Opportunity to collaborate with colleagues and drive exceptional outcomes for customers
Tags & Categories
Answer 10 quick questions to check your fit for Data Engineer-Data Platforms-AWS @ IBM.
10 Questions
~2 Minutes
Instant Score
Related Books and Jobs