Responsibilities

<ul>
<li>Create and maintain optimal data pipeline architecture; assemble data sets that meet functional / non-functional requirements.</li>
</ul>

<ul>
<li>Design the right schema to support the functional requirement and consumption pattern.</li>
</ul>

<ul>
<li>Design and build production data pipelines from ingestion to consumption.</li>
</ul>

<ul>
<li>Build the necessary datamarts, data warehouse required for optimal extraction, transformation, and loading of data from a wide variety of data sources.</li>
</ul>

<ul>
<li>Create necessary preprocessing and postprocessing for various forms of data for training/ retraining and inference ingestions as required</li>
</ul>

<ul>
<li>Create data visualization and business intelligence tools for stakeholders and data scientists for necessary business/ solution insights</li>
</ul>

<ul>
<li>Identify, design, and implement internal process improvements: automating manual data processes, optimizing data delivery, etc.</li>
</ul>

Requirements and Skills

<ul>
<li>You should have a bachelor’s or master’s degree in computer science, Information Technology or other quantitative fields</li>
</ul>

<ul>
<li>You should have at least 3-4 years working as a data engineer in supporting large data transformation initiatives related to machine learning, with experience in building and optimizing pipelines and data sets</li>
</ul>

<ul>
<li>Strong analytic skills related to working with unstructured datasets.</li>
</ul>
 
Must-have Programming Skills:

<ul>
<li>Programming experience python programming, spark is must.</li>
</ul>

<ul>
<li>Hands -on experience in SQL, writing analytical queries and windows functions.</li>
</ul>

<ul>
<li>Hands - on experience in creating external tables, partitioning, parquet files.</li>
</ul>

<ul>
<li>2-3 years of solid experience in Big Data technologies a must.</li>
</ul>

<ul>
<li>Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift)</li>
</ul>

<ul>
<li>Knowledge of Python and Pyspark is an absolute must. </li>
</ul>

Requirements/Skill sets:

<ul>
<li>Experience with AWS cloud services: EC2, EMR, RDS, Redshift, S3, Athena and familiarity with various log formats from AWS.</li>
</ul>

<ul>
<li>Experience in AWS Glue ETL, AWS Crawler, AWS Lambda, Glue Data Catalog, AWS Glue Studio.</li>
</ul>

<ul>
<li>Hands on experience with python programming, spark, shell scripting</li>
</ul>

<ul>
<li>Knowledge of Database Concepts - Indexing, Partitioning is must.</li>
</ul>

<ul>
<li>Knowledge of Data warehousing - Normalization, Denormalization, Star/Snow-flake schemas.</li>
</ul>

<ul>
<li>Good Hands -on the table’s creations, DDL, DML and TCL</li>
</ul>

<ul>
<li>Knowledge of Database Concepts - Indexing, Partitioning</li>
</ul>

<ul>
<li>Knowledge of Data warehousing - Normalization, Denormalization, Star/Snow-flake schemas</li>
</ul>

Data Engineer in Delhi (UID-10509)

Job Description

Job Details

Skills Required for Data Engineer

Job Description for Data Engineer

Explore More Opportunities