Roles and Responsibilities:

<ul>
<li style="margin-left:1.6rem;">Create and maintain optimal data pipeline architecture</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Create and maintain events/streaming based architecture/design</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Work with data scientists to strive for greater functionality in our data systems. </li>
</ul>
Mandatory Qualifications:

<ul>
<li style="margin-left:1.6rem;">We are looking for a candidate with 2+ years of experience in a Data Engineer role.</li>
</ul>
 
<ul>
<li style="margin-left:1.6rem;">Experience with big data tools: HDFS/S3, Spark/Flink,Hive,Hbase, Kafka/Kinesis, etc.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience with relational SQL and NoSQL databases, including Elasticsearch and Cassandra/Mongodb.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience with AWS /GCP cloud services</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience with stream-processing systems: Spark-Streaming/Flink etc.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience with object-oriented/object function scripting languages: Java, Scala, etc.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Strong analytic skills related to working with structured/unstructured datasets.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Build processes supporting data transformation, data structures, dimensional modeling, metadata, dependency, schema registration/evolution and workload management.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience supporting and working with cross-functional teams in a dynamic environment</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Have a few weekend side projects up on GitHub</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Have contributed to an open-source project</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Have worked at a product company</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Have a working knowledge of a backend programming language </li>
</ul>

Data Engineer in Bengaluru (UID-6972)

Job Description

Job Details

Skills Required for Data Engineer

Job Description for Data Engineer

Explore More Opportunities