Primary Skills:

<ul>
<li style="margin-left:1.6rem;">Hadoop (HDFS, Map-Reduce, YARN)</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">SPARK (SQL, DataFrame)</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">ETL/ELT (Professional experience with Teradata, Ab Initio)</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Python or PySpark</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">MongoDB, GCP, Big Query</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">CI/CD, Hive</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Unix, Autosys</li>
</ul>

Responsibilities:

<ul>
<li style="margin-left:1.6rem;">Design, develop, and test robust ETL/ELT data pipelines using Map-Reduce and SPARK.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Process large datasets in multiple file formats such as CSV, JSON, Parquet, and Avro.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Perform metadata configuration and optimize job performance.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Analyze and recommend changes to data models (E-R and Dimensional models) for enhanced efficiency.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Collaborate with cross-functional teams to ensure smooth data workflows and processing.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Implement best practices in coding, performance tuning, and process automation.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Lead the team in troubleshooting complex data issues and provide guidance on best approaches.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Ensure high-quality delivery of data pipelines with regular performance and scalability checks.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Conduct code reviews and mentor junior engineers on technical skills and best practices.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Design and optimize processes for scalable data storage, management, and access.</li>
</ul>

Eligibility Criteria:

<ul>
<li style="margin-left:1.6rem;">10+ years of experience in Big Data Engineering with hands-on expertise in Hadoop and Spark.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Strong understanding and practical experience in designing, coding, and testing ETL/ELT pipelines.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Proficiency in Spark (SQL and DataFrame) for processing large datasets.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Experience with data models (E-R &amp; Dimensional) and their optimization.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Strong skills in Unix Shell scripting (simple to moderate).</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Familiarity with Sparkflow framework (preferred).</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Proficiency in SQL, GCP Big Query, and Python is desirable.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Strong problem-solving and analytical skills.</li>
</ul>

<ul>
<li style="margin-left:1.6rem;">Good communication and collaboration skills to work in a cross-functional team environment.</li>
</ul>

<ul>
<li>Ability to work independently and manage multiple tasks simultaneously. </li>
</ul>

Big Data Developer in Bengaluru (UID-9483)

Job Description

Job Details

Skills Required for Big Data Developer

Job Description for Big Data Developer

Explore More Opportunities