README.md

Problems 1 -> Employee With his Latest Salary

We have a table with employees and their salaries, however, some of the records are old and contain outdated salary information. Find the current salary of each employee assuming that salaries increase each year. Output their id, first name, last name, department ID, and current salary. Order your list by employee ID in ascending order.

Problem Difficulty Level : Medium

Data Structure

Data for this problem

In CSV Format

In JSON Format

Solving using PySpark

In Spark we will solve this problem using two ways

1. Using PySpark Functions
2. Using Spark SQL

Use below notebook for solution

Problem Solution

Solving using PostgreSQL

In Postgre SQL We will load data from CSV using PostgreSQL Import functionality. And then we will solve this problem.

Output Query

Problem Solution

Please also follow below blog for understanding this problem