USTPostClassAssignment.txt

27/09/2023
Create SQL User to query mask data & validate masking

SQL:
write query for customer full name, phone, email, compony.
Write query to show how many customer are in same compony .
Write query to show how many customer are in same country ->[SalesLT].[Address],[SalesLT].[Customer]


28/09/2023
ADF: 
Design a pipeline to unzip file from Blob raw to datalake refined.

29/09/2023
Explore Hadoop & big data
https://data-flair.training/blogs/hadoop-tutorial/
assement of 1st week on cloud needs to complete.

05/10/2023
1. Firstname + middle name + Last Name  = Full Name
2. Gender column F= Female & M = Male
3. Below column should be part of tempview
CustomerID: integer (nullable = true)
FulltName: string (nullable = true)
CompanyName: string (nullable = true)
SalesPerson: string (nullable = true)
EmailAddres
Phone
Gneder

06/10/2023

# TODO
# Replace <<FILL_IN>> with your code. 

pdf = (df.read                    # Our SparkSession & Entry Point
  <<FILL_IN>>                  # Reduce the columns to just the one
  <<FILL_IN>>                  # Produce a unique set of values
)
totalArticles = df.<<FILL_IN>> # Identify the total number of records remaining.

print("Distinct Articles: ".format(totalArticles))