To showcase the proficiency of my data science skills and to simulate my performance in daily duties in a job, I have decided to do simulated project to demonstrate my ability to use the tools (Python, SQL, etc.).
The first project within this portfolio will get a dataset that is not consistently formatted/messy and for me to get the dataset and make it more standardized.
I have decided to use the Brownfield Areas dataset from the Florida Department of Enviromental Protection Geospatial Open Data website [https://geodata.dep.state.fl.us/datasets/FDEP::brownfield-areas/about]. I chose this dataset because it has over 500 records, there was not a consistent format within the METHOD, AREA_NAME, and USERNAME, and it could give the opportunity to apply the ACID principles (Atomicity, Consistency, Isolation and Durability) to construct the schema for the database.
In the "cleaning-and-formatting-brownfield" folder, I began by creating a folder ("data") where I put my downloaded csv file (Brownfield_Areas.csv). Then, I created a python file "examining-data.py" to load in the data and perform basic summary statistics, examining data types of each column/variable, and examine any duplicates or non-standardized records with the columns. To utilize the crucial python libraries within data science (pandas, matplotlib, numpy, etc.), I created a virtual environment and loaded the specific versions of the libraries to the requirements.txt file. I noticed that within the METHODS Column, PAPER does not have a consistent casing throughtout all of the records. With the SOURCE column, I have also seen some of the same issues as well. I also need to convert all of the dates and times in the dataset into the DateTime format in Python.