Skip to content

Commit

Permalink
⚡ refactor: classify learning resources
Browse files Browse the repository at this point in the history
  • Loading branch information
ShAlireza committed Apr 9, 2024
1 parent f7c38b1 commit c1b5f62
Showing 1 changed file with 8 additions and 18 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -20,27 +20,17 @@ Python is widely used in academia for teaching programming concepts, in web deve
Python's development is guided by the Python Enhancement Proposal (PEP) process, a key mechanism for proposing major new features, collecting community input on issues, and documenting Python design decisions.
Python 3, the latest major version of Python, introduced many changes to the language, making it more powerful and consistent.

<> This title is optional
## Python in Data Engineering

Python is a cornerstone in the field of data engineering, offering a broad spectrum of libraries and frameworks that cater to every step of the data processing pipeline.
For data collection and ingestion, tools like Requests, Beautiful Soup, and Scrapy simplify web scraping and API interactions.
Transformation and analysis are powered by Pandas and NumPy, providing robust structures and functions for manipulating large datasets.
When it comes to storage, SQLAlchemy and PyMongo facilitate efficient database operations.
Workflow management is streamlined with platforms such as Apache Airflow and Luigi, enabling sophisticated scheduling and monitoring of data pipelines.

Visualization and reporting are made accessible through Jupyter Notebooks and libraries like Matplotlib, Seaborn, and Dash, which support the creation of dynamic data presentations.
For handling big data, PySpark and Dask offer scalable solutions for processing vast volumes of information.
Ensuring data quality and integrity is achievable with tools like Great Expectations and Pandera, which provide frameworks for data validation and testing.
Across all these stages, Python stands out for its ease of use, flexibility, and the active community that continuously enhances its data engineering capabilities.

Don't worry about the tools just mentioned above, since you will get to know them whenever there is a need for them! We mentioned them here just to realize the capabilities of Python in Data Engineering!


## Learning Resources

Here are some suggestions for learning python, you can choose any of them to start your Python programming journey:

- [https://wiki.python.org/moin/BeginnersGuide/Programmers](https://wiki.python.org/moin/BeginnersGuide/Programmers)
### Books
- [Clean Code in Python](https://www.google.de/books/edition/Clean_Code_in_Python/ZB9sDwAAQBAJ?hl=en&gbpv=0)
- [Expert Python Programming](https://www.packtpub.com/en-ca/product/expert-python-programming-fourth-edition-9781801071109)

### Courses
- [Python Tutorial - Python Full Course for Beginners](https://www.youtube.com/watch?v=_uQrJ0TkZlc)
- [Python Full Course for free](https://www.youtube.com/watch?v=XKHEtdqhLK8)

### Miscellaneous
- [https://wiki.python.org/moin/BeginnersGuide/Programmers](https://wiki.python.org/moin/BeginnersGuide/Programmers)

0 comments on commit c1b5f62

Please sign in to comment.