Training materials for the course "Machine learning"

Russia, Ulyanovsk Ulyanovsk State Technical University

Week 1

1. Basics of Machine Learning

• Brief history of basic technologies
• Definitions
• Basics of ML

2. Python

• Basic syntax
• Arithmetic
• Strings
• Lists
• Exploratory data analysis with Pandas
• Data visualization
• Kaggle competitions
• Titanic task

3. Linear regression

• Linear regression
• Cost function
• Gradient descent
• Linear regression with multiple variables
• Debug (learning rate)
• Normal equation
• Features and polynomial regression
• Multi-class classification
• Feature scaling

4. Decision trees. KNN. Logistic regression. Regularization.

• Decision tree
• K-nearest neighbors
• Classification problem
• Sigmoid function
• Decision boundary
• Cost function
• Regularization. Problem of overfitting

5. Machine learning system design

• Error analysis. Metrics
• Evaluating hypothesis. Train / test / validation set
• High bias / high variance (model selection, regularization, learning curves)
• Feature extraction

6. Advances models

• Support vector machines
• Naive bayes
• Clustering (k-means, c-means, hierarchical clustering)
• Principal component analysis

Week 2

7. Neural networks

• Non-linear hypothesis
• Neurons and the brain
• Forward propagation (XNOR example)
• Back propagation
• Parameters initializing

8. Deep learning. Convolutional neural networks

• Convolution. Feature representation as hierarchy
• Filters, stride, padding
• Pooling
• Popular architectures: AlexNet, VGG, ResNet,
• Classification, localization, regression

9. Working with different data types

• Images
• GEO
• Date and time
• Timeseries
• Texts. One-hot encoding
• Texts. Word2vec
• Sounds

10. Recurrent neural networks

• Basics of recurrent NN
• LSTM
• Time-series analysis
• Text analysis

11. Trees

• Decision trees
• Random forest
• XGBoost
• CatBoost

12. Generative Adversarial networks

13. Reinforcement learning

Course materials

• jupyter environment: http://summer-school.hibers.tech
• assignment submitting: http://moodle.hibers.tech
• form for the registration to assignment submitting: https://docs.google.com/forms/d/e/1FAIpQLSe9oFVq3A5rpJpppMVVxIQZDH417RdrfoAAUITwtxamqJfKEg/viewform

Links

• ML course by Andrew Ng
• Lectures notes of the Coursera Course by Andrew Ng: https://github.com/diefimov/MTH594_MachineLearning
• ML course by Dmitry Efimov
• ML course by OpenData Science
• MIT Deep learning
• Recommended book: Python for Data Analysis ru from Pandas author.
• Recommended book:François Chollet. Deep Learning with Python. Manning, 2018 and (https://github.com/fchollet/deep-learning-with-python-notebooks)
• 100+ Free Data Science Books
• Gallery of interesting IPython notebooks
• NumPy - arrays & matrices
• Pandas - working with datasets
• Matplotlib - data visualization
• SkiKit learn - ML library
• Pandas lessons
• Pandas tutorial
• Google collaboratory - powerful Jupyter Notebook environment for deep learning
• Awesome deepvision links
• SSD: Single-Shot MultiBox Detector implementation in Keras
• Word2vec with Keras
• https://github.com/qati/DeepLearningCourse
• https://github.com/roebius/deeplearning_keras2
• https://github.com/enggen/Deep-Learning-Coursera
• https://github.com/fchollet/deep-learning-with-python-notebooks
• Russian: Отличная статья про то, как создать ИНС, с примерами на Python
• Russian: Анализ данных при помощи Python. Графики в pandas и matplotlib.
• Russian: Основы pandas

Visual Attention Model

• (https://towardsdatascience.com/visual-attention-model-in-deep-learning-708813c2912c)
• (https://medium.com/@sunnerli/visual-attention-in-deep-learning-77653f611855)
• (https://arxiv.org/pdf/1512.01693.pdf)
• (https://arxiv.org/pdf/1406.6247.pdf)
• (https://arxiv.org/pdf/1605.01335.pdf)
• (https://openreview.net/pdf?id=B14TlG-RW)