-
Notifications
You must be signed in to change notification settings - Fork 166
Suggestions for project #6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
8d87d82
1d9f542
4ece89a
71c9edb
bce27ba
f1fd606
8ab9035
de9274c
6c35365
b1fb712
c780755
2a8ba6d
de92c1a
5a7fd4e
6359785
6e86025
6de0f16
2fe0b18
d4fe40c
5285fd4
0af294d
31e8d4a
906d883
3304457
d7c1be3
d1019b3
ad80d93
1384707
1329e82
ad0479c
e032fbb
2d42ffe
e096b00
2a6d5aa
0aac45e
339fe18
d3417a1
8e88a81
7db1943
f6188f7
5cd8133
4e6f58e
a3d4795
c8de445
78e00c4
9e7f016
9837bbc
50e7ca9
2df8280
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
[bumpversion] | ||
current_version = 0.4.3 | ||
commit = True | ||
tag = True | ||
|
||
[bumpversion:file:setup.py] | ||
|
||
[bumpversion:file:scripts/deploy.ps1] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
[MASTER] | ||
extension-pkg-whitelist=numpy,torch | ||
|
||
[TYPECHECK] | ||
|
||
# List of members which are set dynamically and missed by pylint inference | ||
# system, and so shouldn't trigger E1101 when accessed. Python regular | ||
# expressions are accepted. | ||
generated-members=numpy.*,torch.* |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
language: python | ||
cache: | ||
pip: true | ||
directories: | ||
# - datasets | ||
before_install: | ||
- python --version | ||
- pip install -U pip | ||
- pip install codecov | ||
install: | ||
- pip install -e .[test] # install package + test dependencies | ||
script: | ||
- pytest --cov=time_series_transformer # run tests | ||
after_success: | ||
- codecov # submit coverage | ||
Comment on lines
+1
to
+15
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would rather avoid using Travis here, this is not so much a development project that it requires CI/CD. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is a good practice to whenever there is a code change, to run a checklist of all the desired functionality covering all corners of the code base. If you look at codecov with the preliminary tests I came up with 65% of the code was already covered. Now that number can reach 100% with the addition of a few more tests, so that you can see it is always working as intended when there is a code change. And if the build breaks for any reason you are immediatelly informed by email. Then you can go there and fix it fast. Sometimes a dependency breaks the code base, as was the case with numpy 1.19.4 in Windows. The only solution as of now was to stick with a previous numpy version. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes I understand, but we currently don't have any kind of test, and dependency problems are left to the user by not mentioning the precise version of each package in the requirements. If at some point we develop some meaningful tests, we could consider using Travis, but wright now it's just a waste of CPU time. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,67 +1,34 @@ | ||
# Transformers for Time Series | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why are we removing such big chunks of the Readme ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The purpose was to simplify the package documentation so that only essential info is there for SW developers. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes but this content is important, it show how and why I developed that repo. The README is not the documentation, I think we can get away with having a detailed file here. |
||
[](https://badge.fury.io/py/time-series-transformer) [](https://travis-ci.org/github/DanielAtKrypton/time_series_transformer) [](https://codecov.io/gh/DanielAtKrypton/time_series_transformer) [](https://github.com/DanielAtKrypton/time_series_transformer) [](https://requires.io/github/DanielAtKrypton/time_series_transformer/requirements/?branch=master) | ||
|
||
[](https://timeseriestransformer.readthedocs.io/en/latest/?badge=latest) [](https://www.gnu.org/licenses/gpl-3.0) [](https://github.com/maxjcohen/transformer/releases/latest) | ||
|
||
Implementation of Transformer model (originally from [Attention is All You Need](https://arxiv.org/abs/1706.03762)) applied to Time Series (Powered by [PyTorch](https://pytorch.org/)). | ||
|
||
## Transformer model | ||
|
||
Transformer are attention based neural networks designed to solve NLP tasks. Their key features are: | ||
|
||
- linear complexity in the dimension of the feature vector ; | ||
- paralellisation of computing of a sequence, as opposed to sequential computing ; | ||
- long term memory, as we can look at any input time sequence step directly. | ||
|
||
This repo will focus on their application to times series. | ||
|
||
## Dataset and application as metamodel | ||
|
||
Our use-case is modeling a numerical simulator for building consumption prediction. To this end, we created a dataset by sampling random inputs (building characteristics and usage, weather, ...) and got simulated outputs. We then convert these variables in time series format, and feed it to the transformer. | ||
|
||
## Adaptations for time series | ||
|
||
In order to perform well on time series, a few adjustments had to be made: | ||
|
||
- The embedding layer is replaced by a generic linear layer ; | ||
- Original positional encoding are removed. A "regular" version, better matching the input sequence day/night patterns, can be used instead ; | ||
- A window is applied on the attention map to limit backward attention, and focus on short term patterns. | ||
Based on [timeseriestransformer](https://readthedocs.org/projects/timeseriestransformer/badge/?version=latest). | ||
|
||
## Installation | ||
|
||
All required packages can be found in `requirements.txt`, and expect to be run with `python3.7`. Note that you may have to install pytorch manually if you are not using pip with a Debian distribution : head on to [PyTorch installation page](https://pytorch.org/get-started/locally/). Here are a few lines to get started with pip and virtualenv: | ||
|
||
```bash | ||
$ apt-get install python3.7 | ||
$ pip3 install --upgrade --user pip virtualenv | ||
$ virtualenv -p python3.7 .env | ||
$ . .env/bin/activate | ||
(.env) $ pip install -r requirements.txt | ||
```terminal | ||
.\scripts\init-env.ps1 | ||
``` | ||
|
||
## Usage | ||
|
||
### Downloading the dataset | ||
|
||
The dataset is not included in this repo, and must be downloaded manually. It is comprised of two files, `dataset.npz` contains all input and outputs value, `labels.json` is a detailed list of the variables. | ||
|
||
### Running training script | ||
|
||
Using jupyter, run the default `training.ipynb` notebook. All adjustable parameters can be found in the second cell. Careful with the `BATCH_SIZE`, as we are using it to parallelize head and time chunk calculations. | ||
|
||
### Outside usage | ||
|
||
The `Transformer` class can be used out of the box, see the [docs](https://timeseriestransformer.readthedocs.io/en/latest/?badge=latest) for more info. | ||
|
||
```python | ||
from tst import Transformer | ||
from flights_time_series_dataset import FlightsDataset | ||
from time_series_predictor import TimeSeriesPredictor | ||
from time_series_transformer import Transformer | ||
|
||
tsp = TimeSeriesPredictor( | ||
Transformer(), | ||
max_epochs=50, | ||
train_split=None, | ||
) | ||
|
||
net = Transformer(d_input, d_model, d_output, q, v, h, N, TIME_CHUNK, pe) | ||
tsp.fit(FlightsDataset()) | ||
``` | ||
|
||
### Building the docs | ||
### Test | ||
|
||
To build the doc: | ||
To test the package simply run the following command from project's root folder. | ||
|
||
```bash | ||
(.env) $ cd docs && make html | ||
pytest -s | ||
``` |
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
Uh oh!
There was an error while loading. Please reload this page.