Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring VcrGzipSerializer to normalize new cassettes #112

Merged
merged 4 commits into from
Jun 21, 2019

Conversation

victor-torres
Copy link
Contributor

With these changes, it would be easier to add new cassettes when adding new tests and making calls to HubStorage and Dash APIs.

For example, you can use a dummy Scrapy Cloud project and its deploy API Key to add a new test cassette as following:

HS_PROJECT_ID=<project id> HS_AUTH=<api key> AUTH=<api key> USER_AUTH=<api key> HS_ENDPOINT= DASH_ENDPOINT= py.test tests/client/test_collections.py::test_new_case

This will use the production servers with the provided credentials and will create a new cassette file at tests/client/cassettes/test_collections/test_new_case.gz.

From now on you can execute the test with just:

py.test tests/client/test_collections.py::test_new_case

@codecov
Copy link

codecov bot commented Feb 12, 2019

Codecov Report

Merging #112 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #112   +/-   ##
=======================================
  Coverage   93.61%   93.61%           
=======================================
  Files          28       28           
  Lines        1895     1895           
=======================================
  Hits         1774     1774           
  Misses        121      121

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc1f83b...a057f1e. Read the comment docs.

Copy link
Contributor

@vshlapakov vshlapakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@victor-torres That's a very neat idea. Sorry for taking so long to review it, I was continuously failing to allocate enough time to check it carefully and test locally as required. Could you take a quick look into the comments pls?

@victor-torres
Copy link
Contributor Author

Thank you for your comments here, @vshlapakov. I'm just waiting for some time before jumping into this PR again.

now you should be able to add new cassettes easier than before

I should update the readme file with more infor about it
@victor-torres
Copy link
Contributor Author

victor-torres commented May 17, 2019

@vshlapakov I've just updated the pull request.

Easy way to test the feature:

Rename some test case, for example, tests/client/test_collections.py::test_truncate to tests/client/test_collections.py::test_truncate_new.

Execute the test command specifying the environment variables (you can use any dummy Scrapy Cloud project):

HS_PROJECT_ID=<project id> HS_AUTH=<api key> AUTH=<api key> USER_AUTH=<api key> HS_ENDPOINT= DASH_ENDPOINT= py.test tests/client/test_collections.py::test_truncate_new

Check the new cassette file tests/client/cassetes/test_collections/test_truncate_new.gz with:

git status

@vshlapakov vshlapakov merged commit 8319db9 into scrapinghub:master Jun 21, 2019
@vshlapakov
Copy link
Contributor

That's just awesome, thank you so much 👏

@victor-torres victor-torres deleted the VCRGzipSerializer branch June 21, 2019 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants