Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add markers/tags for migration tests #138

Merged
merged 5 commits into from
Oct 28, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,34 @@ class TestDirectMigration(MigratorTestCase):
assert SomeItem.objects.filter(is_clean=True).count() == 1
```

### Choosing only migrations tests

In CI systems it is important to get instant feedback. Running tests that
apply database migration can slow down tests execution, so it is often a good
idea to run standard, fast, regular unit tests without migrations in parallel
with slower migrations tests.

#### pytest

`django_test_migrations` adds `migration_test` marker to each test using
`migrator_factory` or `migrator` fixture.
To run only migrations test, use `-m` option:

```bash
pytest -m migration_test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also give an example here how to ignore migration tests:

pytest -m migration_test  # runs only migraion tests
pytest -m "not migration_test"  # runs all except migration tests

```

#### unittest

`django_test_migrations` adds `migration_test`
[tag](https://docs.djangoproject.com/en/3.0/topics/testing/tools/#tagging-tests)
to every `MigratorTestCase` subclass.
To run only migrations tests, use `--tag` option:

```bash
python mange.py test --tag=migration_test
Copy link
Member

@sobolevn sobolevn Oct 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have something similar to -m "not ..." in unittest?

Copy link
Collaborator Author

@skarzi skarzi Oct 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is a CLI option --exclude-tag, I will add usage example to the docs

```


## Django Checks

Expand Down
4 changes: 4 additions & 0 deletions django_test_migrations/constants.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from typing_extensions import Final

#: marker/tag indicating that marked test is Django's migrations test
MIGRATION_TEST_MARKER: Final = 'migration_test'
24 changes: 24 additions & 0 deletions django_test_migrations/contrib/pytest_plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,30 @@
from django.db import DEFAULT_DB_ALIAS
from django.db.models.signals import post_migrate, pre_migrate

from django_test_migrations.constants import MIGRATION_TEST_MARKER


def pytest_load_initial_conftests(early_config):
"""Register pytest's markers."""
early_config.addinivalue_line(
'markers',
"{0}: mark the test as a Django's migration test.".format(
MIGRATION_TEST_MARKER,
),
)


def pytest_collection_modifyitems(session, items): # noqa: WPS110
"""Mark all tests using ``migrator_factory`` fixture with proper marks.

Add ``MIGRATION_TEST_MARKER`` marker to all items using
``migrator_factory`` fixture.

"""
for pytest_item in items:
if 'migrator_factory' in getattr(pytest_item, 'fixturenames', []):
pytest_item.add_marker(MIGRATION_TEST_MARKER)


@pytest.fixture()
def migrator_factory(request, transactional_db, django_db_use_migrations):
Expand Down
4 changes: 3 additions & 1 deletion django_test_migrations/contrib/unittest_case.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@

from django.db.migrations.state import ProjectState
from django.db.models.signals import post_migrate, pre_migrate
from django.test import TransactionTestCase
from django.test import TransactionTestCase, tag

from django_test_migrations.constants import MIGRATION_TEST_MARKER
from django_test_migrations.migrator import Migrator
from django_test_migrations.types import MigrationSpec


@tag(MIGRATION_TEST_MARKER)
class MigratorTestCase(TransactionTestCase):
"""Used when using raw ``unitest`` library for test."""

Expand Down
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ branch = True
# And we completely test it anyway.
omit =
django_test_migrations/contrib/pytest_plugin.py
django_test_migrations/constants.py

[coverage:report]
skip_covered = True
Expand Down
44 changes: 44 additions & 0 deletions tests/test_contrib/test_pytest_plugin/test_plugin.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
import re
import subprocess

from django_test_migrations.constants import MIGRATION_TEST_MARKER


def test_call_pytest_setup_plan():
"""Checks that module is registered and visible in the meta data."""
Expand All @@ -19,3 +22,44 @@ def test_call_pytest_setup_plan():

assert 'migrator' in output_text
assert 'migrator_factory' in output_text


def test_pytest_registers_marker():
"""Ensure ``MIGRATION_TEST_MARKER`` marker is registered."""
output_text = subprocess.check_output(
['pytest', '--markers'],
stderr=subprocess.STDOUT,
universal_newlines=True,
encoding='utf8',
)

assert MIGRATION_TEST_MARKER in output_text


def test_pytest_markers():
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sobolevn what do you think about using pytester plugin to add more detailed tests of our pytest plugin?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sure! Let's open a new issue for it! 👍

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added issue #140

"""Ensure ``MIGRATION_TEST_MARKER`` markers are properly added."""
output_text = subprocess.check_output(
[
'pytest',
'--collect-only',

# Collect only tests marked with ``MIGRATION_TEST_MARKER`` marker
'-m',
MIGRATION_TEST_MARKER,

# We need this part because otherwise check fails with `1` code:
'--cov-fail-under',
'0',
],
stderr=subprocess.STDOUT,
universal_newlines=True,
encoding='utf8',
)

search_result = re.search(
r'(?P<selected_number>\d+)\s+selected',
output_text,
)
assert search_result
assert int(search_result.group('selected_number') or 0) > 0
assert 'test_pytest_plugin' in output_text
7 changes: 7 additions & 0 deletions tests/test_contrib/test_unittest_case/test_unittest_case.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from django_test_migrations.constants import MIGRATION_TEST_MARKER
from django_test_migrations.contrib.unittest_case import MigratorTestCase


Expand Down Expand Up @@ -38,3 +39,9 @@ def test_migration_main0001(self):
SomeItem = self.new_state.apps.get_model('main_app', 'SomeItem')

assert SomeItem.objects.count() == 2


def test_migration_test_marker_tag():
"""Ensure ``MigratorTestCase`` sublasses are properly tagged."""
assert MIGRATION_TEST_MARKER in TestDirectMigration.tags
assert MIGRATION_TEST_MARKER in TestBackwardMigration.tags