Add model conditions to equality test #1001
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #704
Problem
In #704, it's expressed that it would be useful to limit the rows compared in the
equalitytest. The issue was closed due to inactivity, but the discussion seems to indicate it was a reasonable request, and the only reason the issue was closed was because nobody got around to writing a PR:Solution
This adds the following parameters to
generic_tests/equality.sql:model_conditioncompare_model_conditionIt allows one to filter out rows from either model before comparing the remaining rows in the models as usual.
The rationale and usage is very similar to the
relationships_wheretest. For example, you can exclude recently-modified rows which might not be fully updated in the other table due to e.g. ELT limitations, etc.Note that
model_conditionis introduced, because the normal 'where' clause can't be used because it means the given "model" is no longer a relation, and so the dbt_utils._is_relation check will fail. This was discussed at #704 (comment)Also note that by using two parameters, the user has full control over the filtering, and can choose different filters for each table.
Checklist
NOTE: i tested by writing integration tests, and testing using the postgres target.