Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

align definition of duplicate with its behavior #950

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

align definition of duplicate with its behavior #950

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions soda-cl/numeric-metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,8 +199,8 @@ for each dataset T:
| ------ | ----------- | ------------------- | ---------------------- |
| `avg` | The average value in a numeric column. | number | all |
| `avg_length` | The average length in a text column. | text | all |
| `duplicate_count` | The number of rows that contain duplicate values.<br> Include one column in the argument to compare values relative to that one column. <br/>Include more than one column in the argument to search for duplicate pairs multiple columns. Be sure to add a space between the comma-separated values in the list of column names. <br />See also: [Duplicate check]({% link soda/quick-start-sodacl.md %}#duplicate-check)| number<br /> text<br /> time | all |
| `duplicate_percent` | The percentage of rows in a dataset that contain duplicate values.<br> Include one column in the argument to compare values relative to that one column. Include more than one column in the argument to search for duplicate pairs in multiple columns. Be sure to add a space between the comma-separated values in the list of column names. <br />See also: [Duplicate check]({% link soda/quick-start-sodacl.md %}#duplicate-check)| number<br /> text<br /> time | all |
| `duplicate_count` | The count of distinct values that have duplicates.<br> Include one column in the argument to compare values relative to that one column. <br/>Include more than one column in the argument to search for duplicate pairs multiple columns. Be sure to add a space between the comma-separated values in the list of column names. <br />See also: [Duplicate check]({% link soda/quick-start-sodacl.md %}#duplicate-check)| number<br /> text<br /> time | all |
| `duplicate_percent` | The percentage of rows in a dataset that have dupliates.<br> Include one column in the argument to compare values relative to that one column. Include more than one column in the argument to search for duplicate pairs in multiple columns. Be sure to add a space between the comma-separated values in the list of column names. <br />See also: [Duplicate check]({% link soda/quick-start-sodacl.md %}#duplicate-check)| number<br /> text<br /> time | all |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"dupliates" should be "duplicates"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still I'm not sure if this accurately captures the meaning of the duplicate percent metric...hmm...I think it's literally duplicate_count divided by row_count times 100. Do you think this matches the proposed description?

| `max` | The greatest value in a numeric column. | number | all |
| `max_length` | The greatest length in a text column. | text | all |
| `min` | The smallest value in a numeric column. | number | all |
Expand Down