Pinterest data source #478
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stijn-uva
added
enhancement
commit 513a589 Author: Dale Wahl <[email protected]> Date: Mon Jan 27 17:01:52 2025 +0100 bsky: ensure interrupt commit 0badee7 Author: Dale Wahl <[email protected]> Date: Mon Jan 27 15:19:42 2025 +0100 bsky: no progress bar if no max_posts commit 115a3c1 Author: Dale Wahl <[email protected]> Date: Mon Jan 27 14:18:12 2025 +0100 bsky datasource commit 836a235 Author: Dale Wahl <[email protected]> Date: Thu Jan 23 11:47:51 2025 +0100 post_topic_matrix: rename column when tokenizer created multiple documents per post commit 977d887 Author: Dale Wahl <[email protected]> Date: Thu Jan 23 11:37:52 2025 +0100 rank_attribute: convert to str to lower() commit a1cdd4c Author: Dale Wahl <[email protected]> Date: Wed Jan 22 09:40:29 2025 +0100 fix: allow None for columns.default; remove debug log statement fix occasional error that appears particularly on new processors with no expected default, i.e.: <option value="{{ choice }}"{% if choice in option_settings.default %} selected="selected"{% endif %}>{{ option_settings.options[choice] }}</option> TypeError: argument of type 'NoneType' is not iterable
commit dd2ab72 Author: Stijn Peeters <[email protected]> Date: Tue Feb 4 18:16:59 2025 +0100 Highlight missing fields in CSV preview commit 204ab8a Author: Stijn Peeters <[email protected]> Date: Tue Feb 4 18:16:47 2025 +0100 Add a 'missing fields' key to mapped dataset items commit 11457e0 Author: Stijn Peeters <[email protected]> Date: Tue Feb 4 18:16:26 2025 +0100 Add a 'missing_fields' column to mapped objects commit a3e4f77 Author: Stijn Peeters <[email protected]> Date: Tue Feb 4 18:15:06 2025 +0100 Prevent tooltips from falling (partially) outside the viewport commit 16be136 Author: Dale Wahl <[email protected]> Date: Tue Feb 4 17:18:27 2025 +0100 docker build action possible fix commit b330339 Author: Stijn Peeters <[email protected]> Date: Tue Feb 4 16:48:38 2025 +0100 Parse Markdown in dataset status commit 79cb297 Author: Stijn Peeters <[email protected]> Date: Tue Feb 4 15:43:27 2025 +0100 Indicate whether like amount is hidden for Instagram posts commit 3c62f37 Author: Stijn Peeters <[email protected]> Date: Tue Feb 4 15:43:00 2025 +0100 Do not consider missing geotags in Instagram posts 'missing' fields commit 6d3f9d4 Author: Dale Wahl <[email protected]> Date: Tue Feb 4 13:45:47 2025 +0100 consolidate_urls: better logging/status, better url split commit a84c63b Author: Dale Wahl <[email protected]> Date: Tue Feb 4 13:45:05 2025 +0100 revert 0983a36 commit bf7fe14 Author: Dale Wahl <[email protected]> Date: Tue Feb 4 12:36:26 2025 +0100 consolidate_urls: hide unused settings based on requirements commit ccaf114 Author: Dale Wahl <[email protected]> Date: Tue Feb 4 12:27:57 2025 +0100 consolidate_urls: validate URL before parsing commit 638413a Author: Dale Wahl <[email protected]> Date: Tue Feb 4 11:08:17 2025 +0100 check results exist then delete; error message include dataset key when unable to delete sometimes log files are left behind because FileNotFoundError was raised on the results_path commit 0983a36 Author: Dale Wahl <[email protected]> Date: Tue Feb 4 10:28:51 2025 +0100 possibly address github action build fail issue commit 855d34e Author: Stijn Peeters <[email protected]> Date: Mon Feb 3 18:44:14 2025 +0100 Fix Gephi Lite link commit 4e5752d Author: Stijn Peeters <[email protected]> Date: Mon Feb 3 17:57:16 2025 +0100 Nicer numbers in network processor statuses commit 1e0a24c Author: Stijn Peeters <[email protected]> Date: Mon Feb 3 17:43:20 2025 +0100 Never assume fields are non-null in Telegram data... commit 66d60e9 Author: Stijn Peeters <[email protected]> Date: Mon Feb 3 17:40:09 2025 +0100 Fix forward username mapping in some cases for Telegram commit 8034d1c Merge: 59a1546 9bccdf1 Author: Stijn Peeters <[email protected]> Date: Mon Feb 3 12:05:48 2025 +0100 Merge branch 'master' of https://github.com/digitalmethodsinitiative/4cat commit 59a1546 Author: Stijn Peeters <[email protected]> Date: Mon Feb 3 12:05:44 2025 +0100 Add progress indicators to 'Count values' and 'Thread metadata' processors commit 9bccdf1 Author: Dale Wahl <[email protected]> Date: Fri Jan 31 13:05:29 2025 +0100 Update docker_latest.yml 6.13.0? commit e826283 Author: Dale Wahl <[email protected]> Date: Fri Jan 31 13:01:18 2025 +0100 Same but different commit 54d10cb Author: Dale Wahl <[email protected]> Date: Fri Jan 31 12:59:41 2025 +0100 Update GitHub action to use latest docker commit 2600e55 Author: Dale Wahl <[email protected]> Date: Fri Jan 31 12:34:59 2025 +0100 python 3.11 for Docker Have been using this all winter and have had no issues. Enjoying the better error messages too. commit 513a589 Author: Dale Wahl <[email protected]> Date: Mon Jan 27 17:01:52 2025 +0100 bsky: ensure interrupt commit 0badee7 Author: Dale Wahl <[email protected]> Date: Mon Jan 27 15:19:42 2025 +0100 bsky: no progress bar if no max_posts commit 115a3c1 Author: Dale Wahl <[email protected]> Date: Mon Jan 27 14:18:12 2025 +0100 bsky datasource commit 836a235 Author: Dale Wahl <[email protected]> Date: Thu Jan 23 11:47:51 2025 +0100 post_topic_matrix: rename column when tokenizer created multiple documents per post commit 977d887 Author: Dale Wahl <[email protected]> Date: Thu Jan 23 11:37:52 2025 +0100 rank_attribute: convert to str to lower() commit a1cdd4c Author: Dale Wahl <[email protected]> Date: Wed Jan 22 09:40:29 2025 +0100 fix: allow None for columns.default; remove debug log statement fix occasional error that appears particularly on new processors with no expected default, i.e.: <option value="{{ choice }}"{% if choice in option_settings.default %} selected="selected"{% endif %}>{{ option_settings.options[choice] }}</option> TypeError: argument of type 'NoneType' is not iterable
# Conflicts: # webtool/static/js/fourcat.js
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A data source, for Pinterest (via Zeeschuimer)
Annoying: the composition of Pinterest post objects varies relatively heavily between pages, so sometimes you get (for example) timestamps and sometimes you don't. The mapping tries to account for this, but might need to do some checks if processors can handle e.g. posts with an empty timestamp.