Add scripts to run minhash deduplication on a single node#104

t-sagara wants to merge 1 commit intollm-jp:mainllm-jp/scripts:mainfrom

t-sagara:maint-sagara/llm-jp-scripts:mainCopy head branch name to clipboard