Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow processing files in parallel via new jobs option #186

Merged
merged 2 commits into from
Jan 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .pyspelling.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
spellchecker: aspell
jobs: 8

matrix:
- name: mkdocs
Expand Down
1 change: 1 addition & 0 deletions docs/src/dictionary/en-custom.txt
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ chainable
charset
checkable
chunking
config
could've
dicts
distro
Expand Down
5 changes: 5 additions & 0 deletions docs/src/markdown/about/changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## 2.10

- **NEW**: Allow specifying parallel processes to speed up spell checking. Number of jobs can be specified either
by command line or via the config. Command line overrides the config.

## 2.9

- **NEW**: Officially support Python 3.11 and 3.12.
Expand Down
11 changes: 11 additions & 0 deletions docs/src/markdown/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,17 @@ This can be overridden on the command line.
spellchecker: hunspell
```

You can specify the number of parallel jobs to use by setting the global option `jobs`. This create parallel jobs to
process files in a given task.

```yaml
jobs: 4
```

/// new | New 2.10
Parallel processing is new in 2.10.
///

All of the spelling tasks are contained under the keyword `matrix` and are organized in a list:

```yaml
Expand Down
23 changes: 19 additions & 4 deletions docs/src/markdown/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,20 +53,24 @@ If you want to manually install it, run `#!bash python setup.py build` and `#!ba
## Command Line Usage

```
usage: pyspelling [-h] [--version] [--verbose] [--name NAME] [--binary BINARY]
[--config CONFIG] [--spellchecker SPELLCHECKER]
usage: pyspelling [-h] [--version] [--verbose] [--name NAME | --group GROUP] [--binary BINARY] [--jobs JOBS] [--config CONFIG] [--source SOURCE] [--spellchecker SPELLCHECKER]

Spell checking tool.

optional arguments:
options:
-h, --help show this help message and exit
--version show program's version number and exit
--verbose, -v Verbosity level.
--name NAME, -n NAME Specific spelling task by name to run.
--group GROUP, -g GROUP
Specific spelling task group to run.
--binary BINARY, -b BINARY
Provide path to spell checker's binary.
--jobs JOBS, -j JOBS Specify the number of spell checker processes to run in parallel.
--config CONFIG, -c CONFIG
Spelling config.
--source SOURCE, -S SOURCE
Specify override file pattern. Only applicable when specifying exactly one --name.
--spellchecker SPELLCHECKER, -s SPELLCHECKER
Choose between aspell and hunspell
```
Expand All @@ -82,7 +86,7 @@ If you have multiple Python versions, you can run the PySpelling associated with
Python major and minor version:

```shell-session
$ pyspelling3.7
$ pyspelling3.11
```

To specify a specific configuration other than the default, or even point to a different location:
Expand Down Expand Up @@ -132,6 +136,17 @@ You can specify the spell checker type by specifying it on the command line. PyS
$ pyspelling -s hunspell
```

To run multiple jobs in parallel, you can use the `--job` or `-j` option. Processing files in parallel can speed up
processing time. Specifying jobs on the command line will override the `jobs` setting in the configuration file.

```console
$ pyspelling -n my_task -j 4
```

/// new | New 2.10
Parallel processing is new in 2.10.
///

## Supported Spell Check Versions

PySpelling is tested with Hunspell 1.6+, and recommends using only 1.6 and above. Some lower versions might work, but
Expand Down
29 changes: 29 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,35 @@ markdown_extensions:
- example
- quote
- pymdownx.blocks.details:
types:
- name: details-new
class: new
- name: details-settings
class: settings
- name: details-note
class: note
- name: details-abstract
class: abstract
- name: details-info
class: info
- name: details-tip
class: tip
- name: details-success
class: success
- name: details-question
class: question
- name: details-warning
class: warning
- name: details-failure
class: failure
- name: details-danger
class: danger
- name: details-bug
class: bug
- name: details-example
class: example
- name: details-quote
class: quote
- pymdownx.blocks.html:
- pymdownx.blocks.definition:
- pymdownx.blocks.tab:
Expand Down
Loading