Notebook to HTML script #543

sepandhaghighi · 2024-07-28T08:42:04Z

Reference Issues/PRs

#497

What does this implement/fix? Explain your changes.

doc_to_html.bat --> notebook_to_html.py
RELEASE.md updated
Document build system updated

Any other comments?

codecov · 2024-07-28T08:42:16Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.68%. Comparing base (c1df796) to head (3445b8e).

Additional details and impacted files

@@           Coverage Diff           @@
##              dev     #543   +/-   ##
=======================================
  Coverage   99.68%   99.68%           
=======================================
  Files          14       14           
  Lines        3097     3097           
  Branches      467      467           
=======================================
  Hits         3087     3087           
  Misses          4        4           
  Partials        6        6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sadrasabouri

That looks great but I had dependency issues running it.
I am getting this error:

Traceback (most recent call last):
  File ".../pycm/Otherfiles/notebook_to_html.py", line 11, in <module>
    from jupyter_contrib_nbextensions.nbconvert_support import TocExporter
  File ".../pycm/.venv/lib/python3.10/site-packages/jupyter_contrib_nbextensions/nbconvert_support/__init__.py", line 5, in <module>
    from .collapsible_headings import ExporterCollapsibleHeadings
  File ".../pycm/.venv/lib/python3.10/site-packages/jupyter_contrib_nbextensions/nbconvert_support/collapsible_headings.py", line 6, in <module>
    from notebook.services.config import ConfigManager
ModuleNotFoundError: No module named 'notebook.services'

I installed the earliest version of libraries but it didn't work. You can see my pip freeze here:

anyio==4.4.0
appnope==0.1.4
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
art==6.2
asttokens==2.4.1
async-lru==2.0.4
attrs==23.2.0
Babel==2.15.0
bandit==1.7.9
beautifulsoup4==4.12.3
bleach==6.1.0
certifi==2024.7.4
cffi==1.16.0
charset-normalizer==3.3.2
comm==0.2.2
contourpy==1.2.1
coverage==7.6.0
cycler==0.12.1
debugpy==1.8.2
decorator==5.1.1
defusedxml==0.7.1
exceptiongroup==1.2.2
executing==2.0.1
fastjsonschema==2.20.0
fonttools==4.53.1
fqdn==1.5.1
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
iniconfig==2.0.0
ipykernel==6.29.5
ipython==8.26.0
ipython-genutils==0.2.0
ipywidgets==8.1.3
isoduration==20.11.0
jedi==0.19.1
Jinja2==3.1.4
json5==0.9.25
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2023.12.1
jupyter==1.0.0
jupyter-console==6.6.3
jupyter-events==0.10.0
jupyter-highlight-selected-word==0.2.0
jupyter-lsp==2.2.5
jupyter_client==8.6.2
jupyter_contrib_core==0.4.2
jupyter_contrib_nbextensions==0.7.0
jupyter_core==5.7.2
jupyter_nbextensions_configurator==0.6.4
jupyter_server==2.14.2
jupyter_server_terminals==0.5.3
jupyterlab==4.2.4
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.3
jupyterlab_widgets==3.0.11
kiwisolver==1.4.5
lxml==5.2.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.9.1
matplotlib-inline==0.1.7
mdurl==0.1.2
mistune==3.0.2
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
nest-asyncio==1.6.0
notebook==7.2.1
notebook_shim==0.2.4
numpy==2.0.0
overrides==7.7.0
packaging==24.1
pandas==2.2.2
pandocfilters==1.5.1
parso==0.8.4
pbr==6.0.0
pexpect==4.9.0
pillow==10.4.0
platformdirs==4.2.2
pluggy==1.5.0
prometheus_client==0.20.0
prompt_toolkit==3.0.47
psutil==6.0.0
ptyprocess==0.7.0
pure_eval==0.2.3
pycm @ file:///Users/sadra/OpenSource/pycm
pycparser==2.22
pydocstyle==6.3.0
Pygments==2.18.0
pyparsing==3.1.2
pytest==8.3.2
pytest-cov==5.0.0
python-dateutil==2.9.0.post0
python-json-logger==2.0.7
pytz==2024.1
PyYAML==6.0.1
pyzmq==26.0.3
qtconsole==5.5.2
QtPy==2.4.1
referencing==0.35.1
requests==2.32.3
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.7.1
rpds-py==0.19.1
seaborn==0.13.2
Send2Trash==1.8.3
six==1.16.0
sniffio==1.3.1
snowballstemmer==2.2.0
soupsieve==2.5
stack-data==0.6.3
stevedore==5.2.0
terminado==0.18.1
tinycss2==1.3.0
tomli==2.0.1
tornado==6.4.1
traitlets==5.9.0
types-python-dateutil==2.9.0.20240316
typing_extensions==4.12.2
tzdata==2024.1
uri-template==1.3.0
urllib3==2.2.2
vulture==2.11
wcwidth==0.2.13
webcolors==24.6.0
webencodings==0.5.1
websocket-client==1.8.0
widgetsnbextension==4.0.11

I searched for the solution and this link's suggestion with the most likes, pip install --upgrade notebook==6.4.12, worked finally.
Maybe it's better to specify the requirement for this somewhere.

sadrasabouri · 2024-07-29T00:49:25Z

Plus when I was checking the output, after a run, I realized there is a None at the top left. If it's OK ignore this comment, but if it's a problem, we should fix it.

sepandhaghighi · 2024-07-31T08:14:16Z

@sadrasabouri
You are right 💯
fixed in 314aede

sepandhaghighi · 2024-07-31T08:15:47Z

Plus when I was checking the output, after a run, I realized there is a None at the top left. If it's OK ignore this comment, but if it's a problem, we should fix it.

@sadrasabouri I couldn't reproduce this issue, and I think it's also related to a version difference.

sadrasabouri

That looks good, but again trying to run it I got: (Both in Python 3.10 and Python 3.12)

Traceback (most recent call last):
  File ".../pycm/Otherfiles/notebook_to_html.py", line 7, in <module>
    import nbformat
  File ".../pycm/.venv/lib/python3.10/site-packages/nbformat/__init__.py", line 14, in <module>
    from . import v1
  File ".../pycm/.venv/lib/python3.10/site-packages/nbformat/v1/__init__.py", line 19, in <module>
    from .nbjson import reads as reads_json, writes as writes_json
  File ".../pycm/.venv/lib/python3.10/site-packages/nbformat/v1/nbjson.py", line 19, in <module>
    from base64 import encodestring
ImportError: cannot import name 'encodestring' from 'base64' (/opt/homebrew/Cellar/[email protected]/3.10.14_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/base64.py)

Maybe that's a problem with my operating system (rn it's MacOS)

sepandhaghighi · 2024-08-04T10:32:55Z

@sadrasabouri Could you please install pip install nbformat==5.0.8 and test it again? I think this case is a practical example that proves we need something like freez :D

sadrasabouri · 2024-08-05T05:14:48Z

pip install nbformat==5.0.8

It fixed it and there is another one now:

Traceback (most recent call last):
  File ".../pycm/Otherfiles/notebook_to_html.py", line 9, in <module>
    from nbconvert.preprocessors import ExecutePreprocessor
  File ".../pycm/.venv/lib/python3.12/site-packages/nbconvert/__init__.py", line 4, in <module>
    from .exporters import *
  File ".../pycm/.venv/lib/python3.12/site-packages/nbconvert/exporters/__init__.py", line 3, in <module>
    from .html import HTMLExporter
  File ".../pycm/.venv/lib/python3.12/site-packages/nbconvert/exporters/html.py", line 12, in <module>
    from jinja2 import contextfilter
ImportError: cannot import name 'contextfilter' from 'jinja2' (.../pycm/.venv/lib/python3.12/site-packages/jinja2/__init__.py)

sepandhaghighi · 2024-08-05T06:27:43Z

from jinja2 import contextfilter
ImportError: cannot import name 'contextfilter' from 'jinja2' (.../pycm/.venv/lib/python3.12/site-packages/jinja2/init.py)

@sadrasabouri
Downgrade your jinja2 to 2.10.1 and test it again.

sadrasabouri · 2024-08-05T17:28:56Z

OK, I tried that and it gave me another error.
That one I solved with pip install markupsafe==2.0.1, then it gave me another error:

Traceback (most recent call last):
  File ".../pycm/Otherfiles/notebook_to_html.py", line 9, in <module>
    from nbconvert.preprocessors import ExecutePreprocessor
  File ".../pycm/.venv/lib/python3.10/site-packages/nbconvert/__init__.py", line 4, in <module>
    from .exporters import *
  File ".../pycm/.venv/lib/python3.10/site-packages/nbconvert/exporters/__init__.py", line 3, in <module>
    from .html import HTMLExporter
  File ".../pycm/.venv/lib/python3.10/site-packages/nbconvert/exporters/html.py", line 12, in <module>
    from jinja2 import contextfilter
  File ".../pycm/.venv/lib/python3.10/site-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File ".../pycm/.venv/lib/python3.10/site-packages/jinja2/environment.py", line 16, in <module>
    from jinja2.defaults import BLOCK_START_STRING, \
  File ".../pycm/.venv/lib/python3.10/site-packages/jinja2/defaults.py", line 32, in <module>
    from jinja2.tests import TESTS as DEFAULT_TESTS
  File ".../pycm/.venv/lib/python3.10/site-packages/jinja2/tests.py", line 13, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/opt/homebrew/Cellar/[email protected]/3.10.14_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/collections/__init__.py)

I searched it and apparently it's an issue regarding to Python version. I tried Python 3.10 and 3.12 both failed.
It should be lower than 3.9 I guess.
You should probably test with Python 3.10, or we set a note somewhere to use it only in Python 3.9.

sepandhaghighi · 2024-08-06T08:29:36Z

I searched it and apparently it's an issue regarding to Python version. I tried Python 3.10 and 3.12 both failed. It should be lower than 3.9 I guess. You should probably test with Python 3.10, or we set a note somewhere to use it only in Python 3.9.

@sadrasabouri Yeah, you are right. There are some backward compatibility issues. We will fix them later.

I've made changes to doc-requirements.txt and RELEASE.md
(bb74f23 and a360c39)

Please re-review 🔥

sadrasabouri · 2024-08-07T06:42:59Z

Still not working. I got this on Python 3.9 (this time on Ubuntu):

 ____  _  _   ___  __  __ 
(  _ \( \/ ) / __)(  \/  )
 )___/ \  / ( (__  )    ( 
(__)   (__)  \___)(_/\/\_)

 _  _   __      ___  
( \/ ) /. |    / _ \ 
 \  / (_  _)  ( (_) )
  \/    (_) () \___/ 

. . .-. .-. .-. .-. .-. .-. . .   .-. .-. . . . . .-. .-. .-. 
|\| | |  |  |-  |(  | | | | |<    |   | | |\| | | |-  |(   |  
' ` `-'  '  `-' `-' `-' `-' ' `   `-' `-' ' ` `.' `-' ' '  '  
                                                              

Documents:
Processing ...
Traceback (most recent call last):
  File "/home/sadra/local/pycm/Otherfiles/notebook_to_html.py", line 62, in <module>
    ep.preprocess(
  File "/home/sadra/local/pycm/.venv/lib/python3.9/site-packages/nbconvert/preprocessors/execute.py", line 405, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "/home/sadra/local/pycm/.venv/lib/python3.9/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "/home/sadra/local/pycm/.venv/lib/python3.9/site-packages/nbconvert/preprocessors/execute.py", line 438, in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index, store_history)
  File "/home/sadra/local/pycm/.venv/lib/python3.9/site-packages/nbconvert/preprocessors/execute.py", line 578, in run_cell
    exec_reply = self._poll_for_reply(parent_msg_id, cell, timeout)
  File "/home/sadra/local/pycm/.venv/lib/python3.9/site-packages/nbconvert/preprocessors/execute.py", line 479, in _poll_for_reply
    if msg['parent_header'].get('msg_id') == msg_id:
TypeError: 'coroutine' object is not subscriptable
sys:1: RuntimeWarning: coroutine 'ZMQSocketChannel.get_msg' was never awaited

Also I we should add following lines to doc-requirements.txt:

jinja2==2.10.1
markupsafe==2.0.1

sadrasabouri

I left a comment, still not working.

sepandhaghighi · 2024-08-08T15:31:43Z

Also I we should add following lines to doc-requirements.txt:
jinja2==2.10.1
markupsafe==2.0.1

@sadrasabouri Added in 3445b8e

sepandhaghighi · 2024-08-08T15:34:02Z

I left a comment, still not working.

@sadrasabouri
It's a bit weird.
nbformat, nbconvert, and notebook have a very complex dependency, and it's very hard to find the problem. We should solve this compatibility problem later (in another PR).

sadrasabouri

OK, so we'll fix it later.

sepandhaghighi added 7 commits July 22, 2024 14:34

doc : notebook_to_html.py script added

363fa4c

fix : minor edit in notebook_to_html.py

517f8d4

fix : minor edit in notebook_to_html.py

7d1bdcf

fix : config added to notebook_to_html.py

94299e3

fix : minor edit in notebook_to_html.py config

211158b

fix : TocExporter added to notebook_to_html.py

c72be0c

del : doc_to_html.bat removed

f65deee

sepandhaghighi added Document test labels Jul 28, 2024

sepandhaghighi added this to the PYCM 4.1 milestone Jul 28, 2024

sepandhaghighi self-assigned this Jul 28, 2024

doc : RELEASE.md updated

197d147

sepandhaghighi marked this pull request as ready for review July 28, 2024 09:31

sepandhaghighi requested a review from sadrasabouri July 28, 2024 09:31

sadrasabouri requested changes Jul 29, 2024

View reviewed changes

sepandhaghighi added 2 commits July 31, 2024 12:29

fix : doc-requirements.txt added

314aede

doc : RELEASE.md updated

81a5e42

sepandhaghighi requested a review from sadrasabouri July 31, 2024 08:15

sadrasabouri requested changes Aug 4, 2024

View reviewed changes

sepandhaghighi added 2 commits August 6, 2024 12:49

fix : doc-requirements.txt updated

bb74f23

doc : RELEASE.md updated

a360c39

sepandhaghighi requested a review from sadrasabouri August 6, 2024 08:29

sadrasabouri requested changes Aug 7, 2024

View reviewed changes

fix : doc-requirements.txt updated

3445b8e

sepandhaghighi requested a review from sadrasabouri August 8, 2024 15:34

sadrasabouri approved these changes Aug 8, 2024

View reviewed changes

sadrasabouri merged commit 2ac0ca8 into dev Aug 8, 2024
45 checks passed

sadrasabouri deleted the notebook_to_html branch August 8, 2024 18:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Notebook to HTML script #543

Notebook to HTML script #543

sepandhaghighi commented Jul 28, 2024 •

edited

Loading

codecov bot commented Jul 28, 2024 •

edited

Loading

sadrasabouri left a comment

sadrasabouri commented Jul 29, 2024

sepandhaghighi commented Jul 31, 2024

sepandhaghighi commented Jul 31, 2024

sadrasabouri left a comment

sepandhaghighi commented Aug 4, 2024

sadrasabouri commented Aug 5, 2024

sepandhaghighi commented Aug 5, 2024 •

edited

Loading

sadrasabouri commented Aug 5, 2024

sepandhaghighi commented Aug 6, 2024

sadrasabouri commented Aug 7, 2024

sadrasabouri left a comment

sepandhaghighi commented Aug 8, 2024

sepandhaghighi commented Aug 8, 2024

sadrasabouri left a comment

Notebook to HTML script #543

Notebook to HTML script #543

Conversation

sepandhaghighi commented Jul 28, 2024 • edited Loading

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

codecov bot commented Jul 28, 2024 • edited Loading

Codecov Report

sadrasabouri left a comment

Choose a reason for hiding this comment

sadrasabouri commented Jul 29, 2024

sepandhaghighi commented Jul 31, 2024

sepandhaghighi commented Jul 31, 2024

sadrasabouri left a comment

Choose a reason for hiding this comment

sepandhaghighi commented Aug 4, 2024

sadrasabouri commented Aug 5, 2024

sepandhaghighi commented Aug 5, 2024 • edited Loading

sadrasabouri commented Aug 5, 2024

sepandhaghighi commented Aug 6, 2024

sadrasabouri commented Aug 7, 2024

sadrasabouri left a comment

Choose a reason for hiding this comment

sepandhaghighi commented Aug 8, 2024

sepandhaghighi commented Aug 8, 2024

sadrasabouri left a comment

Choose a reason for hiding this comment

sepandhaghighi commented Jul 28, 2024 •

edited

Loading

codecov bot commented Jul 28, 2024 •

edited

Loading

sepandhaghighi commented Aug 5, 2024 •

edited

Loading