"Hot reload" modules

[lukedays]

Hello Steven @cunnane!
First of all thanks for the library, I'm trying to migrate some VBA code to Python using it.

I'd like to know if there is some form of "hot reload": edit the python module, then immediately (or with the press of a button) see the code changes in Excel.
I tried using =xloPyLoad("module") but it didn't work.
Is there a way to do it? I have large-ish worksheets that take a long time to close and open again.

Best regards!

[lukedays]

I tried to create a master/slave structure, but it didn't work as well (I suspect due to object caching)

master.py

import xloil as xlo
import importlib

@xlo.func
def reload():
    xlo.scan_module(importlib.import_module("slave"))

slave.py

import xloil as xlo

@xlo.func
def add(x, y):
    return x + y

[lukedays]

I was able to do a workaround, not super clean but works:

import asyncio
import importlib
from pathlib import Path

import xloil as xlo
from watchdog.events import FileSystemEventHandler
from watchdog.observers import Observer


@xlo.func
async def Watcher():
    current_path = Path(__file__).parent

    # Initial load
    p = current_path.glob("**/*")
    for f in p:
        if f.is_file():
            reload_module(f)

    # Watch file changes
    obs = Observer()
    obs.schedule(Handler(), path=current_path, recursive=False)
    obs.start()

    try:
        while True:
            await asyncio.sleep(1)
            yield "Watching for file changes..."
    except KeyboardInterrupt:
        obs.stop()
    obs.join()


def reload_module(path):
    filename = Path(path).name.replace(".py", "")

    if filename == "__pycache__" or filename in Path(__file__).name:
        return

    try:
        m = importlib.import_module(filename)
        importlib.reload(m)
        xlo.scan_module(m)
    except ModuleNotFoundError:
        return


class Handler(FileSystemEventHandler):
    def on_any_event(self, event):
        reload_module(event.src_path)

[cunnane]

Nice workaround!

The hot reload feature is actually built into the core C++ layer... but the relevant events don't seem to be firing. I'm cooking up a fix for this.

Unfortunately xloPyLoad is also broken. The fix here is to call sys.path_importer_cache.clear() after the import hooks are installed at the end of xloil.importer. What is supposed to happen is that xlOil hooks python's import mechanism to ensure scan_module gets called, but because of the file loader caching, the hook doesn't affect any already-imported modules.

I'll let you know when these fixes are ready

[lukedays]

Great!! Yeah, it worked, the only problem is that if the functions' names are changed, the old ones keep existing. But the reload works, so no big issue.
I was trying to mimic how hot reload works in JS frameworks (maybe one of the few good things about those haha).
Thanks a lot!

[cunnane]

I was trying to mimic the reload in VBA, long before this new-fangled JS ;)

The auto-reload and xloPyLoad should be now fixed in v0.14.0.

[lukedays]

So xloPyLoad works, but I couldn't make the auto-reload to work... Even with the file save with the same name as the workbook. Do you have any tips?