Could exporting drawings to NoteKit be supported?

[sp1ritCS]

Hi, I hope posting this as an issue here is fine, but I've been thinking about buying myself one of those Intous Paper-Edition tablets to take notes with my preferred note-taking application notekit.

Because of this, please excuse my "dis-knowlege" of tuhi and it's architecture/features, without such a tablet I obviously haven't been able to test it myself yet and besides the screenshots in the screenshot branch, there is practically nothing to go off on.

NoteKit itself already supports "normal" drawings for when the device is connected over usb/bt, however I'm interested in a convenient way to transfer drawings, stored on the device itself, that tuhi exposes.

I kinda imagined it as having an additional buttion (only visible if com.github.blackhole89.notekit is registered on the session bus) on the top bar of the drawing screen.

I've done some work in that direction on notekit already (my dbus branch) that includes the ability to add so-called "external widgets" (basically an updatable image from the external application with an onclick action) via dbus. (So tuhi could provide such a "widget" with the svg of the drawing, and an onclick action to open it back up in tuhi and merge additional layers from there, then save it back to notekit)

I'm obviously intending to make these changes myself and I'd just be interested to know if you'd be willing to merge something like that.

[bentiss]

Hi, and thanks for the bug.

As a foreword, as you can see by looking at the git history, this project is not active, and we occasionally fix a bug here and there, but the original developers won't have time to dedicate to it to implement new features.

Which means 2 things:

1. please do not expect us to do any ground work as we don ´t have time to work on it
2. there is likely a maintainer place available :)

That being said, regarding the RFE (given that you plan on writing stuff yourself):

• I am a little bit hesitant when we start saying "let's add a button for this application". Basically, why would we support Notekit over a KDE application, or Xournal? That being said, if there is a "standard" way of exposing that functionality to the desktop, sure, why not?
• tuhi pulls the raw data from the device, and exports them as svg, for convenience. The UI is a nice touch, but honestly, you can run the program without it, and it works very well in CLI. So depending on what you want to achieve (I am a little bit blurry on the architecture you are having there), I do not see why we couldn't have another dbus user daemon, that talks to a known (standard if possible) dbus interface, that can be triggered by Notekit to pull the drawings into it.

So basically, I am not preventing you to do what you want there, but please just keep in mind that to have your idea more easily accepted, it's best that we don't focus on the remote application, but more on a generic way to serve drawings.

You could either define a tuhi specific dbus protocol to retrieve drawings and have a dedicated plugin in NoteKit, or define a generic protocol to communicate from NoteKit to Tuhi.

PS: thanks for the tip on NoteKit, seems interesting . I'll keep an eye on it :)

[sp1ritCS]

I am a little bit hesitant when we start saying "let's add a button for this application". Basically, why would we support Notekit over a KDE application, or Xournal? That being said, if there is a "standard" way of exposing that functionality to the desktop, sure, why not?

There is sadly no "standard" way to do sharing on GNU systems. Especially in this case, where a "share" implementation like is found on many mobile OS's is lacking the bi-directionality this provides (something more akin to Androids ContentProvider would be required).
If cross-application support is desired, you could make use of g_desktop_app_info_get_implementations and show a drop down (if it's more than one) for each application that implements that interface. But given that my current com.github.blackhole89.NoteKit.Notebook is designed arround notekits needs, other applications might have different requirements.

tuhi pulls the raw data from the device, and exports them as svg, for convenience.

what does that "raw data" contain and how does it look like? Because notekit's internal binary drawing representation might also be a possibility.

The problem with that is:

• that there is no "sophisticated algorithm" that converts it's data into strokes. Upstream just creates a bunch of line segments, with the thickness being the pressure, while my master-branch uses a bunch of bezier-segments converted from catmul-rom. I've been wanting to implement an actually better stroke render algorithm for a long time, but it's quite complex 🙃. How does tuhi render it's strokes onto the vector surface?
• Once the stroke is inserted, new strokes could be added the the current drawing via the normal "usb/bt" mode, however the drawing would obviously not be aligned/scaled with the paper sheet anymore. So merging new layers onto a drawing would become a lot more difficult.
• it would go against your intension of generalizing the sharing problem. while notekit's strokes might be one of the most efficient ways to store stroke information, it is obviously exclusively part of notekit.

You could either define a tuhi specific dbus protocol to retrieve drawings and have a dedicated plugin in NoteKit, or define a generic protocol to communicate from NoteKit to Tuhi.

NoteKit currently doesn't have a plugin system, but since my master has a lot of changes that blackhole89 doesn't want but others might, we've been thinking about implementing a plugin system for such features. We're currently working on getting the two branches less diverged, might look into a plugin system after that.

[bentiss]

Sorry for the delay, your answer went through the cracks

There is sadly no "standard" way to do sharing on GNU systems. Especially in this case, where a "share" implementation like is found on many mobile OS's is lacking the bi-directionality this provides (something more akin to Androids ContentProvider would be required).

What I meant is "define" the standard yourself :)

From a maintenance point of view, I think it would be best not to have to teach a notekit specific language to tuhi, because it would be an endless thing to add.
OTOH, tuhi already heavily relies on dbus for the communication between the UI and the server (see DBusInterface.md)

The interface we have here is deeply rooted with how the devices behind tuhi works, but we could have a thin layer above that would be standard enough to be more easily usable by other applications. Either directly in tuhi, either by using a dbus proxy that converts the high level interface from the tuhi specifics.

If cross-application support is desired, you could make use of g_desktop_app_info_get_implementations and show a drop down (if it's more than one) for each application that implements that interface. But given that my current com.github.blackhole89.NoteKit.Notebook is designed arround notekits needs, other applications might have different requirements.

My whole point is to not have to teach tuhi about others. We are not pulling much effort in the project so if we start adding support for X, and that interface changes in the future, we will likely have angry users.

what does that "raw data" contain and how does it look like? Because notekit's internal binary drawing representation might also be a possibility.

See DBusInterface.md: each point in the drawing, with its unique pressure.

• that there is no "sophisticated algorithm" that converts it's data into strokes. Upstream just creates a bunch of line segments, with the thickness being the pressure, while my master-branch uses a bunch of bezier-segments converted from catmul-rom. I've been wanting to implement an actually better stroke render algorithm for a long time, but it's quite complex upside_down_face. How does tuhi render it's strokes onto the vector surface?

We export the raw points when required from dbus IIRC, and then the application GUI does transformation to convert this into a SVG. I used to create a bunch of segments, but then we decided to reduce the size of the SVG by using some different algorithm I don't recall.

NoteKit currently doesn't have a plugin system, but since my master has a lot of changes that blackhole89 doesn't want but others might, we've been thinking about implementing a plugin system for such features. We're currently working on getting the two branches less diverged, might look into a plugin system after that.

The problem with plugin is that you open up the application and have to provide some sort of stable API. So I understand not willing to integrate plugins. But depending on how much the backend of notokit and the one from tuhi diverges, you might be able to integrate them together. Tuhi has a dbus interface you can use, and that was the whole point of splitting between the server and the GUI: so it can be reused.

[sp1ritCS]

What I meant is "define" the standard yourself :)

yes, this is what I ment with g_desktop_app_info_get_implementations and a common interface. I think if I were to rename the methods, it could in theory act as some kind of ContentProvider replacement.

---

But I totally see why you'd rather have external applications implementing tuhi rather than tuhi implementing external applications. The problem with that being, that there isn't any "tuhi" library that makes interfacing with tuhi(s DBus-Interface) particularity easy.

What might work is re-implementing a lot of tuhi-gui's functionality in a C-library (maybe split in libtuhi-common and libtuhi-gtk3 to allow for future expand-ability, like with gtk4 or qt5) that could then expose a singular drawing widget, but also a widget that displays a grid of drawings stored on the device/tuhi that signals the implementing application when such a drawing has been pressed.

but then we decided to reduce the size of the SVG by using some different algorithm I don't recall.

if I understood the logic in

tuhi/tuhi/export.py

Line 97 in e44e982

for sk_num, stroke_points in enumerate(self.output_strokes):

right, you basically merge similarly think line segments into one path.

This seems more like a workaround to address the inefficiencies of SVG(/Postscript-based vector formats). Ideally there were a free library that converts stroke data to proper paths (like MS WPF.Ink or Wacom WILL).

I also wonder about the necessity of parsing (rather large) json files (also potentially transferring these large files over dbus, can't really make out of the docs, whether GetJSONData returns the file contents or the file path), to get the raw drawing data. There is no way that the tablet transfers that data as json to tuhi, so why convert it to an ascii format? I'd rather have some kind of simple binary blob to parse than a giant json one.

The problem with plugin is that you open up the application and have to provide some sort of stable API.

yeah, I was thinking providing one smaller "stable API", that plugins can develop against, but also provide an "unstable API" for in-tree plugins. "unstable" plugins will have access to notekit's headerfiles and will simply receive a pointer to CMainWindow, the rest is up to them.

[whot]

There is no way that the tablet transfers that data as json to tuhi, so why convert it to an ascii format?

The data the tablet transfers is a binary blob and a good chunk of what makes tuhi useful is parsing that blob into something meaningful. The stroke protocol also depends on the tablet model, so just forwarding it is pretty useless. JSON otoh has decent support in many languages and having a stable json format means we can fix tablet-specific issues while still keeping the data itself as raw as possible so it can easily be converted into whatever format you want it to be.

The real problem though is one of tuhi's design: Tuhi is basically designed as a single application that downloads the drawings. The DBus interface is more a historical artifact, when initially tuhi was intended as separate server/client application. That was too time-consuming to finish though, and it was considered a bit too niche - we don't need yet another daemon constantly active that is only used on the rare occasion where some drawings need to be downloaded.

So now tuhi's server component and GUI are coupled together, even though the DBus interface still exists. But that interface is only active while Tuhi is running and primarily serves as internal interface. Also, yes, the json is transferred as string.

So a better option would likely be to check what we actually want to solve in the end (access to drawings in some non-hw-specific format) and redesign tuhi for that purpose. Whether that's a public directory where the files get dumped in or some other DBus exposure, or something else is the question.

[sp1ritCS]

The stroke protocol also depends on the tablet model

oh, ok. then it's more understandable-

---

I do have such a tablet now, however it seems that tuhi isn't really working with it. It seemingly is only able to download drawings after the initial pairing, often fails to even connect on later attempts of syncing (something to do with le-connection-abort-by-local) and looses the tablet entirely (without the ability to re-pair without deleting ~/.local/share/tuhi/) after a few restarts.

[whot]

Can we move the non-working bit to a separate issue please? Not that anyone has much time to debug this. The logs are in .local/share/tuhi/<MAC>/raw/ btw, that may help debugging what's going on.