Monitor visitors somehow

[jacomyal]

Gephi project needs several server-side actions.
This issue discusses options to support them.

• Gephi.org analytics: we would like to track visits on the main website (mainly counts and geo origins but nothing else to respect GDPR)
• Gephi Desktop usage: the idea is to track the number of visits to the plugin version check which is done daily if Gephi is running (i.e. requesting https://raw.githubusercontent.com/gephi/gephi-plugins/gh-pages/plugins/${gephi.minor.version}/updates.xml)
• Gephi Lite: Github login proxy to fuel the github integration feature (currently done through a proxy on githubapi.gephi.org hosted on ouestware server)

We could do analytics with the same method as the Github API proxy, i.e. a stats.gephi.org subdomain which redirects to ouestware matomo server.

For the Gephi Desktop usage analytics is trickier.

The more straight forward solution would be to add yet another subdomain which points to one proxy on ouestware server which would redirect to the plugins file. But adding analytics on those proxy would require to analyse the proxy raw log files. This could be done by using the same matomo server with log analytics for the proxy : https://matomo.org/guide/tracking-data/import-server-logs/
Or we add a server controller in place of the nginx proxy (some JS or PHP code) which does the reverse proxy + a man in the middle request which triggers the analytic server.

In short we need a bit more of server side complexity.
Thus we might consider setting up a Gephi server to handle those instead or relying on ouestware server.

New server option:

The idea would be to rent a very simple server on which we can set up:

• an analytics server (i.e. Matomo ?) for gephi lite and gephi.org
• a reverse proxy to handle the Github integrations
• a reverse proxy to track plugin updates.xml requests through server log analytics

Such a server can cost 45€HT/year : https://www.ovhcloud.com/fr/vps/configurator/?planCode=vps-2025-model1&brick=VPS%2BModel%2B1&pricing=upfront12&processor=%20&vcore=4__vCore&storage=75__SSD__NVMe

[jacomyma]

My take on this:

• Money is not an issue (when talking about 50€/year). We can afford decades of this, of course.
• Server maintenance is potentially an issue. Questions worth answering: Who makes the updates? What gets impacted if the server gets down or gets hacked? How to we know when that happens, and who is in charge?
• Project ownership is at stake in the sense that in the spirit of how we've done so far, we like to minimize the coupling between entities. It's not a result of a lack of trust, but rather a Ulysse's pact: we won't get tempted to offload more and more things to a company (in this case Ouestware but it is in not specific to it) if we keep things under the direct control (governance) of the project itself.

On this last point, I like the solution to have a redirection managed by the project and pointing at a server potentially hosted at Ouestware or anywhere else. It gives the Gephi project the option to change the routing if needed it one day. I don't mind using the Ouestware infrastructure insofar as there is a safeguard (which does not mean it's necessarily the best option).

For clarity: it feels weird to write this stuff about Ouestware, people I deeply trust, but I intend to set a sustainable policy for the project that we can reuse later, over the long run, with other potential partners (if that happens one day), and potentially for a different Gephi team behind the wheel.

It looks to me that the "rent a new server" solution is the right one provided that we check that there is no significant hidden workload in the maintenance.

[mbastian]

Given that the URL used in Gephi to check for updates start with https://raw.githubusercontent.com it actually doesn't use the gephi.org domain. So, correct me if I'm wrong but whatever solution we put in place it would only be valid for future releases.

[paulgirard]

Yes unfortunately that's only for future release...

[mbastian]

Thanks for confirming. That's okay, there is nothing we can do about it.

Regarding the server, is it possible to keep only the proxy solution? Proxies like nginx are very lightweight/secure and usually require very little maintenance. We had Google Analytics until now for gephi.org and it works fine. Maybe we should consider keeping that, and therefore only have a proxy on that server.

[paulgirard]

Using Google analytics does not change the question IMHO. It actually forbids log analytics.

To track gephi.org users we need an analytic server. We can use Google Analytics (and potentially add a cookie banner for european traffic as data are not hosted in Europe), rent one for 20$/month hosted in Europe (minimum depends on traffic), use ouestware one or use a new dedicated server for Gephi.

To enable Github authentication (used in both Gephi Lite and Retina export plugin in Gephi) we need a proxy i.e. a server.

To track Gephi desktop usage (of the new future releases only), we need a proxy AND a self-hosted analytics server that does log analytics of that proxy. Track proxy requires log analytics and log analytics is affordable only if done self-hosted (as it avoids sending logs somewhere on the cloud).
Or if we have our own/ouestware server we could imagine the already mention "custom controller" which does proxy + ping some analytics (any server: Google or self hosted). But that means developing that and thus testing/maintaining it. I personally do not like this solution as it feels hacky too me.

To sum-up I'll say that tracking Gephi desktop usage requires us a more sophisticated server side solution than we had so far.
Abandoning this lowers our needs but even then we still need an GDPR compliant analytics + a proxy hosted somewhere (for Github auth).

To answer the very good question of Mathieu J. about resilience, let's draw the consequences of failures.
If that proxy/analytic server is down:

• we will lose analytics data (some 404 requests will happen in visitors web browsers)
• people will not be able to activate github authentication but existing connection might still work (I think?)
• Gephi will not be able to check for plugin updates and will receive 500, 501 or 404 HTTP error depending on the failure (handling such error in Gephi code might include a drawback URL i.e. the github one to ensure the update even if the proxy fails).

To answer Mathieu J. comment. Yes I think relying on OuestWare infra is a risk for the community. That's our motivation to raise the opportunity to take a server to kill that dependency. For instance, having a Gephi server will ease opening access to that server for people in Gephi core team but outside ouestware team to ensure redundancy. Opening access to our own ouestware server is a no go IMHO.

As for maintenance, we already take care of maintaining the reverse proxy used for Github authentication. We would happily take care of that server as long as the community asked us to do it or as long as we have the time/energy to do it.

[paulgirard]

Oh actually,

To track Gephi desktop usage we could proceed like this:

• add in the Gephi daily (plugin) check code a new call to a specific stats route like https://stats.gephi.org/desktop
• add somewhere in the configuration a flag to allow user to opt out those stats call (would be elegant)
• no need for proxy
• standard web analytics can be used
• analytics would fail, Gephi would just ignore the error code on the stat call and go on

Simpler.
More resilient.
More transparent to users.
Less demanding server side.

In which case we need to:

• choose an analytic platform + chose a server: cloud/self hosted/OuestWare server
• set up a stats.gephi.org subdomain that point to that server analytic API (might require a proxy)
• a proxy server for the Github Auth (which we already have on OuestWare server)

[mbastian]

Thanks for the explanation, I understand it better now. Adding a call to a stats route would require some work but you're right to call out that we should allow opting out. It's also worth thinking about a future-proof solution that could allow more fine-grained analytics, for instance which features users are using (layout, filters etc.).

While Matomo was primarily developed for Web, they have a Java SDK. I would need to investigate further how this works.

[paulgirard]

I move back this card to backlog to work on that later