Enable support for influxdb v2 as data source (PV) #3542
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The purpose of this PR is to enable the use of an InfluxDB v2 database as a data source.
InfluxDB is an open-source time series database designed to efficiently store and query large volumes of time-stamped data, such as monitoring metrics, events, and real-time analytics. It is optimized for high write and query performance, making it a popular choice for industrial applications and data monitoring solutions.
As @kasemir said :
As for InfluxDB, we had been working on support for that, see presentation under http://www.rri.kyoto-u.ac.jp/EPICS/program.html
It was faster than an RDB, but we eventually decided to not use it because
It's implemented in go, a somewhat uncommon language
The data is transferred in JSON, and the actual implementation resulted in lost detail for small/large floating point values, which was a problem for our vacuum readings which are something like "1e-20" which turned into "0". Reported, Incorrect long values returned due to json to java conversion influxdata/influxdb-java#276, but closed as “wont-fix” which was at the time quite frustrating.
While influx supports auto-aggregation, for example replacing older samples with the 1-hour-average to reduce the data size, how it does that exactly by storing the data under a different name wasn't convenient. The data retrieval would basically need to look in several different places: Is there recent data? Is there data reduced to the 1 hour avg? Is there data reduced to the daily avg?
Compared to RDB systems with a track record of having been available for decades and keeping data for that long, influx is still quite new. Cool for system logs or your Arduino fishtank controller data from last month. But control system data going back to 2006??
Also, the two options we have are pretty good: RDB for those who prefer generality, Archive Appliance for those who prefer something optimized for control system data.
InfluxDB has made some questionable choices regarding its open source version in version 3. We are currently exploring alternatives (as QuestDB) but we have no other option at the moment since certain existing systems rely on this database.
see also: 72 hours limit to fetch data