Skip to content
This repository has been archived by the owner on Mar 12, 2020. It is now read-only.

Use Neo4J backend #19

Closed
tdauth opened this issue Mar 18, 2019 · 7 comments
Closed

Use Neo4J backend #19

tdauth opened this issue Mar 18, 2019 · 7 comments
Assignees
Labels
enhancement New feature or request

Comments

@tdauth
Copy link
Contributor

tdauth commented Mar 18, 2019

See #5 for the investigation of GraphDBs.

Neo4J allows transactions which provide concurrency support.
It can also visualize the state machines/graphs.

Labels group nodes together, so a SutState is not really a label.
An action type can be a label.

Some information about OGM:

We store only the hashes and some additional information of the SutStates and actions in the graphdb.
Otherwise, it would be an antipattern.
Note that we need to use the Bolt protocol to utilize most of the tools for visualization and management.
Embedded databases are useful for storage only.

Open questions:

  • How can we separate graphs with Bolt? Different port numbers?
  • How does the user specify username and password for one graph database using the API?
  • Does the user have to setup a Bolt Neo4J database before he/she can use the API or does our API automatically set it up for the first time?
  • Which tools are the best for management and visualization? I have found the Desktop application https://neo4j.com/developer/neo4j-desktop/ which allows you to start Bolt graph databases and brings a browser which can visualize them. The browser's UI is available in a web browser as well. There are import/export apoc procedures which can be installed for GraphML etc.: http://neo4j-contrib.github.io/neo4j-apoc-procedures/3.5/export-import/graphml/
@tdauth tdauth added the enhancement New feature or request label Mar 18, 2019
@tdauth tdauth self-assigned this Mar 18, 2019
@beatngu13
Copy link
Contributor

The question is also: Do we drop the in-memory representation of the SUT model as soon as we have the DB backend?

tdauth added a commit that referenced this issue Mar 25, 2019
Use an embedded database and the Java API.
tdauth added a commit that referenced this issue Mar 25, 2019
Use an embedded database and the Java API.
tdauth added a commit that referenced this issue Mar 26, 2019
tdauth added a commit that referenced this issue Mar 27, 2019
We use converters and OGM annotations instead of converting SutStates
and Actions manually.
Open sessions for all state machines and create transactions for the sessions.
tdauth added a commit that referenced this issue Mar 27, 2019
It was only necessary for a standalone REST service.
tdauth added a commit that referenced this issue Mar 27, 2019
tdauth added a commit that referenced this issue Mar 27, 2019
@beatngu13
Copy link
Contributor

Regarding the node content, BLOBs seem to be an anti-pattern:

If you have to deal with BLOB data, choose an appropriate store for that use case and use Neo4j to store the URL that points you to the binary data.

This is also what @martin-v suggested, to only reference the state in Neo4j and store the actual data somewhere else, e.g., a key-value store.

The question is: How can we deterministically derive a unique ID from the state data that can be used as a reference. This also heavily depends on the upcoming artifact storage.

For now, we decided to go with a filtered textual representation of the state in Neo4j.

tdauth added a commit that referenced this issue Mar 28, 2019
tdauth added a commit that referenced this issue Mar 28, 2019
tdauth added a commit that referenced this issue Apr 2, 2019
Use an embedded database and the Java API.
tdauth added a commit that referenced this issue Apr 2, 2019
tdauth added a commit that referenced this issue Apr 2, 2019
We use converters and OGM annotations instead of converting SutStates
and Actions manually.
Open sessions for all state machines and create transactions for the sessions.
tdauth added a commit that referenced this issue Apr 2, 2019
It was only necessary for a standalone REST service.
tdauth added a commit that referenced this issue Apr 2, 2019
tdauth added a commit that referenced this issue Apr 2, 2019
tdauth added a commit that referenced this issue Apr 2, 2019
tdauth added a commit that referenced this issue Apr 2, 2019
tdauth added a commit that referenced this issue Apr 2, 2019
@tdauth
Copy link
Contributor Author

tdauth commented Apr 9, 2019

We have to use Scala 2.11.x neo4j/neo4j#8832

For visualization we could use the desktop tool or convert the graph into a .dot file: https://gist.github.com/peterneubauer/2652082

tdauth added a commit that referenced this issue Apr 9, 2019
Use an embedded database and the Java API.
tdauth added a commit that referenced this issue Apr 9, 2019
tdauth added a commit that referenced this issue Apr 9, 2019
We use converters and OGM annotations instead of converting SutStates
and Actions manually.
Open sessions for all state machines and create transactions for the sessions.
tdauth added a commit that referenced this issue Apr 10, 2019
We use converters and OGM annotations instead of converting SutStates
and Actions manually.
Open sessions for all state machines and create transactions for the sessions.
tdauth added a commit that referenced this issue Apr 10, 2019
It was only necessary for a standalone REST service.
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
We do not need converters anymore. Store hash values only.
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
tdauth added a commit that referenced this issue Apr 10, 2019
This fixes persisting entities.
tdauth added a commit that referenced this issue Apr 10, 2019
Sessions are not thread-safe in Neo4J.
Therefore, we have to create a new session per transaction.
Simplify the transactions and name all the properties.
tdauth added a commit that referenced this issue Apr 10, 2019
Otherwise the conversion by OGM throws an exception.
tdauth added a commit that referenced this issue Apr 10, 2019
@beatngu13
Copy link
Contributor

Side note: Neo4j is licensed under GPLv3. So if we use Neo4j, we have to check what surili components need a compatible license.

@beatngu13
Copy link
Contributor

From https://neo4j.com/licensing/:

We chose GPL [v3] because it means that Neo4j Community Edition can be used for free with your project: whether in the cloud or behind the firewall. If you’re building an application that accesses a Neo4j database running inside your organization (or on any personal device), then you are free to use Community Edition, whether or not your software is closed source.

Sounds good, but the questions is also does the CE include everything we need?

@tdauth
Copy link
Contributor Author

tdauth commented Apr 14, 2019

https://neo4j.com/subscriptions/#editions

Maybe better abstraction with http://tinkerpop.apache.org/

tdauth added a commit that referenced this issue Apr 16, 2019
tdauth added a commit that referenced this issue Apr 16, 2019
tdauth added a commit that referenced this issue Apr 16, 2019
tdauth added a commit that referenced this issue Apr 16, 2019
- Store Neo4J databases in a directory.
- Remove execution counter to simplify code.
- Reuse the sutState and action messages when creating identifiers.
- Simplify tests and reuse them for Neo4J.
tdauth added a commit that referenced this issue Apr 17, 2019
tdauth added a commit that referenced this issue Apr 17, 2019
Neo4JConfig allows different implementations.
GuiStateMachineApi must be implemented for embedded databases/Bolt.
We cache the session factories which is required for embedded databases.
@tdauth
Copy link
Contributor Author

tdauth commented Apr 17, 2019

Documented advantages vs disadvantages. Simple database like PostgreSQL or SQLite would be sufficient. If we really need advanced features from Neo4J, we can still use it in the future.

@tdauth tdauth closed this as completed Apr 17, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Development

No branches or pull requests

2 participants