Skip to content

Commit 93b8a5a

Browse files
barrycaceresbarrycaceres
committed
Added README.md files in multiple directories.
Added configuration/InitDefaultConfig.java Updated usage message for SnippetRunner.java
1 parent 5ef6d3f commit 93b8a5a

File tree

13 files changed

+320
-5
lines changed

13 files changed

+320
-5
lines changed

‎java/README.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Java Snippets
2+
3+
The Java snippets are contained in the `snippets` directory under various Java package directories. They can built using the `pom.xml` in this directory using `mvn package`. The result will be the `sz-sdk-snippets.jar` file ni the `target` directory.
4+
5+
There are several ways to run the code snippets.
6+
7+
## Run Directly
8+
9+
You may run any individual Snippet class directly providing you have a Senzing repository to run it with and the `SENZING_ENGINE_CONFIGURATION_JSON` environment variable set for connecting to that repository. Many of the snippets will find a default data file to run with if run from this directory, but also allow the caller to use a different data file if given by the first command-line arguemnt.
10+
11+
1. Run a snippet that takes no command-line arguments.
12+
```
13+
java -cp target/sz-sdk-snippets.jar loading.LoadRecords
14+
```
15+
16+
2. Run a snippet and override the input file using command-line arguments
17+
```
18+
java -cp target/sz-sdk-snippets.jar loading.LoadRecordsViaLoop ../../resources/data/load-500-with-errors.jsonl
19+
```
20+
21+
# Run Individually via Runner
22+
23+
The `com.senzing.runner.SnippetRunner` class will run one or more snippets for you and create a temporary Senzing repository to run
24+
then against. This is the `Main-Class` of the `sz-sdk-snippets.jar` file so it can be executed using `java -jar target/sz-sdk-snippets.jar`.
25+
26+
**NOTE:** When code snippets are run this way you cannot specify command-line arguments for individual snippets, nor can you respond to command-line input requests (they will be automatically be responded by the runner -- including forced termination of a snippet that is intended to run indefinitely).
27+
28+
1. Execute all code snippets:
29+
```
30+
java -jar target/sz-sdk-snippets.jar all
31+
```
32+
33+
2. Execute all code snippets in a Java package:
34+
```
35+
java -jar target/sz-sdk-snippets.jar loading
36+
```
37+
38+
3. Execute all code snippets from multiple packages:
39+
```
40+
java -jar target/sz-sdk-snippets.jar loading redo
41+
```
42+
4. Execute specific code snippets:
43+
```
44+
java -jar target/sz-sdk-snippets.jar loading.LoadViaLoop loading.LoadViaQueue
45+
```
46+
5. Mix and match packages with individual snippets:
47+
```
48+
java -jar target/sz-sdk-snippets.jar redo loading.LoadViaLoop
49+
```
50+
6. Generate a help message by specifying no arguments:
51+
```
52+
java -jar target/sz-sdk-snippets.jar
53+
54+
java -jar sz-sdk-snippets.jar [ all | <group> | <snippet> ]*
55+
56+
- Specifying no arguments will print this message
57+
- Specifying "all" will run all snippets
58+
- Specifying one or more groups will run all snippets in those groups
59+
- Specifying one or more snippets will run those snippet
60+
61+
Examples:
62+
63+
java -jar sz-sdk-snippets.jar all
64+
65+
java -jar sz-sdk-snippets.jar loading.LoadRecords loading.LoadViaFutures
66+
67+
java -jar sz-sdk-snippets.jar initialization deleting loading.LoadRecords
68+
69+
Snippet Group Names:
70+
- configuration
71+
- deleting
72+
- information
73+
- initialization
74+
- loading
75+
- redo
76+
- searching
77+
- stewardship
78+
79+
Snippet Names:
80+
- configuration.AddDataSources
81+
- configuration.InitDefaultConfig
82+
- deleting.DeleteViaFutures
83+
- deleting.DeleteViaLoop
84+
- deleting.DeleteWithInfoViaFutures
85+
- information.CheckDatastorePerformance
86+
- information.GetDatastoreInfo
87+
- information.GetLicense
88+
- information.GetVersion
89+
- initialization.EnginePriming
90+
- initialization.EnvironmentAndHubs
91+
- initialization.PurgeRepository
92+
- loading.LoadRecords
93+
- loading.LoadTruthSetWithInfoViaLoop
94+
- loading.LoadViaFutures
95+
- loading.LoadViaLoop
96+
- loading.LoadViaQueue
97+
- loading.LoadWithInfoViaFutures
98+
- loading.LoadWithStatsViaLoop
99+
- redo.LoadWithRedoViaLoop
100+
- redo.RedoContinuous
101+
- redo.RedoContinuousViaFutures
102+
- redo.RedoWithInfoContinuous
103+
- searching.SearchRecords
104+
- searching.SearchViaFutures
105+
- stewardship.ForceResolve
106+
- stewardship.ForceUnresolve
107+
```

‎java/runner/java/com/senzing/runner/SnippetRunner.java‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -340,7 +340,7 @@ private static void executeSnippet(String snippet,
340340
}
341341

342342
private static void printUsage(SortedMap<String, SortedSet<String>> snippetMap) {
343-
System.err.println("java -jar sz-sdk-snippets.jar [ all | <group> | <snippet> ]* ]");
343+
System.err.println("java -jar sz-sdk-snippets.jar [ all | <group> | <snippet> ]*");
344344
System.err.println();
345345
System.err.println(" - Specifying no arguments will print this message");
346346
System.err.println(" - Specifying \"all\" will run all snippets");
@@ -351,9 +351,9 @@ private static void printUsage(SortedMap<String, SortedSet<String>> snippetMap)
351351
System.err.println();
352352
System.err.println(" java -jar sz-sdk-snippets.jar all");
353353
System.err.println();
354-
System.err.println(" java -jar sz-sdk-snippets.jar loading.AddRecords loading.AddFutures");
354+
System.err.println(" java -jar sz-sdk-snippets.jar loading.LoadRecords loading.LoadViaFutures");
355355
System.err.println();
356-
System.err.println(" java -jar sz-sdk-snippets.jar initialization deleting loading.AddRecords");
356+
System.err.println(" java -jar sz-sdk-snippets.jar initialization deleting loading.LoadRecords");
357357
System.err.println();
358358
System.err.println("Snippet Group Names:");
359359
snippetMap.keySet().forEach(group -> {

‎java/snippets/configuration/InitDefaultConfig.java‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
package configuration;
2+
3+
import com.senzing.sdk.*;
4+
import com.senzing.sdk.core.SzCoreEnvironment;
5+
6+
/**
7+
* Provides a simple example of adding records to the Senzing repository.
8+
*/
9+
public class InitDefaultConfig {
10+
public static void main(String[] args) {
11+
// get the senzing repository settings
12+
String settings = System.getenv("SENZING_ENGINE_CONFIGURATION_JSON");
13+
if (settings == null) {
14+
System.err.println("Unable to get settings.");
15+
throw new IllegalArgumentException("Unable to get settings");
16+
}
17+
18+
// create a descriptive instance name (can be anything)
19+
String instanceName = InitDefaultConfig.class.getSimpleName();
20+
21+
// initialize the Senzing environment
22+
SzEnvironment env = SzCoreEnvironment.newBuilder()
23+
.settings(settings)
24+
.instanceName(instanceName)
25+
.verboseLogging(false)
26+
.build();
27+
28+
try {
29+
// get the config and config manager from the environment
30+
SzConfig config = env.getConfig();
31+
SzConfigManager configMgr = env.getConfigManager();
32+
33+
// prepare an in-memory config to be modified and get the handle
34+
long configHandle = config.createConfig();
35+
String configDefinition = null;
36+
try {
37+
configDefinition = config.exportConfig(configHandle);
38+
39+
} finally {
40+
config.closeConfig(configHandle);
41+
}
42+
43+
// add the modified config to the repository with a comment
44+
long configId = configMgr.addConfig(
45+
configDefinition, "Initial configuration");
46+
47+
// replace the default config
48+
configMgr.setDefaultConfigId(configId);
49+
50+
} catch (SzException e) {
51+
// handle any exception that may have occurred
52+
System.err.println("Senzing Error Message : " + e.getMessage());
53+
System.err.println("Senzing Error Code : " + e.getErrorCode());
54+
e.printStackTrace();
55+
throw new RuntimeException(e);
56+
57+
} catch (Exception e) {
58+
e.printStackTrace();
59+
if (e instanceof RuntimeException) {
60+
throw ((RuntimeException) e);
61+
}
62+
throw new RuntimeException(e);
63+
64+
} finally {
65+
// IMPORTANT: make sure to destroy the environment
66+
env.destroy();
67+
}
68+
69+
}
70+
}

‎java/snippets/configuration/README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Deleting Data
2+
The configuration snippets outline how to modify the Senzing configuration, register the modified configuration with a configuration ID and update the default configuration ID for the repository.
3+
4+
You may either `setDefaultConfigId()` or `replaceDefaultConfigId()`. Initially, the the default config ID must be set since there is no existing config ID to replace. However, when updating you may use `replaceDefaultConfigId()` to guard against race conditions of multiple threads or processes updating at the same time.
5+
6+
## Snippets
7+
* **AddDataSources.java**
8+
* Gets the current default config, creates a modified config with additional data sources, registers that modified config and then replaces the default config ID.
9+
* **InitDefaultConfig.java**
10+
* Initializes the repository with a default config ID using the template configuration provided by Senzing.
11+

‎java/snippets/deleting/README.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Deleting Data
2+
3+
The deletion snippets outline deleting previously added source records. Deleting source records removes the previously added source record from the system, completes the entity resolution process and persists outcomes in the Senzing repository.
4+
5+
Deleting a record only requires the data source code and record ID for the record to be deleted.
6+
7+
## Snippets
8+
9+
- **DeleteViaFutures.java**
10+
- Read and delete source records from a file using multiple threads
11+
- **DeleteViaLoop.java**
12+
- Basic read and delete source records from a file
13+
- **DeleteWithInfoViaFutures.java**
14+
- Read and delete source records from a file using multiple threads
15+
- Collect the response using the [SZ_WITH_INFO flag](../../../README.md#with-info) on the `deleteRecord()` method and track the entity ID's.

‎java/snippets/information/README.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# System & Repository Information
2+
3+
The information snippets outline the retrieval of different informational aspects of a Senzing instance or engine.
4+
5+
## Snippets
6+
7+
- **CheckDatastorePerformance.java**
8+
- Run an insert test against the Senzing repository to gauge performance
9+
- **GetDatastoreInfo.java**
10+
- Return basic information about the Senzing repository(s)
11+
- **GetLicense.java**
12+
- Return the currently in use license details
13+
- **GetVersion.java**
14+
- Return the current Senzing product version details

‎java/snippets/initialization/README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Initialization
2+
3+
## Snippets
4+
5+
- **EnginePriming.java**
6+
- Priming the Senzing engine before use loads resource intensive assets upfront. Without priming the first SDK call to the engine will appear slower than usual as it causes these assets to be loaded
7+
- **EnvironmentsAndHubs.java**
8+
- Basic example of how to create an abstract Senzing factory and each of the available engines
9+
- **PurgeRepository.java**
10+
- **WARNING** This script will remove all data from a Senzing repository, use with caution! **WARNING**
11+
- It will prompt first, still use with caution!

‎java/snippets/loading/README.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Loading Data
2+
3+
The loading snippets outline adding new source records. Adding source records ingests [mapped](https://senzing.zendesk.com/hc/en-us/articles/231925448-Generic-Entity-Specification-JSON-CSV-Mapping) JSON data, completes the entity resolution process and persists outcomes in the Senzing repository. Adding a source record with the same data source code and record ID as an existing record will replace it.
4+
5+
## Snippets
6+
7+
- **LoadRecords.java**
8+
- Basic iteration over a few records, adding each one
9+
- **LoadTruthSetWithInfoViaLoop.java**
10+
- Read and load from multiple source files, adding a sample truth
11+
- Collect the response using the [SZ_WITH_INFO flag](../../../README.md#with-info) on the `addRecord()` method and track the entity ID's for the records.
12+
- **LoaeViaFutures.java**
13+
- Read and load source records from a file using multiple threads
14+
- **LoadViaLoop.java**
15+
- Basic read and add source records from a file
16+
- **LoadViaQueue.java**
17+
- Read and load source records using a queue
18+
- **LoadWithInfoViaFutures.java**
19+
- Read and load source records from a file using multiple threads
20+
- Collect the response using the [SZ_WITH_INFO flag](../../../README.md#with-info) on the `addRecord()` method and track the entity ID's for the records.
21+
- **LoadWithStatsViaLoop.java**
22+
- Basic read and add source records from a file
23+
- Periodic calling to `getStats()` method during load to track loading statistics.

‎java/snippets/redo/README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Redo Records
2+
3+
The redo snippets outline processing redo records. During normal processing of loading, deleting and replacing data the Senzing engine may determine additional work needs to be completed for an entity. There are times the Senzing engine will decide to defer this additional work. Examples of why this may happen include:
4+
5+
- Records loaded in parallel are clustering around the same entities causing contention
6+
- Automatic corrections
7+
- Cleansing decisions made on attributes determined to no longer be useful for entity resolution
8+
9+
When an entity requires additional work a record is automatically created in the system indicating this requirement. These records are called redo records. Redo records need to be periodically or continuously checked for and processed. Periodically is suitable after manipulating smaller portions of data, for example, at the end of a batch load of data. In contrast, a continuous process checking for and processing redo records is suitable in a streaming system that is constantly manipulating data. In general, it is recommended to have a continuous redo process checking for any redo records to process and processing them.
10+
11+
## Snippets
12+
13+
- **LoadWithRedoViaLoop.java**
14+
- Read and load source records from a file and then process any redo records
15+
- **RedoContinuous.java**
16+
- Basic example of continuously monitoring for redo records to process
17+
- **RedoContinuousViaFutures.java**
18+
- Continuously monitor for redo records to process using multiple threads
19+
- **RedoWithInfoContinuous.java**
20+
- Continuously monitor for redo records to process
21+
- Collect the response using the [SZ_WITH_INFO flag](../../../README.md#with-info) on the `processRedoRecord()` method and track the entity ID's for the records.

‎java/snippets/searching/README.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Searching for Entities
2+
3+
The search snippets outline searching for entities in the system. Searching for entities uses the same mapped JSON data [specification](https://senzing.zendesk.com/hc/en-us/articles/231925448-Generic-Entity-Specification-JSON-CSV-Mapping) as SDK methods such as `add_record()` to format the search request.
4+
5+
There are [considerations](https://senzing.zendesk.com/hc/en-us/articles/360007880814-Guidelines-for-Successful-Entity-Searching) to be aware of when searching.
6+
7+
## Snippets
8+
9+
- **SearchRecords.java**
10+
- Basic iteration over a few records, searching for each one
11+
- To see results first load records with [LoadTruthSetWithInfoViaLoop.java](../loading/LoadTruthSetViaLoop.java)
12+
- **SearchViaFutures.java**
13+
- Read and search for records from a file using multiple threads
14+
- To see results first load records with [LoadViaFutures.java](../loading/LoadViaFutures.java)

0 commit comments

Comments
 (0)