bug fixes, doc changes, updates to tests

.gitignore

@@ -13,3 +13,4 @@ dependency-reduced-pom.xml
 
 ## Test
 asyncapi-importer-core/src/test/resources/config.properties
+temp/

README.md

@@ -6,24 +6,35 @@ This project provides tooling to import AsyncAPI specs into Event Portal.
 - Maven 3.6+
 
 # Steps to build the project
-Build the project using maven, skipping JUnit tests:
+Build the project using maven:
+
+`mvn clean install`
+
+Build the project usisng Maven, explicitly skipping JUnit tests:
 
 `mvn clean install -DskipTests=true`
 
+## JUnit tests
+There are a number of JUnit tests designed to test code functionality. Many of these tests require live access to an Event Portal instance and a configured target Application Domain. Accordingly, these tests are not designed to run automatically.
+
 JUnit tests require access and configuration for an Event Portal account and Application Domain.
 
 # Modules
 
 ## AsyncApi Accessor v2
 This module provides the capability to parse and extract objects from an AsyncAPI specifications.
-Convenience methods are provided to obtain objects and fields, including Event Portal Extensions. 
 This module was created to handle AsyncAPI versions 2.X. The intent is for this module to be used in
-applications where needed.
+applications where needed. Convenience methods are provided to obtain objects and fields, including Event Portal Extensions. 
 
-## AsyncApi Importer - Core
-This module provides the core functionality of the importer. References the AsyncApi Accessor v2 module.
+## AsyncApi Importer Core
+This module provides the core functionality of the importer.
+- References the AsyncApi Accessor v2 module.
 
-## AsyncApi Importer - CLI
+## AsyncApi Importer CLI
 This module provides a wrapper to call the AsyncApi Importer Core module from a command line. Building the project successfully will produce an executable jar in the module target/ directory.
 Details for how to invoke the CLI can be found in the module's
 [Readme.md](https://github.com/dennis-brinley/sol-ep-asyncapi-importer/blob/main/asyncapi-importer-cli/Readme.md) file.
+- References the AsyncApi Importer Core module
+
+# Issues and Enhancements
+1. Add ability to specify new objects as Shared upon creation

asyncapi-importer-cli/src/main/java/com/solace/ep/asyncapi/cli/AsyncApiImport.java

@@ -32,6 +32,9 @@
 
 import com.solace.ep.asyncapi.importer.AsyncApiImporter;
 
+/**
+ * Class to wrap AsyncApi import to call from a command line.
+ */
 public class AsyncApiImport {
 
     private static final String CMD_LINE_SYNTAX = "asyncapi-import -a ASYNCAPI_TO_IMPORT -d APP_DOMAIN -t EP_TOKEN [-u BASE_URL] [-m | -i | -p] [-e] [-z]\n";
@@ -153,9 +156,6 @@ private static boolean checkForHelpOption( String[] args )
         try {
             CommandLine commandLine = cliParser.parse(options, args, false);
             hasHelpOption = commandLine.hasOption("h");
-            // if (hasHelpOption) {
-            //     displayHelp(options);
-            // }
         } catch (ParseException parseException) {
             return false;
         }
@@ -168,6 +168,12 @@ private static void displayHelp( final Options options )
         helpFormatter.printHelp(CMD_LINE_SYNTAX, options);
     }
 
+    /**
+     * Read an entire file from disk into a String variable
+     * @param fileName
+     * @return
+     * @throws Exception
+     */
     public static String getFileAsString(final String fileName) throws Exception 
     {
         Path path = Paths.get( fileName );

asyncapi-importer-core/Readme.md

@@ -1,4 +1,64 @@
 # AsyncApi Importer Core
 This module provides the core functionality to import AsyncApi spec files into Event Portal.
 
+## Build and Execute
+
+
 ## Design (to be completed)
+
+
+
+## Field Mapping
+
+
+### Standard AsyncAPI
+In a *Standard* AsyncAPI specification, certain conventions are followed. There is a **components** section that contains **schemas** and **messages**. Message payloads reference schemas. And **channels** reference **messages**. See [AsyncApi](https://www.asyncapi.com/) for more details. The table below defines how fields are mapped from AsyncAPI specifications conforming to these practices.
+
+|Event Portal Field|Source Field Description|AsyncAPI Spec Path|
+|---|---|---|
+|Enumeration Name|Parameter ID in Channel Defintion|$.channels.[channel_name].parameters.[paramId]|
+|Enumeration Version Values|Values Array of `schema.enum` Field in Channel Defintion|$.channels.[channel_name].parameters.[paramId].schema.enum|
+|Schema Name|Schema Component ID|$.components.schemas.[schemaId]|
+|Schema Version Content|Serialized Schema Payload ⦿|$.components.schemas.[schemaId].[object value]|
+|Event Name|Message Component ID|$components.messages.[messageId]|
+|Event Version Topic Address|Channel Name|$.channels.[channel_name]|
+|Event Version Schema|Schema Payload → Message Component → Channel||
+
+
+> ⦿ Both schemas found in Event Portal and in the AsyncAPI spec will be re-serialized for comparision. This step is performed to prevent non-material differences (whitespace, formatting) from resulting in differences between versions.
+
+
+### Enumerations
+
+Consider the following channel. The channel name is `importer/customer/updated/{customerId}/{IMP_regionId}`. The channel contains two parameters: `customerId` and `regionId`. **customerId** is an unbound parameter, meaning that it can effectively take on any value (there are some constraints). Whereas the **regionId** is bound to a set of discrete values: `CA-EAST`, `CA-WEST`, `USA-WEST`, etc. These values will be mapped into **Event Portal** as an Enumeration.
+
+```yaml
+channels:
+  importer/customer/updated/{customerId}/{regionId}:
+    subscribe:
+      message:
+        $ref: "#/components/messages/CustomerUpdated"
+    parameters:
+      customerId:
+        schema:
+          type: "string"
+      regionId:
+        schema:
+          type: "string"
+          enum:
+          - "CA-EAST"
+          - "CA-WEST"
+          - "USA-WEST"
+          - "USA-EAST"
+          - "EU"
+          - "SA"
+          - "AP
+```
+
+Details:
+- Upon import, an Enumeration with the parameter name will be created in Event Portal.
+    - Example above: **regionId**
+- Each distinct value set imported for an enumeration will result in a new version
+
+### Schemas
+

asyncapi-importer-core/pom.xml

@@ -14,7 +14,7 @@
     <version>0.1.0</version>
     <packaging>jar</packaging>
 
-    <name>AsyncApi Importer</name>
+    <name>AsyncApi Importer Core</name>
     <description>
         The purpose of this module is to provide the core capability
         to import AsyncApi spec files into Event Portal. This core

asyncapi-importer-core/src/main/java/com/solace/ep/asyncapi/importer/AsyncApiImporter.java

@@ -19,11 +19,15 @@
 
 import com.solace.ep.asyncapi.accessor.v2.AsyncApiAccessor;
 import com.solace.ep.asyncapi.importer.client.EventPortalClientApi;
-import com.solace.ep.asyncapi.importer.mapper.AsyncApiToDto;
+import com.solace.ep.asyncapi.importer.mapper.AsyncApiV2ToDto;
 import com.solace.ep.asyncapi.importer.model.dto.DtoResultSet;
 
 import lombok.extern.slf4j.Slf4j;
 
+/**
+ * Class to choreograph and execute AsyncApi import into Event Portal.
+ * Can instantiate the object and execute or invoke statically.
+ */
 @Slf4j
 public class AsyncApiImporter {
 
@@ -173,7 +177,7 @@ public void execImportOperation() throws Exception
         );
         this.applicationDomainId = importClient.getAppDomainId();
 
-        final AsyncApiToDto asyncApiToDtoMapper = new AsyncApiToDto(
+        final AsyncApiV2ToDto asyncApiToDtoMapper = new AsyncApiV2ToDto(
             asyncApiAccessor, 
             applicationDomainId, 
             applicationDomainName

asyncapi-importer-core/src/main/java/com/solace/ep/asyncapi/importer/EpImportOperator.java

@@ -355,15 +355,22 @@ private void importApplicationObject(
             appDto.setApplicationType(epApp.getApplicationType());
             appDto.setBrokerType(epApp.getBrokerType().getValue());
         }
+        for (ApplicationVersionDto appVersionDto : appDto.getApplicationVersions()) {
+            if (appVersionDto.isLastestVersionFound()) {
+                appDto.setLastestApplicationSemVer(appVersionDto.getEpApplicationVersion().getVersion());
+                break;
+            }
+        }
         for (ApplicationVersionDto appVersionDto : appDto.getApplicationVersions()) {
             appVersionDto.setApplicationId(appDto.getId());
-            importApplicationVersion(appVersionDto, appName);
+            importApplicationVersion(appVersionDto, appName, appDto);
         }
     }
 
     private void importApplicationVersion(
         final ApplicationVersionDto appVersionDto,
-        final String appName
+        final String appName,
+        final ApplicationDto appDto
     ) throws Exception
     {
         ApplicationVersion epAppVersion;
@@ -375,8 +382,9 @@ private void importApplicationVersion(
                     appVersionDto.getApplicationId(), 
                     appName, 
                     appVersionDto.getDeclaredProducedEventVersionIds(), 
-                    appVersionDto.isLastestVersionFound() ? appVersionDto.getEpApplicationVersion().getVersion() : null
+                    appVersionDto.isLastestVersionFound() ? appDto.getLastestApplicationSemVer() : null
                 );
+                appDto.setLastestApplicationSemVer(epAppVersion.getVersion());
                 log.info("CREATED New Application Version: [{}] v{} in Event Portal", appName, epAppVersion.getVersion());
             } else {
                 epAppVersion = epClient.updateApplicationVersion(
@@ -413,15 +421,22 @@ private void importSchemaObject(
             schemaDto.setMatchFound(true);
             schemaDto.setShared(schemaObject.getShared());
         }
+        for (SchemaVersionDto svDto : schemaDto.getSchemaVersions()) {
+            if (svDto.isLastestVersionFound()) {
+                schemaDto.setLastestSchemaSemVer(svDto.getEpSchemaVersion().getVersion());
+                break;
+            }
+        }
         for (SchemaVersionDto svDto : schemaDto.getSchemaVersions()) {
             svDto.setSchemaId(schemaDto.getId());
-            importSchemaVersion(svDto, schemaName);
+            importSchemaVersion(svDto, schemaName, schemaDto);
         };
     }
 
     private void importSchemaVersion(
         final SchemaVersionDto svDto,
-        final String schemaName
+        final String schemaName,
+        final SchemaDto schemaDto
     ) throws Exception
     {
         SchemaVersion sv;
@@ -433,8 +448,9 @@ private void importSchemaVersion(
                     svDto.getSchemaId(), 
                     svDto.getContent(), 
                     schemaName, 
-                    svDto.isLastestVersionFound() ? svDto.getEpSchemaVersion().getVersion() : null
+                    svDto.isLastestVersionFound() ? schemaDto.getLastestSchemaSemVer() : null
                 );
+                schemaDto.setLastestSchemaSemVer(sv.getVersion());
                 // For cascade object creation
                 // If creating new schemaVersion to update an existing version, then
                 // store the old and new version for retrieval
@@ -476,35 +492,48 @@ private void importEnumObject(
             enumDto.setMatchFound(true);
             enumDto.setShared(topicAddressEnum.getShared());
         }
+
+        for (EnumVersionDto evDto : enumDto.getEnumVersions()) {
+            if (evDto.isLastestVersionFound()) {
+                enumDto.setLastestEnumSemVer(evDto.getEpEnumVersion().getVersion());
+                break;
+            }
+        }
+
         for (EnumVersionDto evDto : enumDto.getEnumVersions()) {
             evDto.setEnumId(enumDto.getId());
-            importEnumVersion(evDto, enumName);
+            importEnumVersion(evDto, enumName, enumDto);
         }
     }
 
     private void importEnumVersion(
         final EnumVersionDto evDto,
-        final String enumName
+        final String enumName,
+        final EnumDto enumDto
     ) throws Exception
     {
         TopicAddressEnumVersion ev;
         if (evDto.isMatchFound()) {
             ev = evDto.getEpEnumVersion();
         } else {
             if (!evDto.isLastestVersionFound() || (evDto.isLastestVersionFound() && !evDto.isLatestVersionFoundInDraftState())) {
+
                 ev = epClient.createEnumVersion(
                     evDto.getEnumId(), 
                     evDto.getValuesAsStringList(), 
                     enumName, 
-                    evDto.isLastestVersionFound() ? evDto.getEpEnumVersion().getVersion() : null
+                    evDto.isLastestVersionFound() ? enumDto.getLastestEnumSemVer() : null
                 );
+                enumDto.setLastestEnumSemVer(ev.getVersion());
+
                 // For cascade object creation
                 // If creating new enumVersion to update an existing version, then
                 // store the old and new version for retrieval
                 if ( evDto.isLastestVersionFound() && ! evDto.isLatestVersionFoundInDraftState() ) {
                     incrementedEnumVersions.put( evDto.getEpEnumVersion().getId(), ev.getId() );
                 }
                 log.info("CREATED Enum Version: [{}] v{} in Event Portal", enumName, ev.getVersion());
+
             } else {
                 ev = epClient.updateEnumVersion(
                     evDto.getEpEnumVersion().getId(),
@@ -539,15 +568,22 @@ private void importEventObject(
             eventDto.setMatchFound(true);
             eventDto.setShared(epEvent.getShared());
         }
+        for (EventVersionDto eventVersionDto : eventDto.getEventVersions()) {
+            if (eventVersionDto.isLastestVersionFound()) {
+                eventDto.setLatestEventSemVer(eventVersionDto.getEpEventVersion().getVersion());
+                break;
+            }
+        }
         for (EventVersionDto eventVersionDto : eventDto.getEventVersions()) {
             eventVersionDto.setEventId(eventDto.getId());
-            importEventVersion(eventVersionDto, eventName);
+            importEventVersion(eventVersionDto, eventName, eventDto);
         }
     }
 
     private void importEventVersion(
         final EventVersionDto eventVersionDto,
-        final String eventName
+        final String eventName,
+        final EventDto eventDto
     ) throws Exception
     {
         EventVersion epEventVersion;
@@ -560,8 +596,9 @@ private void importEventVersion(
                     eventName,
                     eventVersionDto.getSchemaVersionId(),
                     eventVersionDto.getDeliveryDescriptor(),
-                    eventVersionDto.isLastestVersionFound() ? eventVersionDto.getEpEventVersion().getVersion() : null
+                    eventVersionDto.isLastestVersionFound() ? eventDto.getLatestEventSemVer() : null
                 );
+                eventDto.setLatestEventSemVer(epEventVersion.getVersion());
                 // For cascade object creation
                 // If creating new event version to update an existing version, then
                 // store the old and new version for retrieval