Fixes #21

Author: adamfowleruk

data/restapi.json

@@ -6,7 +6,8 @@
     {"name": "processinbox", "methods": ["GET"]},
     {"name": "processqueue", "methods": ["GET"]},
     {"name": "processmodel", "methods": ["GET","POST","PUT"]},
-    {"name": "processsubscription", "methods": ["PUT"]}
+    {"name": "processsubscription", "methods": ["PUT"]},
+    {"name": "processsearch", "methods": ["PUT"]}
   ],
   "transforms": [],
   "triggers": [

documentation/DEVELOPER.md

@@ -2,6 +2,6 @@ Here is the developer documentation
 
 - [Adding a new Activity type](DEV-NEWACTIVITY.md)
 - [Creating a custom step processor](DEV-STEPPROC.md)
-- [Integrating your app to MarkLogic Workflow](./RESTAPI.md)
+- [Integrating your app to MarkLogic Workflow](RESTAPI.md)
 
 Any issues contact me at [email protected]

documentation/RESTAPI.md

@@ -1,6 +1,6 @@
 
 
-See also MODELLING.md for the REST API to install a MarkLogic workflow process model.
+See also the [Modelling Overview](MODELLING.md) for the REST API to install a MarkLogic workflow process model.
 
 
 ## How the workflow engine works

documentation/SPRINTS.md

@@ -34,6 +34,8 @@ steps if a duplicate is found. Requires search (by property hash AND not (same u
 
 ## Sprint 1 - Basic workflow
 
+Completed Sat 18 Apr 2015 14:30 BST by Adam Fowler
+
 - DONE BPMN2: generic blank task
 - DONE BPMN2: user task -> aka human step
 - DONE BPMN2: exclusive gateway -> Decision point with one outcome, multiple options
@@ -50,8 +52,8 @@ steps if a duplicate is found. Requires search (by property hash AND not (same u
 - DONE Tools: Process Data model XSD (for modeler import)
 - DONE Tools: Eclipse BPMN 2 Modeler Palette and Process diagram support, including new diagram creation for MarkLogic
 - DEFERRED UI: Ridiculously basic HTML widget in MLJS for rendering step and choosing action (for ease of testing)
-- TEST Start process using an Alert (content subscription)
-- TEST REST API: Basic process initiation, update and tracking methods
+- DONE Start process using an Alert (content subscription)
+- DONE REST API: Basic process initiation, update and tracking methods
  - DONE processmodel.xqy
   - DONE PUT create and publish process model, accepting BPMN2 content type .bpmn2, and to update process model without publishing
   - DONE GET to fetch process model
@@ -60,8 +62,8 @@ steps if a duplicate is found. Requires search (by property hash AND not (same u
   - DONE PUT create instance of a process (starts a process)
   - DONE POST complete a human task
   - DONE GET fetch the current state of a business process
- - TEST processsubscription.xqy
-  - TEST PUT create a process subscription (alert) to create a new process instance (creating a content doc creates a process doc with an initiating attachment)
+ - DONE processsubscription.xqy
+  - DONE PUT create a process subscription (alert) to create a new process instance (creating a content doc creates a process doc with an initiating attachment)
  - DONE processinbox.xqy
  - DONE processqueue.xqy
  - TEST support for roles (processroleinbox.xqy) on user tasks (For BD)

documentation/USING-SUBSCRIPTIONS.md

@@ -21,8 +21,8 @@ The above is why I decided to have a process document that CPF runs against, rat
 ## How it works
 
 1. A user, or application, creates a new document in MarkLogic - say a Bank Account Opening E-form
-2. A 'Process Subscription' (basically a MarkLogic alert that uses the alert-action-process.xqy action) fires, creating a new Account Opening process document
-3. A CPF Domain (set up by enabling/publishing a MarkLogic Workflow BPMN2 model) fires for this document, invoking the Account Opening V1.2 CPF pipeline process
+2. A 'Process Subscription' (basically a MarkLogic alert that uses the alert-action-process.xqy action) fires, creating a new Account Opening process document, and attaching the account opening form as the InitiatingAttachment attachment
+3. A CPF Domain (set up by enabling/publishing a MarkLogic Workflow BPMN2 model) fires for this process document, invoking the Account Opening V1.2 CPF pipeline process
 4. CPF manages the state transitions and execution of BPMN2 CPF actions throughout the process lifecycle
 5. Process eventually completes (or fails), leaving the process document as an audit record of what happened, with metrics for performance analysis
 
@@ -39,13 +39,19 @@ Several reasons.
 ## Isn't it a bit convoluted?
 
 If you had to create all these things by hand, yes. This is why the Workflow REST API provides a single endpoint
-(POST /v1/resource/process) to take a BPMN2 model, create a set of CPF pipelines, and create a
+(POST /v1/resources/process) to take a BPMN2 model, create a set of CPF pipelines, and create a
 domain configuration, for you.
 
 All you then need to do then is separately configure one or more Alerts (aka Process Subscriptions) via calls to
-PUT /v1/resource/processsubscription to start the relevant workflow based on criteria about a new or updated document
+PUT /v1/resources/processsubscription to start the relevant workflow based on criteria about a new or updated document
 in marklogic.
 
-Alternatively, manually start a process via PUT /v1/resource/process without needing a Process Subscription. This is useful
+Alternatively, manually start a process via PUT /v1/resources/process without needing a Process Subscription. This is useful
 if starting a process via an ESB or application. A good example is starting a process to tell a person to create a
 new document that doesn't exist yet.
+
+## Examples
+
+For examples of how to configure and call the REST extensions for alerting, see the /shtests/ folder, specifically:-
+- Automated start: 25-payload.xml and 25-processsubscription-create.sh
+- Manual start: 06-payload.xml 06-process-create.sh

documentation/USING.md

@@ -7,4 +7,4 @@ This section of the documentation is very much a work in progress. See the [REST
 details right now.
 
 - [Creating a Process Subscription](USING-SUBSCRIPTIONS.md)
-- Starting a process without a content document
+- Starting a process without a content document - see the above link, last paragraph

modules/app/models/alert-action-process.xqy

@@ -21,13 +21,12 @@ declare variable $alert:action as element(alert:action) external;
 (: Create a process instance for this document :)
 
 (: Find appropriate process from alert action option :)
-let $procname := $alert-action/alert:options/wf:process-name/text()
-return
-  wfu:create($procname,$alert:doc/element(),
-    <attachment name="InitiatingAttachment" cardinality="1">
-      <uri>{fn:base-uri($alert:doc)}</uri>
-    </attachment>
+let $procname := xs:string($alert:action/alert:options/wf:process-name)
+let $pid :=   wfu:create($procname,$alert:doc/element(),
+      (<wf:attachment name="InitiatingAttachment" uri="{fn:base-uri($alert:doc)}" cardinality="1"/>)
   )
+return ()
+
 
 (:)
   xdmp:document-insert($procname || "/" || sem:uuid-string() || ".xml",

modules/app/models/lib-search-subscribe.xqy

@@ -252,20 +252,26 @@ declare function ss:create-rule-notify($alert-name as xs:string,$alert-detail as
 
 
 
-declare function ss:add-alert($shortname as xs:string,$query as element(cts:query),
+declare function ss:add-alert($shortname as xs:string,$query as cts:query,
   $ruleoptions as element()*,$module as xs:string,$moduledbname as xs:string?,$actionoptions as element()*) as xs:string {
 
   let $name := ss:do-create-config($shortname)
-  return
-    (
-      ss:do-create-rule($name,$query,$ruleoptions),
-      ss:do-create-action($name,$module,$moduledbname,$actionoptions),
-      $name
-    )
+  let $_ := (
+    ss:do-create-action($name,$module,$moduledbname,$actionoptions)
+    ,
+    ss:do-create-rule($name,$query,$ruleoptions)
+  )
+  return $name
+};
+
+declare function ss:get-alert($shortname as xs:string) as element(alert:config)? {
+  alert:config-get($shortname)
 };
 
 
 declare function ss:do-create-config($shortname as xs:string) as xs:string {
+  let $rem := ss:check-remove-config($shortname)
+  return
   xdmp:eval(
     'xquery version "1.0-ml"; declare namespace my="http://marklogic.com/alerts"; ' ||
     'import module namespace ah = "http://marklogic.com/search/subscribe" at "/app/models/lib-search-subscribe.xqy";' ||
@@ -277,16 +283,16 @@ declare function ss:do-create-config($shortname as xs:string) as xs:string {
   )
 };
 
-declare function ss:do-create-rule($alert-name as xs:string,$query as cts:query,$options as element()*) {
+declare function ss:do-create-rule($alert-name as xs:string,$query as cts:query,$options as element()*) as empty-sequence() {
   xdmp:eval(
     'xquery version "1.0-ml"; declare namespace my="http://marklogic.com/alerts"; ' ||
     'import module namespace ah = "http://marklogic.com/search/subscribe" at "/app/models/lib-search-subscribe.xqy";' ||
     'import module namespace alert="http://marklogic.com/xdmp/alert" at "/MarkLogic/alert.xqy";' ||
     'declare variable $my:alert-name as xs:string external;' ||
     'declare variable $my:query as cts:query external;' ||
-    'declare variable $my:options as element* external;' ||
-    'ah:create-rule($my:alert-name,$my:query,$my:options)',
-    (xs:QName("my:alert-name"),$alert-name,xs:QName("my:query"),$query,xs:QName("my:options"),$options),
+    (:)'declare variable $my:options as element()* external;' ||:)
+    'ah:create-rule($my:alert-name,$my:query,())',
+    (xs:QName("my:alert-name"),$alert-name,xs:QName("my:query"),$query (:,xs:QName("my:options"),($options) :) ),
     <options xmlns="xdmp:eval"><isolation>different-transaction</isolation></options>
   )
 };
@@ -299,13 +305,41 @@ declare function ss:do-create-action($alert-name as xs:string,$alert-module as x
     'declare variable $my:alert-name as xs:string external;' ||
     'declare variable $my:alert-module as xs:string external;' ||
     'declare variable $my:dbname as xs:string external;' ||
-    'declare variable $my:options as element* external;' ||
+    'declare variable $my:options as element()* external;' ||
     'ah:create-action($my:alert-name,$my:alert-module,$my:dbname,$my:options)',
-    (xs:QName("my:alert-name"),$alert-name,xs:QName("my:alert-module"),$alert-module,xs:QName("my:dbname"),$dbname,xs:QName("my:options"),$options),
+    (xs:QName("my:alert-name"),$alert-name,xs:QName("my:alert-module"),$alert-module,xs:QName("my:dbname"),$dbname,xs:QName("my:options"),($options)),
     <options xmlns="xdmp:eval"><isolation>different-transaction</isolation></options>
   )
 };
 
+declare function ss:check-remove-config($alert-name as xs:string) {
+  let $config := alert:config-get($alert-name)
+  return
+    if (fn:not(fn:empty($config))) then
+      (: Check if config used in a cpf domain, if so remove it from that domain :)
+      let $unreg := xdmp:eval(
+        'xquery version "1.0-ml"; declare namespace my="http://marklogic.com/alerts"; ' ||
+        'import module namespace alert="http://marklogic.com/xdmp/alert" at "/MarkLogic/alert.xqy";' ||
+        'declare variable $my:alert-name as xs:string external;' ||
+        'alert:config-insert(' ||
+        '  alert:config-set-cpf-domain-names(alert:config-get($my:alert-name),())' ||
+        ')',
+        (xs:QName("my:alert-name"),$alert-name),
+        <options xmlns="xdmp:eval"><isolation>different-transaction</isolation></options>
+      )
+      (: Do this for each domain - NA all done in one hit:)
+      (: Now remove the alert config :)
+      return xdmp:eval(
+        'xquery version "1.0-ml"; declare namespace my="http://marklogic.com/alerts"; ' ||
+        'import module namespace alert="http://marklogic.com/xdmp/alert" at "/MarkLogic/alert.xqy";' ||
+        'declare variable $my:alert-name as xs:string external;' ||
+        'alert:config-delete($my:alert-name)',
+        (xs:QName("my:alert-name"),$alert-name),
+        <options xmlns="xdmp:eval"><isolation>different-transaction</isolation></options>
+      )
+    else ()
+};
+
 declare function ss:create-config($shortname as xs:string) as xs:string {
   (: add alert :)
   let $alert-name := "/config/alerts/" || $shortname
@@ -318,7 +352,7 @@ declare function ss:create-config($shortname as xs:string) as xs:string {
   return $alert-name
 };
 
-declare function ss:create-rule($alert-name as xs:string,$query as cts:query,$options as element()*) {
+declare function ss:create-rule($alert-name as xs:string,$query as cts:query,$options as element()*) as empty-sequence() {
 
   let $rule := alert:make-rule(
       fn:concat($alert-name,"-rule"),
@@ -347,7 +381,6 @@ declare function ss:create-action($alert-name as xs:string,$alert-module as xs:s
 
 
 
-(: TODO add method for creating cpf domain, and enabling cpf on the database automatically - see 8001 admin code :)
 
 
 declare function ss:cpf-enable($alert-name as xs:string,$cpf-domain as xs:string) {
@@ -356,3 +389,77 @@ declare function ss:cpf-enable($alert-name as xs:string,$cpf-domain as xs:string
       alert:config-get($alert-name),
       ($cpf-domain)))
 };
+
+declare function ss:create-domain($domainname as xs:string,$domaintype as xs:string,$domainpath as xs:string,$domaindepth as xs:string,$pipeline-names as xs:string*,$modulesdb as xs:string) as xs:unsignedLong {
+
+    (: check if domain already exists and recreate :)
+    let $remove :=
+      try {
+        if (fn:not(fn:empty(
+          xdmp:eval(
+           'xquery version "1.0-ml";declare namespace m="http://marklogic.com/alerts"; import module namespace dom = "http://marklogic.com/cpf/domains" at "/MarkLogic/cpf/domains.xqy";declare variable $m:processmodeluri as xs:string external; dom:get($m:processmodeluri)'
+           ,
+            (xs:QName("my:processmodeluri"),$domainname),
+            <options xmlns="xdmp:eval">
+              <database>{xdmp:triggers-database()}</database>
+              <isolation>different-transaction</isolation>
+            </options>
+          )
+        ))) then
+          let $_ := xdmp:log(" GOT DOMAIN TO REMOVE")
+          return
+            xdmp:eval(
+              'xquery version "1.0-ml";declare namespace m="http://marklogic.com/alerts"; import module namespace dom = "http://marklogic.com/cpf/domains" at "/MarkLogic/cpf/domains.xqy";declare variable $m:processmodeluri as xs:string external;'
+              ||
+              'dom:remove($m:processmodeluri)'
+              ,
+              (xs:QName("my:processmodeluri"),$domainname),
+              <options xmlns="xdmp:eval">
+                <database>{xdmp:triggers-database()}</database>
+                <isolation>different-transaction</isolation>
+              </options>
+            )
+        else
+          ()
+      } catch ($e) { ( xdmp:log("Error trying to remove domain: " || $domainname),xdmp:log($e) ) } (: catching domain throwing error if it doesn't exist. We can safely ignore this :)
+
+
+       (: Create domain :)
+
+         (: Configure domain :)
+    return
+      xdmp:eval(
+        'xquery version "1.0-ml";declare namespace m="http://marklogic.com/alerts";' ||
+        'import module namespace p="http://marklogic.com/cpf/pipelines" at "/MarkLogic/cpf/pipelines.xqy"; '||
+        'import module namespace dom = "http://marklogic.com/cpf/domains" at "/MarkLogic/cpf/domains.xqy"; ' ||
+        'declare variable $m:pname as xs:string external;declare variable $m:pnames as xs:string* external;' ||
+        'declare variable $m:mdb as xs:unsignedLong external;'||
+        'declare variable $m:type as xs:string external;' ||
+        'declare variable $m:path as xs:string external;' ||
+        'declare variable $m:otherpipeline as xs:string external;' ||
+        'declare variable $m:depth as xs:string external;' ||
+        'let $_ := xdmp:log("In eval") ' ||
+        (:)'let $pids := for $pn in $m:pnames ' ||
+        '  return xs:unsignedLong(p:pipelines()[p:pipeline-name = $pn]/p:pipeline-id) ' || :)
+        'let $pids := ' ||
+        '(xs:unsignedLong(p:pipelines()[p:pipeline-name = "Status Change Handling"]/p:pipeline-id),xs:unsignedLong(p:pipelines()[p:pipeline-name = $m:otherpipeline]/p:pipeline-id))' ||
+        'let $_ := xdmp:log("second point")' ||
+        'let $ds := dom:domain-scope($m:type,$m:path,$m:depth) ' ||
+        'let $_ := xdmp:log("third point")' ||
+        'let $ec := dom:evaluation-context($m:mdb,"/")' ||
+        'let $_ := xdmp:log("fourth point")' ||
+        'let $dc := dom:create($m:pname,"Domain for "||$m:pname,' ||
+        '  $ds,$ec,$pids,())' ||
+        'let $_ := xdmp:log("fifth point")' ||
+        'return $dc'
+        ,
+        (xs:QName("my:otherpipeline"),($pipeline-names[2]),xs:QName("my:mdb"),xdmp:database($modulesdb),xs:QName("my:pname"),$domainname,
+         xs:QName("my:type"),$domaintype,xs:QName("my:path"),$domainpath,xs:QName("my:depth"),$domaindepth
+        ),
+        <options xmlns="xdmp:eval">
+          <database>{xdmp:triggers-database()}</database>
+          <isolation>different-transaction</isolation>
+        </options>
+      ) (: end eval :)
+
+};

modules/app/models/workflow-util.xqy

@@ -33,11 +33,28 @@ declare function m:create($pipelineName as xs:string,$data as element()*,$attach
   return $id
 };
 
+(:
+ : You must create one or more CPF domains for folders or collections that alerts can be evaluated within.
+ :)
+declare function m:createAlertingDomain($name as xs:string,$type as xs:string,$path as xs:string,$depth as xs:string) as xs:unsignedLong {
+  ss:create-domain($name,$type,$path,$depth,("Status Change Handling","Alerting"),xdmp:database-name(xdmp:modules-database()))
+};
+
 (:
  : Create a new process subscription
  :)
-declare function m:createSubscription($pipelineName as xs:string,$name as xs:string,$query as element(cts:query)) as xs:string {
-  ss:add-alert($name,$query,(),"/app/models/alert-action-process.xqy",(),(<wf:process-name>{$pipelineName}</wf:process-name>))
+declare function m:createSubscription($pipelineName as xs:string,$name as xs:string,$domainname as xs:string,$query as cts:query) as xs:string {
+  let $alert-uri := ss:add-alert($name,$query,(),"/app/models/alert-action-process.xqy",xdmp:database-name(xdmp:modules-database()),
+    (<wf:process-name>{$pipelineName}</wf:process-name>))
+  let $alert-enabled := ss:cpf-enable($alert-uri,$domainname)
+  return $alert-uri
+};
+
+(:
+ : Fetches a process subscription
+ :)
+declare function m:getSubscription($name as xs:string) as element()? {
+  ss:get-alert("/config/alerts/" || $name)
 };
 
 (:
@@ -151,7 +168,27 @@ declare function m:roleinbox($role as xs:string) as element(wf:queue) {
   </wf:queue>
 };
 
-
+(:
+ : Lists all processes, or all those with a specific PROCESS__MAJOR__MINOR name
+ :)
+declare function m:list($processName as xs:string?) as element(wf:list) {
+  <wf:list>
+  {
+    for $process in cts:search(fn:collection("http://marklogic.com/workflow/processes"),
+      cts:and-query(
+        if (fn:not(fn:empty($processName))) then
+          cts:element-attribute-value-query(xs:QName("wf:process"),xs:QName("title"),$processName)
+        else
+          cts:not-query(())
+      ),("unfiltered") (: TODO ordering, prioritisation support, and so on :)
+    )
+    return
+      <wf:listitem processid="{xs:string($process/wf:process/@id)}">
+        {$process}
+      </wf:listitem>
+  }
+  </wf:list>
+};