Skip to content

Commit 316f2b1

Browse files
Introduced EiffelTestCaseTriggeredEvent and EiffelTestCaseCanceledEvent (#134)
As per issue #120. Introduced two new events - EiffelTestCaseTriggeredEvent and EiffelTestCaseCanceledEvent - to bring test case execution representation in line with activity execution. The introduction of these additional events enables expression of concepts which were previously inexpressible, such as planned test scope and queuing times. Documentatin, schemas, examples and reference data updated accordingly.
1 parent 935cf3d commit 316f2b1

File tree

25 files changed

+810
-187
lines changed

25 files changed

+810
-187
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ __IMPORTANT NOTICE:__ The contents of this repository currectly reflect a __DRAF
5050
1. [EiffelSourceChangeCreatedEvent (SCC)](./eiffel-vocabulary/EiffelSourceChangeCreatedEvent.md)
5151
1. [EiffelSourceChangeSubmittedEvent (SCS)](./eiffel-vocabulary/EiffelSourceChangeSubmittedEvent.md)
5252
1. [EiffelFlowContextDefinedEvent (FCD)](./eiffel-vocabulary/EiffelFlowContextDefinedEvent.md)
53+
1. [EiffelTestCaseTriggeredEvent (TCT)](./eiffel-vocabulary/EiffelTestCaseTriggeredEvent.md)
54+
1. [EiffelTestCaseCanceledEvent (TCC)](./eiffel-vocabulary/EiffelTestCaseCanceledEvent.md)
5355
1. [EiffelTestCaseStartedEvent (TCS)](./eiffel-vocabulary/EiffelTestCaseStartedEvent.md)
5456
1. [EiffelTestCaseFinishedEvent (TCF)](./eiffel-vocabulary/EiffelTestCaseFinishedEvent.md)
5557
1. [EiffelTestSuiteStartedEvent (TSS)](./eiffel-vocabulary/EiffelTestSuiteStartedEvent.md)

eiffel-syntax-and-usage/the-links-object.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -149,20 +149,20 @@ __Multiple allowed:__ No
149149
__Description:__ Identifies the relevant test suite execution. In other words, [EiffelTestSuiteStartedEvent](../eiffel-vocabulary/EiffelTestSuiteStartedEvent.md) acts as a handle for a particular test suite execution.
150150

151151
### TEST_CASE_EXECUTION
152-
__Required in:__ [EiffelTestCaseFinishedEvent](../eiffel-vocabulary/EiffelTestCaseFinishedEvent.md)
152+
__Required in:__ [EiffelTestCaseStartedEvent](../eiffel-vocabulary/EiffelTestCaseStartedEvent.md), [EiffelTestCaseFinishedEvent](../eiffel-vocabulary/EiffelTestCaseFinishedEvent.md), [EiffelTestCaseCanceledEvent](../eiffel-vocabulary/EiffelTestCaseCanceledEvent.md)
153153
__Optional in:__ None
154-
__Legal targets:__ [EiffelTestCaseStartedEvent](../eiffel-vocabulary/EiffelTestCaseStartedEvent.md)
154+
__Legal targets:__ [EiffelTestCaseTriggeredEvent](../eiffel-vocabulary/EiffelTestCaseTriggeredEvent.md)
155155
__Multiple allowed:__ No
156-
__Description:__ Identifies the relevant test case execution. In other words, [EiffelTestCaseStartedEvent](../eiffel-vocabulary/EiffelTestCaseStartedEvent.md) acts as a handle for a particular test case execution.
156+
__Description:__ Identifies the relevant test case execution. In other words, [EiffelTestCaseTriggeredEvent](../eiffel-vocabulary/EiffelTestCaseTriggeredEvent.md) acts as a handle for a particular test case execution. This differs from __CONTEXT__. In __TEST_CASE_EXECUTION__ the source carries information pertaining to the target (i.e. the test case execution started, finished or was canceled). In __CONTEXT__, on the other hand, the source constitutes a subset of the target (e.g. this test case was executed as part of that activity or test suite).
157157

158158
### IUT
159-
__Required in:__ [EiffelTestCaseStartedEvent](../eiffel-vocabulary/EiffelTestCaseStartedEvent.md),
159+
__Required in:__ [EiffelTestCaseTriggeredEvent](../eiffel-vocabulary/EiffelTestCaseTriggeredEvent.md),
160160
[EiffelIssueVerifiedEvent](../eiffel-vocabulary/EiffelIssueVerifiedEvent.md)
161161
__Optional in:__ None
162162
__Legal targets:__ [EiffelArtifactCreatedEvent](../eiffel-vocabulary/EiffelArtifactCreatedEvent.md),
163163
[EiffelCompositionDefinedEvent](../eiffel-vocabulary/EiffelCompositionDefinedEvent.md)
164164
__Multiple allowed:__ No
165-
__Description:__ Identifies the Item Under Test: what has been tested and/or been verified to address an issue.
165+
__Description:__ Identifies the Item Under Test: what is about to be tested and/or has been verified to address an issue, respectively.
166166

167167
### TERC
168168
__Required in:__ None

eiffel-vocabulary/EiffelActivityTriggeredEvent.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ __Description:__ A description of the trigger.
5656
__Type:__ String
5757
__Required:__ No
5858
__Legal values:__ MANUAL, SEMI_AUTOMATED, AUTOMATED, OTHER
59-
__Description:__ The type of execution (often related to, but ultimately separate from, __data.trigger.type__).
59+
__Description:__ The type of execution (often related to, but ultimately separate from, __data.triggers.type__).
6060

6161
## Version History
6262
| Version | Introducing Commit |
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
<!---
2+
Copyright 2017 Ericsson AB.
3+
For a full list of individual contributors, please see the commit history.
4+
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
--->
17+
18+
# EiffelTestCaseCanceledEvent (TCC)
19+
The EiffelTestCaseCanceledEvent declares that a previously triggered test case execution (represented by [EiffelTestCaseTriggeredEvent](./EiffelTestCaseTriggeredEvent.md)) has been canceled _before it has started_. This is typically used in queuing situations where a queued execution is dequeued. It is recommended that __CAUSE__ links be used to indicate the reason.
20+
21+
## Data Members
22+
### data.reason
23+
__Type:__ String
24+
__Required:__ No
25+
__Description:__ Any human readable information as to the reason for dequeueing.
26+
27+
## Version History
28+
| Version | Introducing Commit |
29+
| --------- | ------------------ |
30+
| 1.0.0 | _Current version_ |
31+
32+
## Examples
33+
* [Simple example](../examples/events/EiffelTestCaseCanceledEvent/simple.json)

eiffel-vocabulary/EiffelTestCaseStartedEvent.md

Lines changed: 1 addition & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -16,65 +16,14 @@
1616
--->
1717

1818
# EiffelTestCaseStartedEvent (TCS)
19-
The EiffelTestCaseStartedEvent declares that the execution of a test case has commenced. This can either be declared stand-alone or as part of an activity or test suite, using either a __CAUSE__ or a __CONTEXT__ link type, respectively.
19+
The EiffelTestCaseStartedEvent declares that the execution of a test case has commenced. This event SHALL be preceded by a [EiffelTestCaseTriggeredEvent](./EiffelTestCaseTriggeredEvent.md), and appropriately linked to via __TEST_CASE_EXECUTION__.
2020

2121
## Data Members
22-
### data.testCase
23-
__Type:__ Object
24-
__Required:__ Yes
25-
__Description:__ Identification of the executed test case.
26-
27-
#### data.testCase.tracker
28-
__Type:__ String
29-
__Required:__ No
30-
__Description:__ The name of the test case tracker - typically a test management system.
31-
32-
#### data.testCase.id
33-
__Type:__ String
34-
__Required:__ Yes
35-
__Description:__ The unique identity of the executed test case.
36-
37-
#### data.testCase.version
38-
__Type:__ String
39-
__Required:__ No
40-
__Description:__ The unique version of the executed test case identity. Where this property is not used it is assumed that test cases are not version controlled.
41-
42-
#### data.testCase.uri
43-
__Type:__ String
44-
__Required:__ No
45-
__Description:__ A location where a description of the test case can be retrieved. To the extent that multiple versions of the same test case co-exist, this property SHALL identify the exact version executed.
46-
4722
### data.executor
4823
__Type:__ String
4924
__Required:__ No
5025
__Description:__ The name of the test case executor, if applicable. This property can be used to identify tests executed by a particular test framework.
5126

52-
### data.recipeId
53-
__Type:__ String
54-
__Required:__ No
55-
__Description:__ If this test case execution was the result of an Execution Recipe, as defined by an [EiffelTestExecutionRecipeCollectionCreatedEvent](./EiffelTestExecutionRecipeCollectionCreatedEvent.md), this UUID SHALL match the relevant __data.batches.recipes.id__ in that event.
56-
57-
### data.parameters
58-
__Type:__ Object[]
59-
__Required:__ No
60-
__Description:__ A list of parameters passed to the test case execution.
61-
62-
#### data.parameters.name
63-
__Type:__ String
64-
__Required:__ Yes
65-
__Description:__ The name of the parameter.
66-
67-
#### data.parameters.value
68-
__Type:__ String
69-
__Required:__ Yes
70-
__Description:__ The value of the parameter.
71-
72-
### data.executionType
73-
__Type:__ String
74-
__Required:__ No
75-
__Legal values:__ MANUAL, SEMI_AUTOMATED, AUTOMATED, OTHER
76-
__Description:__ The type of test case execution.
77-
7827
### data.liveLogs
7928
__Type:__ Object[]
8029
__Required:__ No
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
<!---
2+
Copyright 2017 Ericsson AB.
3+
For a full list of individual contributors, please see the commit history.
4+
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
--->
17+
18+
# EiffelTestCaseTriggeredEvent (TCT)
19+
The EiffelTestCaseTriggeredEvent declares that the execution of a test case has been triggered, but not yet started. This can either be declared stand-alone or as part of an activity or test suite, using either a __CAUSE__ or a __CONTEXT__ link type, respectively.
20+
21+
This event is used to communicate intent, and thereby serves a similar purpose to that of [EiffelActivityTriggeredEvent](./EiffelActivityTriggeredEvent.md). A triggered test case execution is expected to either be started (represented by [EiffelTestCaseStartedEvent](./EiffelTestCaseStartedEvent.md)) or canceled (represented by [EiffelTestCaseCanceledEvent](./EiffelTestCaseCanceledEvent.md)). Consequently, any delay between triggering and execution can be assumed to imply queuing time (e.g. waiting for available test resources) and monitored as such.
22+
23+
## Data Members
24+
### data.testCase
25+
__Type:__ Object
26+
__Required:__ Yes
27+
__Description:__ Identification of the test case to be executed.
28+
29+
#### data.testCase.tracker
30+
__Type:__ String
31+
__Required:__ No
32+
__Description:__ The name of the test case tracker - typically a test management system.
33+
34+
#### data.testCase.id
35+
__Type:__ String
36+
__Required:__ Yes
37+
__Description:__ The unique identity of the test case to be executed.
38+
39+
#### data.testCase.version
40+
__Type:__ String
41+
__Required:__ No
42+
__Description:__ The unique version of the identified test case to be executed. Where this property is not used it is assumed that test cases are not version controlled.
43+
44+
#### data.testCase.uri
45+
__Type:__ String
46+
__Required:__ No
47+
__Description:__ A location where a description of the test case can be retrieved. To the extent that multiple versions of the same test case co-exist, this property SHALL identify the exact version to be executed.
48+
49+
### data.recipeId
50+
__Type:__ String
51+
__Required:__ No
52+
__Description:__ If triggering this test case execution was the result of an Execution Recipe, as defined by an [EiffelTestExecutionRecipeCollectionCreatedEvent](./EiffelTestExecutionRecipeCollectionCreatedEvent.md), this UUID SHALL match the relevant __data.batches.recipes.id__ in that event.
53+
54+
### data.triggers
55+
__Type:__ Object[]
56+
__Required:__ No
57+
__Description:__ The circumstances triggering the test case execution.
58+
59+
#### data.triggers.type
60+
__Type:__ String
61+
__Required:__ Yes
62+
__Legal values:__ MANUAL, EIFFEL_EVENT, SOURCE_CHANGE, TIMER, OTHER
63+
__Description:__ The type of trigger.
64+
MANUAL signifies that the test case execution was manually triggered.
65+
EIFFEL_EVENT signifies that the test case execution was triggered by one or more Eiffel events. These events should be represented via __CAUSE__ links.
66+
SOURCE_CHANGE signifies that the test case execution was triggered as a consequence of a detected source change __not__ represented by Eiffel events.
67+
TIMER signifies that the test case execution was triggered by a timer.
68+
OTHER signifies any other triggering cause.
69+
70+
#### data.triggers.description
71+
__Type:__ String
72+
__Required:__ No
73+
__Description:__ A description of the trigger.
74+
75+
### data.executionType
76+
__Type:__ String
77+
__Required:__ No
78+
__Legal values:__ MANUAL, SEMI_AUTOMATED, AUTOMATED, OTHER
79+
__Description:__ The type of execution (often related to, but ultimately separate from, __data.triggers.type__).
80+
81+
### data.parameters
82+
__Type:__ Object[]
83+
__Required:__ No
84+
__Description:__ A list of parameters to be passed to the test case execution.
85+
86+
#### data.parameters.name
87+
__Type:__ String
88+
__Required:__ Yes
89+
__Description:__ The name of the parameter.
90+
91+
#### data.parameters.value
92+
__Type:__ String
93+
__Required:__ Yes
94+
__Description:__ The value of the parameter.
95+
96+
## Version History
97+
| Version | Introducing Commit |
98+
| --------- | ------------------ |
99+
| 1.0.0 | _Current version_ |
100+
101+
## Examples
102+
* [Simple example](../examples/events/EiffelTestCaseTriggeredEvent/simple.json)
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"meta": {
3+
"type": "EiffelTestCaseCanceledEvent",
4+
"version": "1.0.0",
5+
"time": 1234567890,
6+
"id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee0"
7+
},
8+
"data": {
9+
"reason": "Skipping this test execution due to new IUT version becoming available."
10+
},
11+
"links": [
12+
{
13+
"type": "CAUSE",
14+
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee1"
15+
},
16+
{
17+
"type": "TEST_CASE_EXECUTION",
18+
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee2"
19+
}
20+
]
21+
}

examples/events/EiffelTestCaseStartedEvent/simple.json

Lines changed: 1 addition & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -6,21 +6,7 @@
66
"id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee0"
77
},
88
"data": {
9-
"testCase": {
10-
"tracker": "My Test Management System",
11-
"id": "TC-1234",
12-
"version": "42",
13-
"uri": "http://tm.company.com/browse/TC-1234/42"
14-
},
159
"executor": "My Test Framework",
16-
"recipeId": "aaaaaaaa-bbbb-5ccc-addd-eeeeeeeeeee0",
17-
"parameters": [
18-
{
19-
"name": "login",
20-
"value": "admin"
21-
}
22-
],
23-
"executionType": "AUTOMATED",
2410
"liveLogs": [
2511
{
2612
"name": "My test log",
@@ -29,21 +15,9 @@
2915
]
3016
},
3117
"links": [
32-
{
33-
"type": "CONTEXT",
34-
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee1"
35-
},
36-
{
37-
"type": "IUT",
38-
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee2"
39-
},
4018
{
4119
"type": "ENVIRONMENT",
42-
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee3"
43-
},
44-
{
45-
"type": "TERC",
46-
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee4"
20+
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee1"
4721
}
4822
]
4923
}
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
{
2+
"meta": {
3+
"type": "EiffelTestCaseTriggeredEvent",
4+
"version": "1.0.0",
5+
"time": 1234567890,
6+
"id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee0",
7+
"security": {
8+
"sdm": {
9+
"authorIdentity": "My Test Management System",
10+
"encryptedDigest": "a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a"
11+
}
12+
}
13+
},
14+
"data": {
15+
"testCase": {
16+
"tracker": "My Test Management System",
17+
"id": "TC-1234",
18+
"version": "42",
19+
"uri": "http://tm.company.com/browse/TC-1234/42"
20+
},
21+
"recipeId": "aaaaaaaa-bbbb-5ccc-addd-eeeeeeeeeee0",
22+
"parameters": [
23+
{
24+
"name": "login",
25+
"value": "admin"
26+
}
27+
],
28+
"executionType": "AUTOMATED",
29+
"triggers": [
30+
{
31+
"type": "TIMER",
32+
"description": "Nightly test timer."
33+
}
34+
]
35+
},
36+
"links": [
37+
{
38+
"type": "CONTEXT",
39+
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee1"
40+
},
41+
{
42+
"type": "IUT",
43+
"target": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeee2"
44+
}
45+
]
46+
}

0 commit comments

Comments
 (0)