Date and time of the report: 2023-06-14 08:20:38.636532
Analysed period: 2023-05-24 to 2023-06-14.
Database update:
Retrieving from database...
DONE
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_result
is the outcome of the build, reported by Jenkins. It can be: SUCCESS
, FAILURE
, UNSTABLE
or ABORTED
.test_result
is the summary of the concerned Robot tests. It can be: FAIL
, UNAVAILABLE
or PASS
.Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
grp_build_result
.grp_test_result
.grp_success_fail
.For the identification of these sequences, the following mapping applies:
Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
DONE
DONE
job | Master branch | Release TEN | Release THIRTEEN | Release TWELVE | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
suites | ||||||||||||||||
Basic 01-Crud Operations On Vim Targets | 0.000000 | 20.113113 | 20.113113 | 0.000000 | 0.000000 | 19.415394 | 19.415394 | 0.000000 | 0.000000 | 20.624444 | 20.624444 | 0.000000 | 0.0 | 21.543171 | 21.543171 | 0.0 |
Basic 05-Instantiation Parameters In Cloud Init | 0.000000 | 20.113113 | 20.113113 | 0.000000 | 0.000000 | 19.415394 | 19.415394 | 0.000000 | 0.000000 | 20.624444 | 20.624444 | 0.000000 | 0.0 | 21.543171 | 21.543171 | 0.0 |
Basic 06-Vnf With Charm | 0.000000 | 20.113113 | 20.113113 | 0.000000 | 0.999954 | 18.415440 | 19.415394 | 0.051503 | 0.000000 | 20.624444 | 20.624444 | 0.000000 | 0.0 | 21.543171 | 21.543171 | 0.0 |
Basic 07-Secure Key Management | 0.000000 | 20.113113 | 20.113113 | 0.000000 | 0.000000 | 19.415394 | 19.415394 | 0.000000 | 0.000000 | 20.624444 | 20.624444 | 0.000000 | 0.0 | 21.543171 | 21.543171 | 0.0 |
Basic 08-Disable Port Security Network Level | 0.000000 | 20.113113 | 20.113113 | 0.000000 | 0.000000 | 19.415394 | 19.415394 | 0.000000 | 0.000000 | 20.624444 | 20.624444 | 0.000000 | 0.0 | 21.543171 | 21.543171 | 0.0 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
Slice 01-Network Slicing | 2.000012 | 18.113102 | 20.113113 | 0.099438 | 0.000000 | 19.415394 | 19.415394 | 0.000000 | 0.999988 | 19.624456 | 20.624444 | 0.048486 | 0.0 | 21.543171 | 21.543171 | 0.0 |
Slice 02-Shared Network Slicing | 2.000012 | 18.113102 | 20.113113 | 0.099438 | 0.000000 | 19.415394 | 19.415394 | 0.000000 | 0.999988 | 19.624456 | 20.624444 | 0.048486 | 0.0 | 21.543171 | 21.543171 | 0.0 |
Sol003 01-Vnf-Lifecycle-Management | 2.000012 | 18.113102 | 20.113113 | 0.099438 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.999988 | 19.624456 | 20.624444 | 0.048486 | 0.0 | 21.543171 | 21.543171 | 0.0 |
basic 23-delete vnf package SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.000000 | 0.000000 | 0.0 |
k8s 10-k8s proxy charms SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.000000 | 0.000000 | 0.0 |
63 rows × 16 columns
job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
---|---|---|---|---|---|---|---|---|
0 | osm-stage_3-merge/master | 2623 | 2023-06-14 01:06:30 | 17271536 | SUCCESS | FAIL | 602.0 | 2.0 |
1 | osm-stage_3-merge/v13.0 | 227 | 2023-06-09 22:07:24 | 18623441 | UNSTABLE | FAIL | 538.0 | 20.0 |
2 | osm-stage_3-merge/v12.0 | 421 | 2023-06-10 00:04:28 | 14110100 | UNSTABLE | FAIL | 465.0 | 65.0 |
3 | osm-stage_3-merge/v10.0 | 837 | 2023-06-10 03:08:25 | 9932594 | UNSTABLE | FAIL | 368.0 | 12.0 |
osm-stage_3-merge/master: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
245 | Basic 31-Multivdu Volume Multiattach | FAIL | 2023-06-14 03:08:34 | 2023-06-14 03:11:24 | 8 | 1 | Ensure that VDUs can share data | Execute Remote Command Check Rc Return Output |
255 | Heal 02-Scale Vdu Healing | FAIL | 2023-06-14 03:37:10 | 2023-06-14 03:53:48 | 16 | 1 | Check VNF After Healing | Should Contain |
osm-stage_3-merge/v13.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
187 | Basic 26-Secure Helm Execution Environment | FAIL | 2023-06-09 23:57:43 | 2023-06-10 00:09:42 | 5 | 4 | Instantiate Network Service | Create Network Service |
188 | Basic 27-Update Helm Ee In Running Vnf Instance | FAIL | 2023-06-10 00:09:42 | 2023-06-10 00:21:00 | 6 | 10 | Instantiate Network Service | Create Network Service |
199 | Heal 03-Multiple Healing | FAIL | 2023-06-10 01:05:11 | 2023-06-10 01:23:50 | 15 | 1 | Scale Out Charm VNF | Execute Manual VNF Scale |
203 | K8S 04-Openldap Helm | FAIL | 2023-06-10 01:34:19 | 2023-06-10 01:46:15 | 11 | 3 | Create Network Service Instance | Create Network Service |
215 | Sa 08-Vnf With Vnf Indicators Snmp | FAIL | 2023-06-10 02:40:56 | 2023-06-10 02:56:13 | 5 | 2 | Instantiate Network Service | Create Network Service |
osm-stage_3-merge/v12.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
130 | Basic 20-Manual Native Charm Vdu Scaling | FAIL | 2023-06-10 01:25:52 | 2023-06-10 01:40:47 | 11 | 7 | Perform Manual Vdu Scale Out | Execute Manual VNF Scale |
131 | Basic 21-Support Of Volumes | FAIL | 2023-06-10 01:40:47 | 2023-06-10 01:43:13 | 0 | 10 | Create VNF Descriptor | Create VNFD |
132 | Basic 22-Cross Model Relations | FAIL | 2023-06-10 01:43:13 | 2023-06-10 01:45:02 | 8 | 2 | Add K8s Cluster To OSM | Create K8s Cluster |
144 | Heal 01-Volume Vdu Healing | FAIL | 2023-06-10 02:08:58 | 2023-06-10 02:10:29 | 1 | 15 | Create VNF Descriptors | Create VNFD |
145 | Heal 02-Scale Vdu Healing | FAIL | 2023-06-10 02:10:29 | 2023-06-10 02:11:58 | 1 | 14 | Create VNF Descriptors | Create VNFD |
146 | Heal 03-Multiple Healing | FAIL | 2023-06-10 02:11:58 | 2023-06-10 02:13:28 | 1 | 15 | Create VNF Descriptors | Create VNFD |
162 | Sa 08-Vnf With Vnf Indicators Snmp | FAIL | 2023-06-10 03:27:22 | 2023-06-10 03:42:56 | 5 | 2 | Instantiate Network Service | Create Network Service |
osm-stage_3-merge/v10.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
104 | K8S 05-K8S Proxy Charms | FAIL | 2023-06-10 04:53:15 | 2023-06-10 04:55:15 | 5 | 6 | Instantiate Charm Network Service | Create Network Service |
105 | K8S 06-K8S Secure Key Management | FAIL | 2023-06-10 04:55:15 | 2023-06-10 04:59:02 | 8 | 4 | Instantiate Charm Network Service | Create Network Service |
112 | Sa 08-Vnf With Vnf Indicators Snmp | FAIL | 2023-06-10 05:31:00 | 2023-06-10 05:41:51 | 5 | 2 | Instantiate Network Service | Create Network Service |
osm-stage_3-merge/master: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
525 | Basic 31-Multivdu Volume Multiattach | Ensure that VDUs can share data | Execute Remote Command Check Rc Return Output | FAIL | 2023-06-14 03:10:43 | 2023-06-14 03:10:46 |
582 | Heal 02-Scale Vdu Healing | Check VNF After Healing | Should Contain | FAIL | 2023-06-14 03:50:16 | 2023-06-14 03:50:16 |
osm-stage_3-merge/v13.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
359 | Basic 26-Secure Helm Execution Environment | Instantiate Network Service | Create Network Service | FAIL | 2023-06-09 23:57:46 | 2023-06-10 00:09:00 |
361 | Basic 26-Secure Helm Execution Environment | Get Management Ip Address | Get Vnf Management Ip Address | FAIL | 2023-06-10 00:09:00 | 2023-06-10 00:09:00 |
364 | Basic 26-Secure Helm Execution Environment | Test SSH Access | Variable Should Exist | FAIL | 2023-06-10 00:09:00 | 2023-06-10 00:09:00 |
367 | Basic 26-Secure Helm Execution Environment | Check Remote File Created Via Day 1 Operation | Check If remote File Exists | FAIL | 2023-06-10 00:09:00 | 2023-06-10 00:09:00 |
374 | Basic 27-Update Helm Ee In Running Vnf Instance | Instantiate Network Service | Create Network Service | FAIL | 2023-06-10 00:09:46 | 2023-06-10 00:20:19 |
376 | Basic 27-Update Helm Ee In Running Vnf Instance | Set NS Update Config | Variable Should Exist | FAIL | 2023-06-10 00:20:19 | 2023-06-10 00:20:19 |
381 | Basic 27-Update Helm Ee In Running Vnf Instance | Get Management Ip Address | Get Vnf Management Ip Address | FAIL | 2023-06-10 00:20:19 | 2023-06-10 00:20:19 |
384 | Basic 27-Update Helm Ee In Running Vnf Instance | Test SSH Access | Variable Should Exist | FAIL | 2023-06-10 00:20:19 | 2023-06-10 00:20:19 |
387 | Basic 27-Update Helm Ee In Running Vnf Instance | Check Remote Files Created Via Day 1 Operation | Check If remote File Exists | FAIL | 2023-06-10 00:20:19 | 2023-06-10 00:20:19 |
388 | Basic 27-Update Helm Ee In Running Vnf Instance | Execute Day 2 Operation | Variable Should Exist | FAIL | 2023-06-10 00:20:19 | 2023-06-10 00:20:19 |
390 | Basic 27-Update Helm Ee In Running Vnf Instance | Check Remote File Created Via Day 2 Operation | Check If remote File Exists | FAIL | 2023-06-10 00:20:19 | 2023-06-10 00:20:19 |
392 | Basic 27-Update Helm Ee In Running Vnf Instance | Update Running VNF Instance | Update Network Service | FAIL | 2023-06-10 00:20:21 | 2023-06-10 00:20:21 |
393 | Basic 27-Update Helm Ee In Running Vnf Instance | Execute Day 2 Operation on Upgraded Helm EE | Variable Should Exist | FAIL | 2023-06-10 00:20:21 | 2023-06-10 00:20:21 |
395 | Basic 27-Update Helm Ee In Running Vnf Instance | Check Remote Folder Created Via Day 2 Operation | Check If Remote Folder Exists | FAIL | 2023-06-10 00:20:21 | 2023-06-10 00:20:21 |
410 | Heal 03-Multiple Healing | Scale Out Charm VNF | Execute Manual VNF Scale | FAIL | 2023-06-10 01:10:25 | 2023-06-10 01:14:40 |
474 | K8S 04-Openldap Helm | Create Network Service Instance | Create Network Service | FAIL | 2023-06-10 01:35:15 | 2023-06-10 01:43:06 |
482 | K8S 04-Openldap Helm | Execute Upgrade Operation | Execute NS K8s Action | FAIL | 2023-06-10 01:43:08 | 2023-06-10 01:44:15 |
486 | K8S 04-Openldap Helm | Check Replicas After Upgrade Operation | Should Be Equal As Integers | FAIL | 2023-06-10 01:44:47 | 2023-06-10 01:44:47 |
499 | Sa 08-Vnf With Vnf Indicators Snmp | Instantiate Network Service | Create Network Service | FAIL | 2023-06-10 02:41:00 | 2023-06-10 02:46:33 |
505 | Sa 08-Vnf With Vnf Indicators Snmp | Get VNF SNMP Metrics | Wait Until Keyword Succeeds | FAIL | 2023-06-10 02:46:33 | 2023-06-10 02:55:34 |
osm-stage_3-merge/v12.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
84 | Basic 20-Manual Native Charm Vdu Scaling | Perform Manual Vdu Scale Out | Execute Manual VNF Scale | FAIL | 2023-06-10 01:31:05 | 2023-06-10 01:35:23 |
96 | Basic 20-Manual Native Charm Vdu Scaling | Perform Manual Vdu Scale In | Execute Manual VNF Scale | FAIL | 2023-06-10 01:35:31 | 2023-06-10 01:37:54 |
101 | Basic 20-Manual Native Charm Vdu Scaling | Check Vdus After Scaling In | Run Keyword If | FAIL | 2023-06-10 01:38:09 | 2023-06-10 01:38:09 |
106 | Basic 20-Manual Native Charm Vdu Scaling | Check Number of Duplicated Application Name Af... | Run Keyword If | FAIL | 2023-06-10 01:38:25 | 2023-06-10 01:38:25 |
107 | Basic 20-Manual Native Charm Vdu Scaling | Delete NS Instance | Delete NS | FAIL | 2023-06-10 01:38:25 | 2023-06-10 01:38:40 |
... | ... | ... | ... | ... | ... | ... |
336 | Heal 03-Multiple Healing | Delete NS Instance | Delete NS | FAIL | 2023-06-10 02:12:06 | 2023-06-10 02:12:07 |
337 | Heal 03-Multiple Healing | Delete NS Descriptor | Delete NSD | FAIL | 2023-06-10 02:12:07 | 2023-06-10 02:12:09 |
338 | Heal 03-Multiple Healing | Delete VNF Descriptors | Delete VNFD | FAIL | 2023-06-10 02:12:09 | 2023-06-10 02:12:11 |
343 | Sa 08-Vnf With Vnf Indicators Snmp | Instantiate Network Service | Create Network Service | FAIL | 2023-06-10 03:27:26 | 2023-06-10 03:32:59 |
349 | Sa 08-Vnf With Vnf Indicators Snmp | Get VNF SNMP Metrics | Wait Until Keyword Succeeds | FAIL | 2023-06-10 03:32:59 | 2023-06-10 03:42:00 |
61 rows × 6 columns
osm-stage_3-merge/v10.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
2 | K8S 05-K8S Proxy Charms | Instantiate Charm Network Service | Create Network Service | FAIL | 2023-06-10 04:53:19 | 2023-06-10 04:53:59 |
4 | K8S 05-K8S Proxy Charms | Get Management Ip Addresses | Get Vnf Management Ip Address | FAIL | 2023-06-10 04:53:59 | 2023-06-10 04:53:59 |
10 | K8S 05-K8S Proxy Charms | Test SSH Access | Variable Should Exist | FAIL | 2023-06-10 04:53:59 | 2023-06-10 04:53:59 |
15 | K8S 05-K8S Proxy Charms | Check Remote Files Created Via Day 1 Operations | Check If remote File Exists | FAIL | 2023-06-10 04:53:59 | 2023-06-10 04:53:59 |
17 | K8S 05-K8S Proxy Charms | Execute Day 2 Operations | Variable Should Exist | FAIL | 2023-06-10 04:53:59 | 2023-06-10 04:53:59 |
20 | K8S 05-K8S Proxy Charms | Check Remote Files Created Via Day 2 Operations | Check If remote File Exists | FAIL | 2023-06-10 04:53:59 | 2023-06-10 04:53:59 |
27 | K8S 06-K8S Secure Key Management | Instantiate Charm Network Service | Create Network Service | FAIL | 2023-06-10 04:55:19 | 2023-06-10 04:55:59 |
41 | K8S 06-K8S Secure Key Management | Check Remote Files Created Via Day 1 Operations | Check If remote File Exists | FAIL | 2023-06-10 04:56:39 | 2023-06-10 04:56:41 |
44 | K8S 06-K8S Secure Key Management | Execute Day 2 Operations | Execute NS Action | FAIL | 2023-06-10 04:56:41 | 2023-06-10 04:57:46 |
46 | K8S 06-K8S Secure Key Management | Check Remote Files Created Via Day 2 Operations | Check If remote File Exists | FAIL | 2023-06-10 04:57:46 | 2023-06-10 04:57:47 |
53 | Sa 08-Vnf With Vnf Indicators Snmp | Instantiate Network Service | Create Network Service | FAIL | 2023-06-10 05:31:03 | 2023-06-10 05:36:08 |
59 | Sa 08-Vnf With Vnf Indicators Snmp | Get VNF SNMP Metrics | Wait Until Keyword Succeeds | FAIL | 2023-06-10 05:36:08 | 2023-06-10 05:41:08 |
Powered by osm-analytics