Date and time of the report: 2022-05-25 08:20:30.432932
Analysed period: 2022-05-04 to 2022-05-25.
Database update:
Retrieving from database...
DONE
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_result
is the outcome of the build, reported by Jenkins. It can be: SUCCESS
, FAILURE
, UNSTABLE
or ABORTED
.test_result
is the summary of the concerned Robot tests. It can be: FAIL
, UNAVAILABLE
or PASS
.Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
grp_build_result
.grp_test_result
.grp_success_fail
.For the identification of these sequences, the following mapping applies:
Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
DONE
DONE
job | Master branch | Release ELEVEN | Release TEN | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
suites | ||||||||||||
Basic 01-Crud Operations On Vim Targets | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 05-Instantiation Parameters In Cloud Init | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 06-Vnf With Charm | 0.173796 | 21.145382 | 21.319178 | 0.008152 | 2.000081 | 18.672060 | 20.672141 | 0.096752 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 07-Secure Key Management | 0.734468 | 20.584711 | 21.319178 | 0.034451 | 1.472674 | 19.199468 | 20.672141 | 0.071240 | 0.372523 | 21.127512 | 21.500035 | 0.017327 |
Basic 08-Disable Port Security Network Level | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 09-Manual Vdu Scaling | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 11-Native Charms | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 3.000012 | 17.672130 | 20.672141 | 0.145123 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 12-Ns Primitives | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 13-Ns Relations | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 2.000035 | 18.672106 | 20.672141 | 0.096750 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 14-Vnf Relations | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 2.000035 | 18.672106 | 20.672141 | 0.096750 | 1.000000 | 20.500035 | 21.500035 | 0.046512 |
Basic 15-Rbac Configurations | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 16-Advanced Onboarding And Scaling | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 17-Delete Vnf Package | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 18-Ns Ip Profile | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.999988 | 19.672153 | 20.672141 | 0.048374 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 19-Vnf Ip Profile | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Basic 20-Manual Native Charm Vdu Scaling | 2.734456 | 18.584722 | 21.319178 | 0.128263 | 7.472685 | 13.199456 | 20.672141 | 0.361486 | 9.000046 | 12.499988 | 21.500035 | 0.418606 |
Basic 21-Persistent Volumes | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Basic 21-Support Of Volumes | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Basic 22-Cross Model Relations | 2.311238 | 19.007940 | 21.319178 | 0.108411 | 6.359456 | 14.312685 | 20.672141 | 0.307634 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Basic 23-Sol004 Sol007 Packages | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Basic 24-Affinity Groups | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 8.672164 | 8.672164 | 0.000000 | 0.000000 | 13.499954 | 13.499954 | 0.000000 |
Basic 25-Update Charm In Running Vnf Instance | 0.869317 | 6.180417 | 7.049734 | 0.123312 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Epa 01-Epa Sriov | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Epa 02-Additional Capabilities | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Epa 03-Crud Operations On Sdnc | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Epa 04-Epa Underlay Sriov | 4.592662 | 16.726516 | 21.319178 | 0.215424 | 3.672280 | 16.999861 | 20.672141 | 0.177644 | 4.456852 | 17.043183 | 21.500035 | 0.207295 |
Epa 05-Epa Underlay Passthrough | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Fail 01-Insufficient Resources | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Hackfest Basic | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Hackfest Cloudinit | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Hackfest Multivdu | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
K8S 01-Create K8S Cluster | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
K8S 02-K8Scluster Creation | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
K8S 03-Simple K8S | 1.000046 | 20.319132 | 21.319178 | 0.046908 | 2.000035 | 18.672106 | 20.672141 | 0.096750 | 2.000000 | 19.500035 | 21.500035 | 0.093023 |
K8S 04-Openldap Helm | 0.323125 | 20.996053 | 21.319178 | 0.015157 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
K8S 05-K8S Proxy Charms | 1.000012 | 20.319167 | 21.319178 | 0.046907 | 0.672234 | 19.999907 | 20.672141 | 0.032519 | 1.000000 | 20.500035 | 21.500035 | 0.046512 |
K8S 06-K8S Secure Key Management | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.999977 | 19.672164 | 20.672141 | 0.048373 | 2.627350 | 18.872685 | 21.500035 | 0.122202 |
K8S 07-Dummy Helm | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
K8S 08-Simple K8S Scaling | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 1.672222 | 18.999919 | 20.672141 | 0.080893 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
K8S 09-Pebble Charm K8S | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 2.359560 | 18.312581 | 20.672141 | 0.114142 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
K8S 10-Sol004 Sol007 With K8S Proxy Charms | 0.671238 | 20.647940 | 21.319178 | 0.031485 | 2.527373 | 18.144769 | 20.672141 | 0.122260 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
K8S 11-Simple Helm K8S Scaling | 7.084792 | 0.000000 | 7.084792 | 1.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Quotas 01-Quota Enforcement | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Sa 01-Vnf With Vim Metrics | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Sa 02-Vnf With Vim Metrics And Autoscaling | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Sa 07-Alarms From Sa-Related Vnfs | 1.000012 | 20.319167 | 21.319178 | 0.046907 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Sa 08-Vnf With Vnf Indicators Snmp | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Slice 01-Network Slicing | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Slice 02-Shared Network Slicing | 0.000000 | 21.319178 | 21.319178 | 0.000000 | 0.000000 | 20.672141 | 20.672141 | 0.000000 | 0.000000 | 21.500035 | 21.500035 | 0.000000 |
Sol003 01-Vnf-Lifecycle-Management | 5.999907 | 2.584757 | 8.584664 | 0.698910 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
basic 23-delete vnf package SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
k8s 10-k8s proxy charms SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
---|---|---|---|---|---|---|---|---|
0 | osm-stage_3-merge/master | 2116 | 2022-05-25 01:06:15 | 12393885 | UNSTABLE | FAIL | 444.0 | 9.0 |
1 | osm-stage_3-merge/v11.0 | 252 | 2022-05-24 23:00:14 | 12450863 | UNSTABLE | FAIL | 407.0 | 14.0 |
2 | osm-stage_3-merge/v10.0 | 430 | 2022-05-25 03:08:15 | 9704772 | UNSTABLE | FAIL | 372.0 | 4.0 |
osm-stage_3-merge/master: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
49 | Basic 22-Cross Model Relations | FAIL | 2022-05-25 02:33:11 | 2022-05-25 02:48:47 | 5 | 5 | Delete NS Instance | Delete NS |
56 | Epa 04-Epa Underlay Sriov | FAIL | 2022-05-25 02:59:50 | 2022-05-25 03:01:20 | 5 | 4 | Instantiate Network Service | Create Network Service |
osm-stage_3-merge/v11.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
140 | Epa 04-Epa Underlay Sriov | FAIL | 2022-05-25 00:45:57 | 2022-05-25 00:47:25 | 5 | 4 | Instantiate Network Service | Create Network Service |
148 | K8S 05-K8S Proxy Charms | FAIL | 2022-05-25 01:04:43 | 2022-05-25 01:17:25 | 5 | 6 | Instantiate Charm Network Service | Create Network Service |
151 | K8S 08-Simple K8S Scaling | FAIL | 2022-05-25 01:24:41 | 2022-05-25 01:41:10 | 10 | 4 | Delete NS K8s Instance Test | Delete NS |
osm-stage_3-merge/v10.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
98 | Epa 04-Epa Underlay Sriov | FAIL | 2022-05-25 04:35:52 | 2022-05-25 04:37:28 | 5 | 4 | Instantiate Network Service | Create Network Service |
osm-stage_3-merge/master: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
6 | Basic 22-Cross Model Relations | Delete NS Instance | Delete NS | FAIL | 2022-05-25 02:39:11 | 2022-05-25 02:43:23 |
7 | Basic 22-Cross Model Relations | Remove K8s Cluster from OSM | Delete K8s Cluster | FAIL | 2022-05-25 02:43:23 | 2022-05-25 02:43:25 |
8 | Basic 22-Cross Model Relations | Delete NS Descriptor | Delete NSD | FAIL | 2022-05-25 02:43:25 | 2022-05-25 02:43:27 |
9 | Basic 22-Cross Model Relations | Delete VNF Descriptor 1 | Delete VNFD | FAIL | 2022-05-25 02:43:27 | 2022-05-25 02:43:28 |
10 | Basic 22-Cross Model Relations | Delete VNF Descriptor 2 | Delete VNFD | FAIL | 2022-05-25 02:43:28 | 2022-05-25 02:43:30 |
14 | Epa 04-Epa Underlay Sriov | Instantiate Network Service | Create Network Service | FAIL | 2022-05-25 02:59:54 | 2022-05-25 03:00:36 |
16 | Epa 04-Epa Underlay Sriov | Get Management Ip Addresses | Variable Should Exist | FAIL | 2022-05-25 03:00:36 | 2022-05-25 03:00:36 |
23 | Epa 04-Epa Underlay Sriov | Ping from Vnf1 to Vnf2 | Variable Should Exist | FAIL | 2022-05-25 03:00:36 | 2022-05-25 03:00:36 |
27 | Epa 04-Epa Underlay Sriov | Ping from Vnf2 to Vnf1 | Variable Should Exist | FAIL | 2022-05-25 03:00:36 | 2022-05-25 03:00:36 |
osm-stage_3-merge/v11.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
58 | Epa 04-Epa Underlay Sriov | Instantiate Network Service | Create Network Service | FAIL | 2022-05-25 00:46:00 | 2022-05-25 00:46:42 |
60 | Epa 04-Epa Underlay Sriov | Get Management Ip Addresses | Variable Should Exist | FAIL | 2022-05-25 00:46:42 | 2022-05-25 00:46:42 |
67 | Epa 04-Epa Underlay Sriov | Ping from Vnf1 to Vnf2 | Variable Should Exist | FAIL | 2022-05-25 00:46:42 | 2022-05-25 00:46:42 |
71 | Epa 04-Epa Underlay Sriov | Ping from Vnf2 to Vnf1 | Variable Should Exist | FAIL | 2022-05-25 00:46:42 | 2022-05-25 00:46:42 |
79 | K8S 05-K8S Proxy Charms | Instantiate Charm Network Service | Create Network Service | FAIL | 2022-05-25 01:04:47 | 2022-05-25 01:16:05 |
81 | K8S 05-K8S Proxy Charms | Get Management Ip Addresses | Get Vnf Management Ip Address | FAIL | 2022-05-25 01:16:05 | 2022-05-25 01:16:05 |
87 | K8S 05-K8S Proxy Charms | Test SSH Access | Variable Should Exist | FAIL | 2022-05-25 01:16:05 | 2022-05-25 01:16:05 |
92 | K8S 05-K8S Proxy Charms | Check Remote Files Created Via Day 1 Operations | Check If remote File Exists | FAIL | 2022-05-25 01:16:05 | 2022-05-25 01:16:05 |
94 | K8S 05-K8S Proxy Charms | Execute Day 2 Operations | Variable Should Exist | FAIL | 2022-05-25 01:16:05 | 2022-05-25 01:16:05 |
97 | K8S 05-K8S Proxy Charms | Check Remote Files Created Via Day 2 Operations | Check If remote File Exists | FAIL | 2022-05-25 01:16:05 | 2022-05-25 01:16:05 |
120 | K8S 08-Simple K8S Scaling | Delete NS K8s Instance Test | Delete NS | FAIL | 2022-05-25 01:28:30 | 2022-05-25 01:32:34 |
121 | K8S 08-Simple K8S Scaling | Remove K8s Cluster from OSM | Delete K8s Cluster | FAIL | 2022-05-25 01:32:34 | 2022-05-25 01:32:36 |
122 | K8S 08-Simple K8S Scaling | Delete NS Descriptor Test | Delete NSD | FAIL | 2022-05-25 01:32:36 | 2022-05-25 01:32:37 |
123 | K8S 08-Simple K8S Scaling | Delete VNF Descriptor Test | Delete VNFD | FAIL | 2022-05-25 01:32:37 | 2022-05-25 01:32:39 |
osm-stage_3-merge/v10.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
36 | Epa 04-Epa Underlay Sriov | Instantiate Network Service | Create Network Service | FAIL | 2022-05-25 04:35:56 | 2022-05-25 04:36:45 |
38 | Epa 04-Epa Underlay Sriov | Get Management Ip Addresses | Variable Should Exist | FAIL | 2022-05-25 04:36:45 | 2022-05-25 04:36:45 |
45 | Epa 04-Epa Underlay Sriov | Ping from Vnf1 to Vnf2 | Variable Should Exist | FAIL | 2022-05-25 04:36:45 | 2022-05-25 04:36:45 |
49 | Epa 04-Epa Underlay Sriov | Ping from Vnf2 to Vnf1 | Variable Should Exist | FAIL | 2022-05-25 04:36:45 | 2022-05-25 04:36:45 |