Analysis of Robot reports from OSM Jenkins¶
Date and time of the report: 2025-11-30 13:45:38.078097
Analysed period: 2025-11-09 to 2025-11-30.
1. Retrieval of all currrent data for aggregate analytics¶
Database update:
Retrieving new jobs...
Getting new builds from: osm-stage_3-merge/master, osm-stage_3-merge/v18.0, osm-stage_3-merge/v17.0, osm-stage_3-merge/v16.0, osm-stage_3-merge/v14.0
DONE
Retrieving from database...
DONE
2. Aggregated analysis of stability¶
2.1 Restricts data to time window for analysis¶
2.2 Finding sequences of successful builds and Robot reports¶
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_resultis the outcome of the build, reported by Jenkins. It can be:SUCCESS,FAILURE,UNSTABLEorABORTED.test_resultis the summary of the concerned Robot tests. It can be:FAIL,UNAVAILABLEorPASS.
Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
- Successful builds/failed builds in a row:
grp_build_result. - Successful test reports vs. test reports with fails in a row:
grp_test_result. - Clean builds and tests vs. failures (of any kind) in a row:
grp_success_fail.
For the identification of these sequences, the following mapping applies:
| Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
|---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
DONE
2.3 Finding sequences of pass/fails per test suite¶
DONE
3. Reports¶
3.1 Aggregated success rate per test step¶
3.2 Overall success of Jenkins builds and Robot tests¶
3.3 Sequences of pass/fails per test suites¶
3.4 Failing days per test suite¶
Out[58]:
| job | Master branch | Release EIGHTEEN | Release FOURTEEN | Release SEVENTEEN | Release SIXTEEN | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
| suites | ||||||||||||||||||||
| Basic 01-Crud Operations On Vim Targets | 0.000000 | 20.708646 | 20.708646 | 0.000000 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
| Basic 05-Instantiation Parameters In Cloud Init | 0.000000 | 20.708646 | 20.708646 | 0.000000 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
| Basic 06-Vnf With Charm | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 |
| Basic 07-Secure Key Management | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 |
| Basic 08-Disable Port Security Network Level | 0.000000 | 20.708646 | 20.708646 | 0.000000 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| Sa 08-Vnf With Vnf Indicators Snmp | 0.708634 | 20.000012 | 20.708646 | 0.034219 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
| Slice 01-Network Slicing | 0.708634 | 20.000012 | 20.708646 | 0.034219 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
| Slice 02-Shared Network Slicing | 0.708634 | 20.000012 | 20.708646 | 0.034219 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
| Sol003 01-Vnf-Lifecycle-Management | 0.708634 | 20.000012 | 20.708646 | 0.034219 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 20.832211 | 20.832211 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
| Sol003 02-Dualstack Ip Vnfm | 0.708634 | 20.000012 | 20.708646 | 0.034219 | 0.0 | 18.558657 | 18.558657 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 20.637118 | 20.637118 | 0.0 | 0.0 | 21.795428 | 21.795428 | 0.0 |
62 rows × 20 columns
4. Information about the latest builds of relevant jobs¶
Latest build of each job¶
| auto_id | job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 8795 | osm-stage_3-merge/master | 3573 | 2025-11-30 02:06:32 | 36328058 | FAILURE | FAIL | 213.0 | 272.0 |
| 1 | 8796 | osm-stage_3-merge/v18.0 | 102 | 2025-11-30 07:06:58 | 20149360 | UNSTABLE | FAIL | 412.0 | 73.0 |
| 2 | 8797 | osm-stage_3-merge/v17.0 | 328 | 2025-11-30 03:49:33 | 27365645 | FAILURE | FAIL | 329.0 | 156.0 |
| 3 | 8798 | osm-stage_3-merge/v16.0 | 438 | 2025-11-30 00:01:31 | 25518578 | FAILURE | FAIL | 302.0 | 170.0 |
| 4 | 8799 | osm-stage_3-merge/v14.0 | 855 | 2025-11-29 23:08:31 | 1910315 | FAILURE | UNAVAILABLE | NaN | NaN |
Failed test suites per job (if any):¶
osm-stage_3-merge/master: (full report)
| auto_id | name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|---|
| 105 | 253833 | Basic 21-Support Of Volumes | FAIL | 2025-11-30 02:52:12 | 2025-11-30 03:08:26 | 3 | 7 | Network Service Instance Test | Create Network Service |
| 107 | 253835 | Basic 24-Affinity Groups | FAIL | 2025-11-30 03:08:41 | 2025-11-30 03:24:49 | 2 | 5 | Network Service Instance Test | Create Network Service |
| 108 | 253836 | Basic 27-Update Helm Ee In Running Vnf Instance | FAIL | 2025-11-30 03:24:49 | 2025-11-30 03:50:49 | 3 | 13 | Instantiate Network Service | Create Network Service |
| 109 | 253837 | Basic 28-Keep Persistent Volumes | FAIL | 2025-11-30 03:50:49 | 2025-11-30 04:11:19 | 2 | 14 | Network Service Instance Test | Create Network Service |
| 110 | 253838 | Basic 29-Vnf Ipv6 Profile | FAIL | 2025-11-30 04:11:19 | 2025-11-30 04:27:36 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 111 | 253839 | Basic 30-Ns Ipv6 Profile | FAIL | 2025-11-30 04:27:36 | 2025-11-30 04:43:51 | 2 | 10 | Instantiate Network Service | Create Network Service |
| 112 | 253840 | Basic 31-Multivdu Volume Multiattach | FAIL | 2025-11-30 04:43:51 | 2025-11-30 05:00:07 | 2 | 7 | Network Service Instance Test | Create Network Service |
| 113 | 253841 | Epa 01-Epa Sriov | FAIL | 2025-11-30 05:00:07 | 2025-11-30 05:16:53 | 2 | 6 | Instantiate Network Service | Create Network Service |
| 114 | 253842 | Epa 02-Additional Capabilities | FAIL | 2025-11-30 05:16:53 | 2025-11-30 05:33:06 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 116 | 253844 | Epa 04-Epa Underlay Sriov | FAIL | 2025-11-30 05:34:16 | 2025-11-30 05:50:31 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 117 | 253845 | Fail 01-Insufficient Resources | FAIL | 2025-11-30 05:50:31 | 2025-11-30 06:02:13 | 2 | 4 | Cannot Instantiate Network Service | Wait Until Keyword Succeeds |
| 122 | 253850 | Heal 01-Volume Vdu Healing | FAIL | 2025-11-30 06:40:17 | 2025-11-30 06:58:17 | 2 | 14 | Create VNF Descriptors | Create VNFD |
| 123 | 253851 | Heal 02-Scale Vdu Healing | FAIL | 2025-11-30 06:58:17 | 2025-11-30 07:16:14 | 2 | 16 | Create VNF Descriptors | Create VNFD |
| 124 | 253852 | Heal 03-Multiple Healing | FAIL | 2025-11-30 07:16:14 | 2025-11-30 07:34:08 | 1 | 17 | Create VNF Descriptors | Create VNFD |
| 125 | 253853 | Heal 04-Autohealing | FAIL | 2025-11-30 07:34:08 | 2025-11-30 08:00:27 | 3 | 11 | Network Service Instance Test | Create Network Service |
| 127 | 253855 | K8S 04-Openldap Helm | FAIL | 2025-11-30 08:01:13 | 2025-11-30 08:22:57 | 5 | 8 | Create Network Service Instance | Create Network Service |
| 128 | 253856 | K8S 07-Dummy Helm | FAIL | 2025-11-30 08:22:57 | 2025-11-30 08:32:10 | 4 | 6 | Create Package For OpenLDAP CNF | Create VNFD |
| 129 | 253857 | K8S 13-Two Helm Kdu | FAIL | 2025-11-30 08:32:10 | 2025-11-30 08:35:09 | 10 | 8 | Execute Upgrade Operation over first KDU | Execute NS K8s Action |
| 130 | 253858 | K8S 14-Helm Upgrade | FAIL | 2025-11-30 08:35:09 | 2025-11-30 08:46:33 | 5 | 11 | Create Package For OpenLDAP CNF | Create VNFD |
| 131 | 253859 | Lcmop 01-Cancel Operation Basic | FAIL | 2025-11-30 08:46:33 | 2025-11-30 08:57:28 | 6 | 3 | Delete NS Instance Test | Delete NS |
| 133 | 253861 | Sa 01-Vnf With Vim Metrics | FAIL | 2025-11-30 08:58:37 | 2025-11-30 09:09:52 | 3 | 4 | Instantiate Network Service | Create Network Service |
| 134 | 253862 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2025-11-30 09:09:52 | 2025-11-30 09:43:14 | 0 | 15 | Create VNF Descriptor | Create VNFD |
| 135 | 253863 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2025-11-30 09:43:14 | 2025-11-30 10:07:30 | 0 | 16 | Create Webhook Service VNF Descriptor | Create VNFD |
| 136 | 253864 | Sa 08-Vnf With Vnf Indicators Snmp | FAIL | 2025-11-30 10:07:30 | 2025-11-30 10:29:41 | 0 | 7 | Create VNF Descriptor | Create VNFD |
| 137 | 253865 | Slice 01-Network Slicing | FAIL | 2025-11-30 10:29:41 | 2025-11-30 10:37:20 | 1 | 12 | Create Slice VNF Descriptors | Create VNFD |
| 138 | 253866 | Slice 02-Shared Network Slicing | FAIL | 2025-11-30 10:37:20 | 2025-11-30 10:47:35 | 2 | 18 | Create Slice VNF Descriptors | Create VNFD |
| 139 | 253867 | Sol003 01-Vnf-Lifecycle-Management | FAIL | 2025-11-30 10:47:35 | 2025-11-30 11:50:01 | 0 | 10 | Create VNF Package From JSON File | Get VNFDs List |
| 140 | 253868 | Sol003 02-Dualstack Ip Vnfm | FAIL | 2025-11-30 11:50:01 | 2025-11-30 12:08:45 | 0 | 9 | Create VNFD Descriptor | Create VNFD |
osm-stage_3-merge/v18.0: (full report)
| auto_id | name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|---|
| 150 | 253878 | Basic 21-Support Of Volumes | FAIL | 2025-11-30 07:55:59 | 2025-11-30 08:12:02 | 3 | 7 | Network Service Instance Test | Create Network Service |
| 152 | 253880 | Basic 24-Affinity Groups | FAIL | 2025-11-30 08:12:16 | 2025-11-30 08:28:43 | 2 | 5 | Network Service Instance Test | Create Network Service |
| 153 | 253881 | Basic 27-Update Helm Ee In Running Vnf Instance | FAIL | 2025-11-30 08:28:43 | 2025-11-30 08:55:09 | 3 | 13 | Instantiate Network Service | Create Network Service |
| 154 | 253882 | Basic 28-Keep Persistent Volumes | FAIL | 2025-11-30 08:55:09 | 2025-11-30 09:15:26 | 2 | 14 | Network Service Instance Test | Create Network Service |
| 155 | 253883 | Basic 29-Vnf Ipv6 Profile | FAIL | 2025-11-30 09:15:26 | 2025-11-30 09:31:15 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 156 | 253884 | Basic 30-Ns Ipv6 Profile | FAIL | 2025-11-30 09:31:15 | 2025-11-30 09:47:15 | 2 | 10 | Instantiate Network Service | Create Network Service |
| 157 | 253885 | Basic 31-Multivdu Volume Multiattach | FAIL | 2025-11-30 09:47:15 | 2025-11-30 10:03:34 | 2 | 7 | Network Service Instance Test | Create Network Service |
| 167 | 253895 | Heal 01-Volume Vdu Healing | FAIL | 2025-11-30 10:52:12 | 2025-11-30 10:58:23 | 14 | 2 | Create VNF Descriptors | Create VNFD |
| 168 | 253896 | Heal 02-Scale Vdu Healing | FAIL | 2025-11-30 10:58:23 | 2025-11-30 11:04:42 | 16 | 2 | Create VNF Descriptors | Create VNFD |
| 169 | 253897 | Heal 03-Multiple Healing | FAIL | 2025-11-30 11:04:42 | 2025-11-30 11:11:59 | 16 | 2 | Create VNF Descriptors | Create VNFD |
| 172 | 253900 | K8S 04-Openldap Helm | FAIL | 2025-11-30 11:21:16 | 2025-11-30 11:26:21 | 11 | 2 | Execute Upgrade Operation | Execute NS K8s Action |
| 175 | 253903 | K8S 14-Helm Upgrade | FAIL | 2025-11-30 11:34:01 | 2025-11-30 11:40:22 | 14 | 2 | Execute First NS Upgrade Using KDU_RESET Flag ... | Execute NS K8s Action |
osm-stage_3-merge/v17.0: (full report)
| auto_id | name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|---|
| 195 | 253923 | Basic 21-Support Of Volumes | FAIL | 2025-11-30 04:46:59 | 2025-11-30 05:03:11 | 3 | 7 | Network Service Instance Test | Create Network Service |
| 197 | 253925 | Basic 24-Affinity Groups | FAIL | 2025-11-30 05:03:25 | 2025-11-30 05:19:36 | 2 | 5 | Network Service Instance Test | Create Network Service |
| 198 | 253926 | Basic 27-Update Helm Ee In Running Vnf Instance | FAIL | 2025-11-30 05:19:36 | 2025-11-30 05:45:39 | 3 | 13 | Instantiate Network Service | Create Network Service |
| 199 | 253927 | Basic 28-Keep Persistent Volumes | FAIL | 2025-11-30 05:45:39 | 2025-11-30 06:06:07 | 2 | 14 | Network Service Instance Test | Create Network Service |
| 200 | 253928 | Basic 29-Vnf Ipv6 Profile | FAIL | 2025-11-30 06:06:07 | 2025-11-30 06:22:21 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 201 | 253929 | Basic 30-Ns Ipv6 Profile | FAIL | 2025-11-30 06:22:21 | 2025-11-30 06:38:34 | 2 | 10 | Instantiate Network Service | Create Network Service |
| 202 | 253930 | Basic 31-Multivdu Volume Multiattach | FAIL | 2025-11-30 06:38:34 | 2025-11-30 06:54:45 | 2 | 7 | Network Service Instance Test | Create Network Service |
| 203 | 253931 | Epa 01-Epa Sriov | FAIL | 2025-11-30 06:54:45 | 2025-11-30 07:11:26 | 2 | 6 | Instantiate Network Service | Create Network Service |
| 204 | 253932 | Epa 02-Additional Capabilities | FAIL | 2025-11-30 07:11:26 | 2025-11-30 07:27:36 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 206 | 253934 | Epa 04-Epa Underlay Sriov | FAIL | 2025-11-30 07:28:46 | 2025-11-30 07:45:00 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 207 | 253935 | Fail 01-Insufficient Resources | FAIL | 2025-11-30 07:45:00 | 2025-11-30 07:57:10 | 2 | 4 | Cannot Instantiate Network Service | Wait Until Keyword Succeeds |
| 208 | 253936 | Gitops 02-Declarative Cluster Registration Oka... | FAIL | 2025-11-30 07:57:10 | 2025-11-30 08:54:08 | 3 | 10 | Add OKA Packages For The Testsuite | Add OKA Package |
| 212 | 253940 | Heal 01-Volume Vdu Healing | FAIL | 2025-11-30 09:01:25 | 2025-11-30 09:19:20 | 2 | 14 | Create VNF Descriptors | Create VNFD |
| 213 | 253941 | Heal 02-Scale Vdu Healing | FAIL | 2025-11-30 09:19:20 | 2025-11-30 09:37:14 | 2 | 16 | Create VNF Descriptors | Create VNFD |
| 214 | 253942 | Heal 03-Multiple Healing | FAIL | 2025-11-30 09:37:14 | 2025-11-30 09:55:02 | 1 | 17 | Create VNF Descriptors | Create VNFD |
| 215 | 253943 | Heal 04-Autohealing | FAIL | 2025-11-30 09:55:02 | 2025-11-30 10:11:16 | 6 | 8 | Network Service Instance Test | Create Network Service |
| 221 | 253949 | Lcmop 01-Cancel Operation Basic | FAIL | 2025-11-30 10:29:30 | 2025-11-30 10:32:28 | 5 | 4 | Create VNF Descriptor | Create VNFD |
osm-stage_3-merge/v16.0: (full report)
| auto_id | name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|---|
| 244 | 253972 | Basic 28-Keep Persistent Volumes | FAIL | 2025-11-30 01:00:36 | 2025-11-30 01:20:30 | 2 | 14 | Network Service Instance Test | Create Network Service |
| 245 | 253973 | Basic 29-Vnf Ipv6 Profile | FAIL | 2025-11-30 01:20:30 | 2025-11-30 01:36:20 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 246 | 253974 | Basic 30-Ns Ipv6 Profile | FAIL | 2025-11-30 01:36:20 | 2025-11-30 01:52:12 | 2 | 10 | Instantiate Network Service | Create Network Service |
| 247 | 253975 | Basic 31-Multivdu Volume Multiattach | FAIL | 2025-11-30 01:52:12 | 2025-11-30 02:08:07 | 2 | 7 | Network Service Instance Test | Create Network Service |
| 248 | 253976 | Epa 01-Epa Sriov | FAIL | 2025-11-30 02:08:07 | 2025-11-30 02:24:30 | 2 | 6 | Instantiate Network Service | Create Network Service |
| 249 | 253977 | Epa 02-Additional Capabilities | FAIL | 2025-11-30 02:24:30 | 2025-11-30 02:40:27 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 250 | 253978 | Epa 03-Crud Operations On Sdnc | FAIL | 2025-11-30 02:40:27 | 2025-11-30 02:41:40 | 2 | 1 | Create Basic SDNC | Wait Until Keyword Succeeds |
| 251 | 253979 | Epa 04-Epa Underlay Sriov | FAIL | 2025-11-30 02:41:40 | 2025-11-30 02:57:33 | 2 | 7 | Instantiate Network Service | Create Network Service |
| 252 | 253980 | Fail 01-Insufficient Resources | FAIL | 2025-11-30 02:57:33 | 2025-11-30 03:07:01 | 5 | 1 | Cannot Instantiate Network Service | Wait Until Keyword Succeeds |
| 256 | 253984 | Heal 01-Volume Vdu Healing | FAIL | 2025-11-30 03:14:43 | 2025-11-30 03:32:55 | 3 | 13 | Network Service Instance Test | Create Network Service |
| 257 | 253985 | Heal 02-Scale Vdu Healing | FAIL | 2025-11-30 03:32:55 | 2025-11-30 03:50:59 | 2 | 16 | Create VNF Descriptors | Create VNFD |
| 258 | 253986 | Heal 03-Multiple Healing | FAIL | 2025-11-30 03:50:59 | 2025-11-30 04:09:07 | 1 | 17 | Create VNF Descriptors | Create VNFD |
| 259 | 253987 | Heal 04-Autohealing | FAIL | 2025-11-30 04:09:07 | 2025-11-30 04:35:02 | 3 | 11 | Network Service Instance Test | Create Network Service |
| 261 | 253989 | K8S 04-Openldap Helm | FAIL | 2025-11-30 04:35:47 | 2025-11-30 04:55:01 | 5 | 8 | Create Network Service Instance | Create Network Service |
| 262 | 253990 | K8S 07-Dummy Helm | FAIL | 2025-11-30 04:55:01 | 2025-11-30 05:03:31 | 4 | 6 | Create Package For OpenLDAP CNF | Create VNFD |
| 263 | 253991 | K8S 13-Two Helm Kdu | FAIL | 2025-11-30 05:03:31 | 2025-11-30 05:06:24 | 10 | 8 | Execute Upgrade Operation over first KDU | Execute NS K8s Action |
| 264 | 253992 | K8S 14-Helm Upgrade | FAIL | 2025-11-30 05:06:24 | 2025-11-30 05:13:25 | 5 | 11 | Create Package For OpenLDAP CNF | Create VNFD |
| 265 | 253993 | Lcmop 01-Cancel Operation Basic | FAIL | 2025-11-30 05:13:25 | 2025-11-30 05:24:04 | 6 | 3 | Delete NS Instance Test | Delete NS |
| 267 | 253995 | Sa 01-Vnf With Vim Metrics | FAIL | 2025-11-30 05:25:07 | 2025-11-30 05:41:03 | 2 | 5 | Instantiate Network Service | Create Network Service |
| 268 | 253996 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2025-11-30 05:41:03 | 2025-11-30 06:22:41 | 5 | 10 | Create VNF Descriptor | Create VNFD |
| 269 | 253997 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2025-11-30 06:22:41 | 2025-11-30 06:43:35 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v14.0: (full report)
| auto_id | name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
|---|
Details of failed tests into failing test suites (if any):¶
osm-stage_3-merge/master: (full report)
| auto_id | suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|---|
| 89 | 6605473 | Basic 21-Support Of Volumes | Network Service Instance Test | Create Network Service | FAIL | 2025-11-30 02:52:16 | 2025-11-30 02:57:46 |
| 97 | 6605481 | Basic 21-Support Of Volumes | Get VNF IP Address | Variable Should Exist | FAIL | 2025-11-30 02:57:48 | 2025-11-30 02:57:48 |
| 101 | 6605485 | Basic 21-Support Of Volumes | Check VDU disks | Variable Should Exist | FAIL | 2025-11-30 02:57:48 | 2025-11-30 02:57:48 |
| 107 | 6605491 | Basic 21-Support Of Volumes | Delete NS Instance Test | Delete NS | FAIL | 2025-11-30 02:57:48 | 2025-11-30 03:01:59 |
| 108 | 6605492 | Basic 21-Support Of Volumes | Delete NS Descriptor Test | Delete NSD | FAIL | 2025-11-30 03:01:59 | 2025-11-30 03:02:01 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 1071 | 6606606 | Sol003 02-Dualstack Ip Vnfm | Get Dual Ip | Get Dual Ip | FAIL | 2025-11-30 11:50:32 | 2025-11-30 11:50:32 |
| 1078 | 6606613 | Sol003 02-Dualstack Ip Vnfm | Verify Dual Ip | Should Be Equal | FAIL | 2025-11-30 11:50:32 | 2025-11-30 11:50:32 |
| 1080 | 6606615 | Sol003 02-Dualstack Ip Vnfm | Terminate VNF | Get Auth Token | FAIL | 2025-11-30 11:50:32 | 2025-11-30 11:59:15 |
| 1091 | 6606626 | Sol003 02-Dualstack Ip Vnfm | Delete VNF | Get Auth Token | FAIL | 2025-11-30 11:59:15 | 2025-11-30 12:07:59 |
| 1096 | 6606631 | Sol003 02-Dualstack Ip Vnfm | Delete VNF Descriptor Test | Delete VNFD | FAIL | 2025-11-30 12:07:59 | 2025-11-30 12:08:14 |
267 rows × 7 columns
osm-stage_3-merge/v18.0: (full report)
| auto_id | suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|---|
| 1099 | 6606869 | Basic 21-Support Of Volumes | Network Service Instance Test | Create Network Service | FAIL | 2025-11-30 07:56:03 | 2025-11-30 08:01:07 |
| 1107 | 6606877 | Basic 21-Support Of Volumes | Get VNF IP Address | Variable Should Exist | FAIL | 2025-11-30 08:01:09 | 2025-11-30 08:01:09 |
| 1111 | 6606881 | Basic 21-Support Of Volumes | Check VDU disks | Variable Should Exist | FAIL | 2025-11-30 08:01:09 | 2025-11-30 08:01:09 |
| 1117 | 6606887 | Basic 21-Support Of Volumes | Delete NS Instance Test | Delete NS | FAIL | 2025-11-30 08:01:09 | 2025-11-30 08:05:28 |
| 1118 | 6606888 | Basic 21-Support Of Volumes | Delete NS Descriptor Test | Delete NSD | FAIL | 2025-11-30 08:05:28 | 2025-11-30 08:05:30 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 1522 | 6607452 | Heal 03-Multiple Healing | Delete VNF Descriptors | Delete VNFD | FAIL | 2025-11-30 11:10:27 | 2025-11-30 11:10:29 |
| 1539 | 6607524 | K8S 04-Openldap Helm | Execute Upgrade Operation | Execute NS K8s Action | FAIL | 2025-11-30 11:22:44 | 2025-11-30 11:23:51 |
| 1542 | 6607527 | K8S 04-Openldap Helm | Check Replicas After Upgrade Operation | Get Vnf Kdu Replica Count | FAIL | 2025-11-30 11:23:51 | 2025-11-30 11:24:24 |
| 1575 | 6607616 | K8S 14-Helm Upgrade | Execute First NS Upgrade Using KDU_RESET Flag ... | Execute NS K8s Action | FAIL | 2025-11-30 11:37:51 | 2025-11-30 11:38:57 |
| 1578 | 6607619 | K8S 14-Helm Upgrade | Get Additional Params And Values From Kdu Afte... | Should Be Equal | FAIL | 2025-11-30 11:38:59 | 2025-11-30 11:38:59 |
72 rows × 7 columns
osm-stage_3-merge/v17.0: (full report)
| auto_id | suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|---|
| 1592 | 6608265 | Basic 21-Support Of Volumes | Network Service Instance Test | Create Network Service | FAIL | 2025-11-30 04:47:02 | 2025-11-30 04:52:35 |
| 1600 | 6608273 | Basic 21-Support Of Volumes | Get VNF IP Address | Variable Should Exist | FAIL | 2025-11-30 04:52:37 | 2025-11-30 04:52:37 |
| 1604 | 6608277 | Basic 21-Support Of Volumes | Check VDU disks | Variable Should Exist | FAIL | 2025-11-30 04:52:37 | 2025-11-30 04:52:37 |
| 1610 | 6608283 | Basic 21-Support Of Volumes | Delete NS Instance Test | Delete NS | FAIL | 2025-11-30 04:52:37 | 2025-11-30 04:56:46 |
| 1611 | 6608284 | Basic 21-Support Of Volumes | Delete NS Descriptor Test | Delete NSD | FAIL | 2025-11-30 04:56:46 | 2025-11-30 04:56:48 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 2161 | 6608888 | Heal 04-Autohealing | Update VIM Objects | Variable Should Exist | FAIL | 2025-11-30 10:10:37 | 2025-11-30 10:10:37 |
| 2168 | 6609027 | Lcmop 01-Cancel Operation Basic | Create VNF Descriptor | Create VNFD | FAIL | 2025-11-30 10:29:30 | 2025-11-30 10:29:33 |
| 2169 | 6609028 | Lcmop 01-Cancel Operation Basic | Create NS Descriptor | Create NSD | FAIL | 2025-11-30 10:29:33 | 2025-11-30 10:29:34 |
| 2178 | 6609037 | Lcmop 01-Cancel Operation Basic | Delete NS Descriptor Test | Delete NSD | FAIL | 2025-11-30 10:30:07 | 2025-11-30 10:30:09 |
| 2179 | 6609038 | Lcmop 01-Cancel Operation Basic | Delete VNF Descriptor Test | Delete VNFD | FAIL | 2025-11-30 10:30:09 | 2025-11-30 10:30:11 |
151 rows × 7 columns
osm-stage_3-merge/v16.0: (full report)
| auto_id | suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|---|
| 2182 | 6609729 | Basic 28-Keep Persistent Volumes | Network Service Instance Test | Create Network Service | FAIL | 2025-11-30 01:00:40 | 2025-11-30 01:06:05 |
| 2184 | 6609731 | Basic 28-Keep Persistent Volumes | Check Persistent Volume Counts | Variable Should Exist | FAIL | 2025-11-30 01:06:05 | 2025-11-30 01:06:05 |
| 2189 | 6609736 | Basic 28-Keep Persistent Volumes | Set Persistent Volume Ids | Variable Should Exist | FAIL | 2025-11-30 01:06:05 | 2025-11-30 01:06:05 |
| 2192 | 6609739 | Basic 28-Keep Persistent Volumes | Delete NS Instance Test | Delete NS | FAIL | 2025-11-30 01:06:05 | 2025-11-30 01:10:08 |
| 2193 | 6609740 | Basic 28-Keep Persistent Volumes | Check Persistent Root Volume in VIM after NS D... | Variable Should Exist | FAIL | 2025-11-30 01:10:08 | 2025-11-30 01:10:08 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 2850 | 6610510 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2025-11-30 06:00:36 | 2025-11-30 06:14:37 |
| 2853 | 6610513 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2025-11-30 06:16:37 | 2025-11-30 06:16:37 |
| 2855 | 6610515 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VDUs After Auto-scaling | Get Vnf Vdur Names | FAIL | 2025-11-30 06:18:37 | 2025-11-30 06:18:37 |
| 2884 | 6610544 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2025-11-30 06:24:52 | 2025-11-30 06:38:53 |
| 2885 | 6610545 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2025-11-30 06:38:53 | 2025-11-30 06:42:21 |
166 rows × 7 columns
osm-stage_3-merge/v14.0: (full report)
| auto_id | suite_name | test_name | keyword_name | status | starttime | endtime |
|---|
Powered by osm-analytics