Empty dataframe. Skipping.
Empty dataframe. Skipping.
Date and time of the report: 2024-02-25 08:20:23.622976
Analysed period: 2024-02-04 to 2024-02-25.
Database update:
Retrieving from database...
DONE
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_result is the outcome of the build, reported by Jenkins. It can be: SUCCESS, FAILURE, UNSTABLE or ABORTED.test_result is the summary of the concerned Robot tests. It can be: FAIL, UNAVAILABLE or PASS.Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
grp_build_result.grp_test_result.grp_success_fail.For the identification of these sequences, the following mapping applies:
| Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
|---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
DONE
DONE
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
| job | Master branch | Release FIFTEEN | Release FOURTEEN | Release TWELVE | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
| suites | ||||||||||||||||
| Basic 01-Crud Operations On Vim Targets | 0.000000 | 21.500289 | 21.500289 | 0.000000 | 0.000000 | 21.439201 | 21.439201 | 0.000000 | 0.000000 | 20.623935 | 20.623935 | 0.000000 | 0.000000 | 21.543356 | 21.543356 | 0.000000 |
| Basic 05-Instantiation Parameters In Cloud Init | 0.000000 | 21.500289 | 21.500289 | 0.000000 | 0.000000 | 21.439201 | 21.439201 | 0.000000 | 0.000000 | 20.623935 | 20.623935 | 0.000000 | 0.000000 | 21.543356 | 21.543356 | 0.000000 |
| Basic 06-Vnf With Charm | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 4.623646 | 16.000289 | 20.623935 | 0.224188 | 7.543148 | 14.000208 | 21.543356 | 0.350138 |
| Basic 07-Secure Key Management | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 8.623657 | 12.000278 | 20.623935 | 0.418138 | 7.543194 | 14.000162 | 21.543356 | 0.350140 |
| Basic 08-Disable Port Security Network Level | 9.500139 | 12.000150 | 21.500289 | 0.441861 | 0.000000 | 21.439201 | 21.439201 | 0.000000 | 9.623738 | 11.000197 | 20.623935 | 0.466630 | 9.543183 | 12.000174 | 21.543356 | 0.442976 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| Slice 02-Shared Network Slicing | 8.000104 | 13.500185 | 21.500289 | 0.372093 | 0.000000 | 21.439201 | 21.439201 | 0.000000 | 8.623738 | 12.000197 | 20.623935 | 0.418142 | 8.543183 | 13.000174 | 21.543356 | 0.396558 |
| Sol003 01-Vnf-Lifecycle-Management | 0.000000 | 21.500289 | 21.500289 | 0.000000 | 0.000000 | 21.439201 | 21.439201 | 0.000000 | 0.000000 | 20.623935 | 20.623935 | 0.000000 | 0.000000 | 21.543356 | 21.543356 | 0.000000 |
| Sol003 02-Dualstack Ip Vnfm | 0.000000 | 21.500289 | 21.500289 | 0.000000 | 2.000081 | 19.439120 | 21.439201 | 0.093291 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| basic 23-delete vnf package SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| k8s 10-k8s proxy charms SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
66 rows × 16 columns
| job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
|---|---|---|---|---|---|---|---|---|
| 0 | osm-stage_3-merge/master | 2905 | 2024-02-25 02:07:00 | 13852799 | UNSTABLE | FAIL | 403.0 | 48.0 |
| 1 | osm-stage_3-merge/v15.0 | 70 | 2024-02-25 03:34:57 | 0 | FAILURE | UNAVAILABLE | NaN | NaN |
| 2 | osm-stage_3-merge/v14.0 | 246 | 2024-02-24 23:08:57 | 0 | FAILURE | UNAVAILABLE | NaN | NaN |
| 3 | osm-stage_3-merge/v13.0 | 446 | 2024-01-31 21:07:31 | 15541814 | SUCCESS | PASS | 558.0 | 0.0 |
| 4 | osm-stage_3-merge/v12.0 | 663 | 2024-02-25 01:04:59 | 18683535 | UNSTABLE | FAIL | 431.0 | 99.0 |
| 5 | osm-stage_3-merge/v10.0 | 1053 | 2024-02-01 04:08:33 | 2336073 | FAILURE | UNAVAILABLE | NaN | NaN |
osm-stage_3-merge/master: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|
| 131 | Basic 08-Disable Port Security Network Level | FAIL | 2024-02-25 02:39:27 | 2024-02-25 02:47:08 | 8 | 1 | Instantiate Network Service | Create Network Service |
| 143 | Basic 29-Vnf Ipv6 Profile | FAIL | 2024-02-25 03:11:08 | 2024-02-25 03:17:46 | 5 | 4 | Instantiate Network Service | Create Network Service |
| 144 | Basic 30-Ns Ipv6 Profile | FAIL | 2024-02-25 03:17:46 | 2024-02-25 03:24:57 | 5 | 7 | Instantiate Network Service | Create Network Service |
| 146 | Epa 01-Epa Sriov | FAIL | 2024-02-25 03:28:17 | 2024-02-25 03:35:02 | 5 | 3 | Instantiate Network Service | Create Network Service |
| 152 | Hackfest Cloudinit | FAIL | 2024-02-25 03:48:04 | 2024-02-25 03:54:55 | 5 | 4 | Network Service Instance Test | Create Network Service |
| 153 | Hackfest Multivdu | FAIL | 2024-02-25 03:54:55 | 2024-02-25 04:02:02 | 5 | 4 | Network Service Instance Test | Create Network Service |
| 155 | Heal 02-Scale Vdu Healing | FAIL | 2024-02-25 04:06:50 | 2024-02-25 04:14:18 | 7 | 9 | Network Service Instance Test | Create Network Service |
| 156 | Heal 03-Multiple Healing | FAIL | 2024-02-25 04:14:18 | 2024-02-25 04:21:40 | 6 | 10 | Network Service Instance Test | Create Network Service |
| 167 | Sa 08-Vnf With Vnf Indicators Snmp | FAIL | 2024-02-25 05:17:57 | 2024-02-25 05:32:45 | 5 | 2 | Instantiate Network Service | Create Network Service |
| 168 | Slice 01-Network Slicing | FAIL | 2024-02-25 05:32:45 | 2024-02-25 05:42:45 | 9 | 4 | Network Slice Instance Test | Create Network Slice |
osm-stage_3-merge/v15.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
|---|
osm-stage_3-merge/v14.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
|---|
osm-stage_3-merge/v13.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
|---|
osm-stage_3-merge/v12.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|
| 174 | Basic 06-Vnf With Charm | FAIL | 2024-02-25 01:42:21 | 2024-02-25 01:49:13 | 5 | 6 | Instantiate Charm Network Service | Create Network Service |
| 175 | Basic 07-Secure Key Management | FAIL | 2024-02-25 01:49:13 | 2024-02-25 01:57:07 | 7 | 5 | Instantiate Nopasswd Charm Network Service | Create Network Service |
| 176 | Basic 08-Disable Port Security Network Level | FAIL | 2024-02-25 01:57:07 | 2024-02-25 02:04:08 | 8 | 1 | Instantiate Network Service | Create Network Service |
| 178 | Basic 11-Native Charms | FAIL | 2024-02-25 02:06:26 | 2024-02-25 02:13:45 | 5 | 4 | Instantiate Charm Network Service | Create Network Service |
| 179 | Basic 12-Ns Primitives | FAIL | 2024-02-25 02:13:45 | 2024-02-25 02:21:07 | 6 | 1 | Instantiate NS | Create Network Service |
| 181 | Basic 14-Vnf Relations | FAIL | 2024-02-25 02:25:28 | 2024-02-25 02:41:54 | 5 | 1 | Instantiate Charm Network Service | Create Network Service |
| 187 | Basic 20-Manual Native Charm Vdu Scaling | FAIL | 2024-02-25 02:50:31 | 2024-02-25 02:57:11 | 10 | 8 | Instantiate Network Service | Create Network Service |
| 192 | Basic 25-Update Charm In Running Vnf Instance | FAIL | 2024-02-25 03:10:12 | 2024-02-25 03:17:06 | 6 | 10 | Instantiate Charm Network Service | Create Network Service |
| 193 | Epa 01-Epa Sriov | FAIL | 2024-02-25 03:17:06 | 2024-02-25 03:23:51 | 5 | 3 | Instantiate Network Service | Create Network Service |
| 199 | Hackfest Cloudinit | FAIL | 2024-02-25 03:37:10 | 2024-02-25 03:43:44 | 5 | 4 | Network Service Instance Test | Create Network Service |
| 200 | Hackfest Multivdu | FAIL | 2024-02-25 03:43:44 | 2024-02-25 03:50:48 | 5 | 4 | Network Service Instance Test | Create Network Service |
| 201 | Heal 01-Volume Vdu Healing | FAIL | 2024-02-25 03:50:48 | 2024-02-25 03:57:33 | 6 | 10 | Network Service Instance Test | Create Network Service |
| 202 | Heal 02-Scale Vdu Healing | FAIL | 2024-02-25 03:57:33 | 2024-02-25 04:04:17 | 6 | 9 | Network Service Instance Test | Create Network Service |
| 203 | Heal 03-Multiple Healing | FAIL | 2024-02-25 04:04:17 | 2024-02-25 04:10:28 | 6 | 10 | Network Service Instance Test | Create Network Service |
| 209 | K8S 06-K8S Secure Key Management | FAIL | 2024-02-25 04:29:58 | 2024-02-25 04:39:02 | 7 | 5 | Instantiate Charm Network Service | Create Network Service |
| 213 | K8S 10-Sol004 Sol007 With K8S Proxy Charms | FAIL | 2024-02-25 04:52:39 | 2024-02-25 05:00:42 | 5 | 6 | Instantiate Charm Network Service | Create Network Service |
| 219 | Sa 08-Vnf With Vnf Indicators Snmp | FAIL | 2024-02-25 05:28:34 | 2024-02-25 05:43:50 | 5 | 2 | Instantiate Network Service | Create Network Service |
| 220 | Slice 01-Network Slicing | FAIL | 2024-02-25 05:43:50 | 2024-02-25 05:53:50 | 9 | 4 | Network Slice Instance Test | Create Network Slice |
| 221 | Slice 02-Shared Network Slicing | FAIL | 2024-02-25 05:53:50 | 2024-02-25 06:11:22 | 14 | 6 | Network Slice First Instance | Create Network Slice |
osm-stage_3-merge/v10.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
|---|
osm-stage_3-merge/master: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|
| 5 | Basic 08-Disable Port Security Network Level | Instantiate Network Service | Create Network Service | FAIL | 2024-02-25 02:40:05 | 2024-02-25 02:45:07 |
| 15 | Basic 29-Vnf Ipv6 Profile | Instantiate Network Service | Create Network Service | FAIL | 2024-02-25 03:11:12 | 2024-02-25 03:16:49 |
| 17 | Basic 29-Vnf Ipv6 Profile | Get Vnf Info | Variable Should Exist | FAIL | 2024-02-25 03:16:49 | 2024-02-25 03:16:49 |
| 24 | Basic 29-Vnf Ipv6 Profile | Check Vnf IPs | Variable Should Exist | FAIL | 2024-02-25 03:16:49 | 2024-02-25 03:16:49 |
| 31 | Basic 29-Vnf Ipv6 Profile | Check That Ipv6 Address_mode And Ra_mode Are Set | Run And Return Rc And Output | FAIL | 2024-02-25 03:16:49 | 2024-02-25 03:16:49 |
| 45 | Basic 30-Ns Ipv6 Profile | Instantiate Network Service | Create Network Service | FAIL | 2024-02-25 03:17:50 | 2024-02-25 03:23:24 |
| 47 | Basic 30-Ns Ipv6 Profile | Get Vnfs Info | Variable Should Exist | FAIL | 2024-02-25 03:23:24 | 2024-02-25 03:23:24 |
| 58 | Basic 30-Ns Ipv6 Profile | Check Vnf1 IPs | Variable Should Exist | FAIL | 2024-02-25 03:23:24 | 2024-02-25 03:23:24 |
| 63 | Basic 30-Ns Ipv6 Profile | Check Vnf2 IPs | Variable Should Exist | FAIL | 2024-02-25 03:23:24 | 2024-02-25 03:23:24 |
| 68 | Basic 30-Ns Ipv6 Profile | Verify Vnf1 Interfaces | Variable Should Exist | FAIL | 2024-02-25 03:23:24 | 2024-02-25 03:23:24 |
| 75 | Basic 30-Ns Ipv6 Profile | Verify Vnf2 Interfaces | Variable Should Exist | FAIL | 2024-02-25 03:23:24 | 2024-02-25 03:23:24 |
| 81 | Basic 30-Ns Ipv6 Profile | Check that ipv6 address_mode and ra_mode are set | Run And Return Rc And Output | FAIL | 2024-02-25 03:23:24 | 2024-02-25 03:23:24 |
| 95 | Epa 01-Epa Sriov | Instantiate Network Service | Create Network Service | FAIL | 2024-02-25 03:28:20 | 2024-02-25 03:33:54 |
| 97 | Epa 01-Epa Sriov | Get VNF IP Address | Get Vnf Management Ip Address | FAIL | 2024-02-25 03:33:54 | 2024-02-25 03:33:54 |
| 101 | Epa 01-Epa Sriov | Check SR-IOV Interface | Execute Remote Command Check Rc Return Output | FAIL | 2024-02-25 03:34:24 | 2024-02-25 03:34:24 |
| 107 | Hackfest Cloudinit | Network Service Instance Test | Create Network Service | FAIL | 2024-02-25 03:48:07 | 2024-02-25 03:53:10 |
| 109 | Hackfest Cloudinit | Get Vnf Ip Address | Get Vnf Management Ip Address | FAIL | 2024-02-25 03:53:10 | 2024-02-25 03:53:10 |
| 113 | Hackfest Cloudinit | Test SSH Access | Test SSH Connection | FAIL | 2024-02-25 03:53:40 | 2024-02-25 03:53:40 |
| 114 | Hackfest Cloudinit | Check Remote File Injected Via Cloud-init | Execute Remote Command Check Rc Return Output | FAIL | 2024-02-25 03:53:40 | 2024-02-25 03:53:40 |
| 123 | Hackfest Multivdu | Network Service Instance Test | Create Network Service | FAIL | 2024-02-25 03:54:59 | 2024-02-25 04:00:34 |
| 126 | Hackfest Multivdu | Get Vnf Ip Address | Get Vnf Management Ip Address | FAIL | 2024-02-25 04:00:34 | 2024-02-25 04:00:34 |
| 129 | Hackfest Multivdu | Test Ping | Test Connectivity | FAIL | 2024-02-25 04:00:34 | 2024-02-25 04:01:04 |
| 132 | Hackfest Multivdu | Test SSH Access | Test SSH Connection | FAIL | 2024-02-25 04:01:04 | 2024-02-25 04:01:04 |
| 144 | Heal 02-Scale Vdu Healing | Network Service Instance Test | Create Network Service | FAIL | 2024-02-25 04:06:56 | 2024-02-25 04:13:10 |
| 147 | Heal 02-Scale Vdu Healing | Scale Out Manual Scale VNF | Get Vnf Id | FAIL | 2024-02-25 04:13:10 | 2024-02-25 04:13:10 |
| 155 | Heal 02-Scale Vdu Healing | Get VIM Objects | Variable Should Exist | FAIL | 2024-02-25 04:13:10 | 2024-02-25 04:13:10 |
| 160 | Heal 02-Scale Vdu Healing | Get Manual Scale VNF Info | Variable Should Exist | FAIL | 2024-02-25 04:13:10 | 2024-02-25 04:13:10 |
| 168 | Heal 02-Scale Vdu Healing | Halt Manual Scale VDU | Variable Should Exist | FAIL | 2024-02-25 04:13:10 | 2024-02-25 04:13:10 |
| 171 | Heal 02-Scale Vdu Healing | Heal Manual Scale VDU | Variable Should Exist | FAIL | 2024-02-25 04:13:10 | 2024-02-25 04:13:10 |
| 173 | Heal 02-Scale Vdu Healing | Check VNF After Healing | Variable Should Exist | FAIL | 2024-02-25 04:13:11 | 2024-02-25 04:13:11 |
| 187 | Heal 02-Scale Vdu Healing | Update VIM Objects | Variable Should Exist | FAIL | 2024-02-25 04:13:11 | 2024-02-25 04:13:11 |
| 200 | Heal 03-Multiple Healing | Network Service Instance Test | Create Network Service | FAIL | 2024-02-25 04:14:22 | 2024-02-25 04:20:36 |
| 203 | Heal 03-Multiple Healing | Scale Out Manual Scale VNF | Get Vnf Id | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 211 | Heal 03-Multiple Healing | Get VIM Objects | Variable Should Exist | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 216 | Heal 03-Multiple Healing | Get Manual Scale VNF Info | Variable Should Exist | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 224 | Heal 03-Multiple Healing | Get Volume VNF Info | Variable Should Exist | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 236 | Heal 03-Multiple Healing | Stop VDUs | Variable Should Exist | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 242 | Heal 03-Multiple Healing | Heal VDUs | Variable Should Exist | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 245 | Heal 03-Multiple Healing | Check VNFs After Healing | Variable Should Exist | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 258 | Heal 03-Multiple Healing | Update VIM Objects | Variable Should Exist | FAIL | 2024-02-25 04:20:36 | 2024-02-25 04:20:36 |
| 270 | Sa 08-Vnf With Vnf Indicators Snmp | Instantiate Network Service | Create Network Service | FAIL | 2024-02-25 05:18:02 | 2024-02-25 05:23:05 |
| 276 | Sa 08-Vnf With Vnf Indicators Snmp | Get VNF SNMP Metrics | Wait Until Keyword Succeeds | FAIL | 2024-02-25 05:23:05 | 2024-02-25 05:32:05 |
| 286 | Slice 01-Network Slicing | Network Slice Instance Test | Create Network Slice | FAIL | 2024-02-25 05:32:55 | 2024-02-25 05:39:06 |
| 298 | Slice 01-Network Slicing | Test Middle Ns Ping | Test Connectivity | FAIL | 2024-02-25 05:40:17 | 2024-02-25 05:40:48 |
| 300 | Slice 01-Network Slicing | Test Middle Vnf SSH Access | Test SSH Connection | FAIL | 2024-02-25 05:41:18 | 2024-02-25 05:41:21 |
| 301 | Slice 01-Network Slicing | Test Slice Connectivity | Ping Many | FAIL | 2024-02-25 05:41:21 | 2024-02-25 05:41:21 |
osm-stage_3-merge/v15.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime |
|---|
osm-stage_3-merge/v14.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime |
|---|
osm-stage_3-merge/v13.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime |
|---|
osm-stage_3-merge/v12.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|
| 310 | Basic 06-Vnf With Charm | Instantiate Charm Network Service | Create Network Service | FAIL | 2024-02-25 01:42:25 | 2024-02-25 01:48:33 |
| 312 | Basic 06-Vnf With Charm | Get Management Ip Addresses | Get Vnf Management Ip Address | FAIL | 2024-02-25 01:48:33 | 2024-02-25 01:48:33 |
| 318 | Basic 06-Vnf With Charm | Test SSH Access | Variable Should Exist | FAIL | 2024-02-25 01:48:33 | 2024-02-25 01:48:33 |
| 323 | Basic 06-Vnf With Charm | Check Remote Files Created Via Day 1 Operations | Check If remote File Exists | FAIL | 2024-02-25 01:48:33 | 2024-02-25 01:48:33 |
| 325 | Basic 06-Vnf With Charm | Execute Day 2 Operations | Variable Should Exist | FAIL | 2024-02-25 01:48:33 | 2024-02-25 01:48:33 |
| ... | ... | ... | ... | ... | ... | ... |
| 818 | Slice 02-Shared Network Slicing | Network Slice Second Instance | Create Network Slice | FAIL | 2024-02-25 06:00:14 | 2024-02-25 06:06:26 |
| 832 | Slice 02-Shared Network Slicing | Test Middle Ns Ping | Test Connectivity | FAIL | 2024-02-25 06:07:44 | 2024-02-25 06:08:14 |
| 834 | Slice 02-Shared Network Slicing | Test Middle Vnf SSH Access | Test SSH Connection | FAIL | 2024-02-25 06:08:44 | 2024-02-25 06:08:47 |
| 835 | Slice 02-Shared Network Slicing | Test First Slice Connectivity | Ping Many | FAIL | 2024-02-25 06:08:47 | 2024-02-25 06:08:47 |
| 844 | Slice 02-Shared Network Slicing | Test Second Slice Connectivity | Ping Many | FAIL | 2024-02-25 06:09:44 | 2024-02-25 06:09:47 |
96 rows × 6 columns
osm-stage_3-merge/v10.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime |
|---|
Powered by osm-analytics