Date and time of the report: 2022-11-23 08:20:25.090593
Analysed period: 2022-11-02 to 2022-11-23.
Database update:
Retrieving from database...
DONE
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_result
is the outcome of the build, reported by Jenkins. It can be: SUCCESS
, FAILURE
, UNSTABLE
or ABORTED
.test_result
is the summary of the concerned Robot tests. It can be: FAIL
, UNAVAILABLE
or PASS
.Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
grp_build_result
.grp_test_result
.grp_success_fail
.For the identification of these sequences, the following mapping applies:
Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
DONE
DONE
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
job | Master branch | Release TEN | Release TWELVE | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
suites | ||||||||||||
Basic 01-Crud Operations On Vim Targets | 1.737847 | 18.719109 | 20.456956 | 0.084951 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 05-Instantiation Parameters In Cloud Init | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 06-Vnf With Charm | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.588461 | 19.911551 | 20.500012 | 0.028705 |
Basic 07-Secure Key Management | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.588461 | 19.911551 | 20.500012 | 0.028705 |
Basic 08-Disable Port Security Network Level | 2.386100 | 18.070856 | 20.456956 | 0.116640 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.588461 | 19.911551 | 20.500012 | 0.028705 |
Basic 09-Manual Vdu Scaling | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.588461 | 19.911551 | 20.500012 | 0.028705 |
Basic 11-Native Charms | 1.512627 | 18.944329 | 20.456956 | 0.073942 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.588461 | 19.911551 | 20.500012 | 0.028705 |
Basic 12-Ns Primitives | 0.648252 | 19.808704 | 20.456956 | 0.031689 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.588461 | 19.911551 | 20.500012 | 0.028705 |
Basic 13-Ns Relations | 0.878218 | 19.578738 | 20.456956 | 0.042930 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 14-Vnf Relations | 0.878218 | 19.578738 | 20.456956 | 0.042930 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 15-Rbac Configurations | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 16-Advanced Onboarding And Scaling | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 17-Delete Vnf Package | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 18-Ns Ip Profile | 0.013843 | 20.443113 | 20.456956 | 0.000677 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 19-Vnf Ip Profile | 0.013843 | 20.443113 | 20.456956 | 0.000677 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 20-Manual Native Charm Vdu Scaling | 1.173044 | 19.283912 | 20.456956 | 0.057342 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 21-Persistent Volumes | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Basic 21-Support Of Volumes | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 22-Cross Model Relations | 8.878264 | 11.578692 | 20.456956 | 0.433997 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 3.337153 | 17.162859 | 20.500012 | 0.162788 |
Basic 23-Sol004 Sol007 Packages | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 24-Affinity Groups | 0.634410 | 19.822546 | 20.456956 | 0.031012 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 1.308623 | 19.191389 | 20.500012 | 0.063835 |
Basic 25-Update Charm In Running Vnf Instance | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Basic 27-Update Helm Ee In Running Vnf Instance | 4.456944 | 0.000000 | 4.456944 | 1.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Epa 01-Epa Sriov | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Epa 02-Additional Capabilities | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Epa 03-Crud Operations On Sdnc | 1.737847 | 18.719109 | 20.456956 | 0.084951 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Epa 04-Epa Underlay Sriov | 1.435150 | 19.021806 | 20.456956 | 0.070155 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 8.773773 | 11.726238 | 20.500012 | 0.427989 |
Epa 05-Epa Underlay Passthrough | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
Fail 01-Insufficient Resources | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Hackfest Basic | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Hackfest Cloudinit | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Hackfest Multivdu | 6.351748 | 14.105208 | 20.456956 | 0.310493 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Heal 01-Volume Vdu Healing | 1.878171 | 18.578785 | 20.456956 | 0.091811 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Heal 02-Scale Vdu Healing | 1.512627 | 18.944329 | 20.456956 | 0.073942 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 7.827951 | 12.672060 | 20.500012 | 0.381851 |
Heal 03-Multiple Healing | 7.229965 | 13.226991 | 20.456956 | 0.353423 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.748657 | 19.751354 | 20.500012 | 0.036520 |
Heal 04-Autohealing | 10.089595 | 10.367361 | 20.456956 | 0.493211 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.748657 | 19.751354 | 20.500012 | 0.036520 |
K8S 01-Create K8S Cluster | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
K8S 02-K8Scluster Creation | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
K8S 03-Simple K8S | 1.648299 | 18.808657 | 20.456956 | 0.080574 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 2.588495 | 17.911516 | 20.500012 | 0.126268 |
K8S 04-Openldap Helm | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
K8S 05-K8S Proxy Charms | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
K8S 06-K8S Secure Key Management | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
K8S 07-Dummy Helm | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
K8S 08-Simple K8S Scaling | 1.648299 | 18.808657 | 20.456956 | 0.080574 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 2.588495 | 17.911516 | 20.500012 | 0.126268 |
K8S 09-Pebble Charm K8S | 1.648299 | 18.808657 | 20.456956 | 0.080574 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 2.588495 | 17.911516 | 20.500012 | 0.126268 |
K8S 10-Sol004 Sol007 With K8S Proxy Charms | 6.351748 | 14.105208 | 20.456956 | 0.310493 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
K8S 11-Simple Helm K8S Scaling | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
K8S 12-Openldap Helm Day-2 | 1.648299 | 18.808657 | 20.456956 | 0.080574 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 2.588495 | 17.911516 | 20.500012 | 0.126268 |
Quotas 01-Quota Enforcement | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Sa 01-Vnf With Vim Metrics | 4.194780 | 16.262176 | 20.456956 | 0.205054 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Sa 02-Vnf With Vim Metrics And Autoscaling | 4.194780 | 16.262176 | 20.456956 | 0.205054 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.500000 | 20.000012 | 20.500012 | 0.024390 |
Sa 07-Alarms From Sa-Related Vnfs | 10.546528 | 9.910428 | 20.456956 | 0.515547 | 0.000000 | 20.372234 | 20.372234 | 0.000000 | 1.500023 | 18.999988 | 20.500012 | 0.073172 |
Sa 08-Vnf With Vnf Indicators Snmp | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 2.000012 | 18.372222 | 20.372234 | 0.098173 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Slice 01-Network Slicing | 6.351748 | 14.105208 | 20.456956 | 0.310493 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Slice 02-Shared Network Slicing | 6.351748 | 14.105208 | 20.456956 | 0.310493 | 8.000035 | 12.372199 | 20.372234 | 0.392693 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
Sol003 01-Vnf-Lifecycle-Management | 0.000000 | 20.456956 | 20.456956 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.500012 | 20.500012 | 0.000000 |
basic 23-delete vnf package SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
k8s 10-k8s proxy charms SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
---|---|---|---|---|---|---|---|---|
0 | osm-stage_3-merge/master | 2387 | 2022-11-23 02:06:13 | 24624 | FAILURE | UNAVAILABLE | NaN | NaN |
1 | osm-stage_3-merge/v12.0 | 215 | 2022-11-23 01:04:17 | 16431085 | SUCCESS | FAIL | 514.0 | 4.0 |
2 | osm-stage_3-merge/v11.0 | 332 | 2022-08-08 23:00:14 | 11064864 | SUCCESS | PASS | 421.0 | 0.0 |
3 | osm-stage_3-merge/v10.0 | 633 | 2022-11-23 04:08:17 | 0 | FAILURE | UNAVAILABLE | NaN | NaN |
osm-stage_3-merge/master: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
---|
osm-stage_3-merge/v12.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
121 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2022-11-23 04:29:21 | 2022-11-23 05:00:59 | 10 | 3 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
122 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2022-11-23 05:00:59 | 2022-11-23 05:19:14 | 15 | 1 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v11.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
---|
osm-stage_3-merge/v10.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
---|
osm-stage_3-merge/master: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime |
---|
osm-stage_3-merge/v12.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
20 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2022-11-23 04:30:12 | 2022-11-23 04:44:12 |
27 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2022-11-23 04:44:14 | 2022-11-23 04:58:15 |
32 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VDUs After Auto-scaling | Run Keyword If | FAIL | 2022-11-23 05:00:16 | 2022-11-23 05:00:16 |
58 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2022-11-23 05:03:47 | 2022-11-23 05:17:47 |
osm-stage_3-merge/v11.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime |
---|
osm-stage_3-merge/v10.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime |
---|