Date and time of the report: 2023-03-14 08:20:44.111727
Analysed period: 2023-02-21 to 2023-03-14.
Database update:
Retrieving from database...
DONE
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_result
is the outcome of the build, reported by Jenkins. It can be: SUCCESS
, FAILURE
, UNSTABLE
or ABORTED
.test_result
is the summary of the concerned Robot tests. It can be: FAIL
, UNAVAILABLE
or PASS
.Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
grp_build_result
.grp_test_result
.grp_success_fail
.For the identification of these sequences, the following mapping applies:
Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
DONE
DONE
job | Master branch | Release TEN | Release THIRTEEN | Release TWELVE | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
suites | ||||||||||||||||
Basic 01-Crud Operations On Vim Targets | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 05-Instantiation Parameters In Cloud Init | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 06-Vnf With Charm | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 1.000000 | 19.708970 | 20.70897 | 0.048288 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 07-Secure Key Management | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 1.000000 | 19.708970 | 20.70897 | 0.048288 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 08-Disable Port Security Network Level | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 09-Manual Vdu Scaling | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 11-Native Charms | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 12-Ns Primitives | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 1.000000 | 19.708970 | 20.70897 | 0.048288 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 13-Ns Relations | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 14-Vnf Relations | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 15-Rbac Configurations | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 16-Advanced Onboarding And Scaling | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 17-Delete Vnf Package | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 18-Ns Ip Profile | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 19-Vnf Ip Profile | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 20-Manual Native Charm Vdu Scaling | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 1.000000 | 20.499931 | 21.499931 | 0.046512 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 21-Persistent Volumes | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
Basic 21-Support Of Volumes | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 22-Cross Model Relations | 5.000000 | 16.584606 | 21.584606 | 0.231647 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 2.999988 | 17.708981 | 20.70897 | 0.144864 | 2.999988 | 18.627731 | 21.62772 | 0.138710 |
Basic 23-Sol004 Sol007 Packages | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 24-Affinity Groups | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 25-Update Charm In Running Vnf Instance | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.000000 | 19.708970 | 20.70897 | 0.048288 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Basic 26-Secure Helm Execution Environment | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
Basic 27-Update Helm Ee In Running Vnf Instance | 1.000000 | 20.584606 | 21.584606 | 0.046329 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 5.999988 | 14.708981 | 20.70897 | 0.289729 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
Basic 28-Keep Persistent Volumes | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
Epa 01-Epa Sriov | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Epa 02-Additional Capabilities | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Epa 03-Crud Operations On Sdnc | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Epa 04-Epa Underlay Sriov | 3.999907 | 17.584699 | 21.584606 | 0.185313 | 2.999896 | 18.500035 | 21.499931 | 0.139530 | 1.000000 | 19.708970 | 20.70897 | 0.048288 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Epa 05-Epa Underlay Passthrough | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
Fail 01-Insufficient Resources | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Hackfest Basic | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Hackfest Cloudinit | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Hackfest Multivdu | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Heal 01-Volume Vdu Healing | 4.000000 | 17.584606 | 21.584606 | 0.185317 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.627789 | 20.999931 | 21.62772 | 0.029027 |
Heal 02-Scale Vdu Healing | 1.584745 | 19.999861 | 21.584606 | 0.073420 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Heal 03-Multiple Healing | 1.000012 | 20.584595 | 21.584606 | 0.046330 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Heal 04-Autohealing | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
K8S 01-Create K8S Cluster | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
K8S 02-K8Scluster Creation | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
K8S 03-Simple K8S | 4.000000 | 17.584606 | 21.584606 | 0.185317 | 4.000000 | 17.499931 | 21.499931 | 0.186047 | 2.999988 | 17.708981 | 20.70897 | 0.144864 | 4.000000 | 17.627720 | 21.62772 | 0.184948 |
K8S 04-Openldap Helm | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
K8S 05-K8S Proxy Charms | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
K8S 06-K8S Secure Key Management | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
K8S 07-Dummy Helm | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
K8S 08-Simple K8S Scaling | 4.000000 | 17.584606 | 21.584606 | 0.185317 | 4.000000 | 17.499931 | 21.499931 | 0.186047 | 2.999988 | 17.708981 | 20.70897 | 0.144864 | 4.000000 | 17.627720 | 21.62772 | 0.184948 |
K8S 09-Pebble Charm K8S | 4.000000 | 17.584606 | 21.584606 | 0.185317 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 2.999988 | 17.708981 | 20.70897 | 0.144864 | 4.000000 | 17.627720 | 21.62772 | 0.184948 |
K8S 10-Sol004 Sol007 With K8S Proxy Charms | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
K8S 11-Simple Helm K8S Scaling | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
K8S 12-Openldap Helm Day-2 | 5.000012 | 16.584595 | 21.584606 | 0.231647 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 2.999988 | 17.708981 | 20.70897 | 0.144864 | 4.000000 | 17.627720 | 21.62772 | 0.184948 |
Quotas 01-Quota Enforcement | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Sa 01-Vnf With Vim Metrics | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Sa 02-Vnf With Vim Metrics And Autoscaling | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Sa 07-Alarms From Sa-Related Vnfs | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Sa 08-Vnf With Vnf Indicators Snmp | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Slice 01-Network Slicing | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Slice 02-Shared Network Slicing | 0.000000 | 21.584606 | 21.584606 | 0.000000 | 0.000000 | 21.499931 | 21.499931 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
Sol003 01-Vnf-Lifecycle-Management | 2.999954 | 18.584653 | 21.584606 | 0.138986 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 20.708970 | 20.70897 | 0.000000 | 0.000000 | 21.627720 | 21.62772 | 0.000000 |
basic 23-delete vnf package SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
k8s 10-k8s proxy charms SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 | 0.000000 | 0.000000 | 0.00000 | 0.000000 |
job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
---|---|---|---|---|---|---|---|---|
0 | osm-stage_3-merge/master | 2513 | 2023-03-14 02:06:11 | 15271516 | SUCCESS | FAIL | 567.0 | 1.0 |
1 | osm-stage_3-merge/v13.0 | 137 | 2023-03-13 23:07:11 | 15416210 | SUCCESS | PASS | 558.0 | 0.0 |
2 | osm-stage_3-merge/v12.0 | 331 | 2023-03-14 01:04:11 | 15135529 | SUCCESS | FAIL | 526.0 | 4.0 |
3 | osm-stage_3-merge/v10.0 | 749 | 2023-03-14 04:08:12 | 9516669 | SUCCESS | PASS | 380.0 | 0.0 |
osm-stage_3-merge/master: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
109 | Heal 02-Scale Vdu Healing | FAIL | 2023-03-14 04:26:50 | 2023-03-14 04:39:27 | 14 | 1 | Scale Out Charm VNF | Execute Manual VNF Scale |
osm-stage_3-merge/v13.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
---|
osm-stage_3-merge/v12.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
198 | Heal 01-Volume Vdu Healing | FAIL | 2023-03-14 03:04:13 | 2023-03-14 03:11:24 | 12 | 4 | Heal Persistent Volume VDU | Heal Network Service |
osm-stage_3-merge/v10.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
---|
osm-stage_3-merge/master: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
11 | Heal 02-Scale Vdu Healing | Scale Out Charm VNF | Execute Manual VNF Scale | FAIL | 2023-03-14 04:30:57 | 2023-03-14 04:31:49 |
osm-stage_3-merge/v13.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime |
---|
osm-stage_3-merge/v12.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
97 | Heal 01-Volume Vdu Healing | Heal Persistent Volume VDU | Heal Network Service | FAIL | 2023-03-14 03:09:56 | 2023-03-14 03:10:00 |
104 | Heal 01-Volume Vdu Healing | Check VNF After Healing | Should Not Be Equal | FAIL | 2023-03-14 03:10:04 | 2023-03-14 03:10:04 |
osm-stage_3-merge/v10.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime |
---|