Date and time of the report: 2024-12-30 08:20:23.261236
Analysed period: 2024-12-09 to 2024-12-30.
Database update:
Retrieving from database...
DONE
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_result
is the outcome of the build, reported by Jenkins. It can be: SUCCESS
, FAILURE
, UNSTABLE
or ABORTED
.test_result
is the summary of the concerned Robot tests. It can be: FAIL
, UNAVAILABLE
or PASS
.Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
grp_build_result
.grp_test_result
.grp_success_fail
.For the identification of these sequences, the following mapping applies:
Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
DONE
DONE
job | Master branch | Release FIFTEEN | Release FOURTEEN | Release SIXTEEN | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
suites | ||||||||||||||||
Basic 01-Crud Operations On Vim Targets | 0.000000 | 21.561111 | 21.561111 | 0.000000 | 0.0 | 21.5 | 21.5 | 0.000000 | 0.0 | 20.684722 | 20.684722 | 0.0 | 0.0 | 21.647917 | 21.647917 | 0.0 |
Basic 05-Instantiation Parameters In Cloud Init | 0.000000 | 21.561111 | 21.561111 | 0.000000 | 0.0 | 21.5 | 21.5 | 0.000000 | 0.0 | 20.684722 | 20.684722 | 0.0 | 0.0 | 21.647917 | 21.647917 | 0.0 |
Basic 06-Vnf With Charm | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.0 | 0.000000 | 0.0 | 20.684722 | 20.684722 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 |
Basic 07-Secure Key Management | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.0 | 0.000000 | 0.0 | 20.684722 | 20.684722 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 |
Basic 08-Disable Port Security Network Level | 0.000000 | 21.561111 | 21.561111 | 0.000000 | 0.0 | 21.5 | 21.5 | 0.000000 | 0.0 | 20.684722 | 20.684722 | 0.0 | 0.0 | 21.647917 | 21.647917 | 0.0 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
Slice 02-Shared Network Slicing | 11.584549 | 9.976562 | 21.561111 | 0.537289 | 0.5 | 21.0 | 21.5 | 0.023256 | 0.0 | 20.684722 | 20.684722 | 0.0 | 0.0 | 21.647917 | 21.647917 | 0.0 |
Sol003 01-Vnf-Lifecycle-Management | 0.000000 | 21.561111 | 21.561111 | 0.000000 | 0.0 | 21.5 | 21.5 | 0.000000 | 0.0 | 20.684722 | 20.684722 | 0.0 | 0.0 | 21.647917 | 21.647917 | 0.0 |
Sol003 02-Dualstack Ip Vnfm | 0.000000 | 21.561111 | 21.561111 | 0.000000 | 0.0 | 21.5 | 21.5 | 0.000000 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 21.647917 | 21.647917 | 0.0 |
basic 23-delete vnf package SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.0 | 0.000000 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 |
k8s 10-k8s proxy charms SOL004 SOL007 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.0 | 0.000000 | 0.0 | 0.000000 | 0.000000 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.0 |
67 rows × 16 columns
job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
---|---|---|---|---|---|---|---|---|
0 | osm-stage_3-merge/master | 3226 | 2024-12-30 02:06:15 | 12104288 | UNSTABLE | FAIL | 465.0 | 7.0 |
1 | osm-stage_3-merge/v16.0 | 106 | 2024-12-30 00:01:16 | 12622184 | UNSTABLE | FAIL | 465.0 | 7.0 |
2 | osm-stage_3-merge/v15.0 | 364 | 2024-12-30 03:34:15 | 12369719 | UNSTABLE | FAIL | 445.0 | 10.0 |
3 | osm-stage_3-merge/v14.0 | 519 | 2024-12-29 23:08:19 | 18562735 | UNSTABLE | FAIL | 596.0 | 7.0 |
osm-stage_3-merge/master: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
165 | Sa 01-Vnf With Vim Metrics | FAIL | 2024-12-30 03:58:39 | 2024-12-30 04:14:00 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
166 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2024-12-30 04:14:00 | 2024-12-30 04:47:33 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
167 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2024-12-30 04:47:33 | 2024-12-30 05:08:28 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v16.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
309 | Sa 01-Vnf With Vim Metrics | FAIL | 2024-12-30 02:01:28 | 2024-12-30 02:16:50 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
310 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2024-12-30 02:16:50 | 2024-12-30 02:50:22 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
311 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2024-12-30 02:50:22 | 2024-12-30 03:11:21 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v15.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
265 | Sa 01-Vnf With Vim Metrics | FAIL | 2024-12-30 05:29:10 | 2024-12-30 05:44:29 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
266 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2024-12-30 05:44:29 | 2024-12-30 06:18:31 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
267 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2024-12-30 06:18:31 | 2024-12-30 06:40:29 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
270 | Slice 02-Shared Network Slicing | FAIL | 2024-12-30 06:48:29 | 2024-12-30 06:54:36 | 17 | 3 | Network Slice First Instance | Create Network Slice |
osm-stage_3-merge/v14.0: (full report)
name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
---|---|---|---|---|---|---|---|---|
223 | Sa 01-Vnf With Vim Metrics | FAIL | 2024-12-30 02:48:29 | 2024-12-30 03:03:47 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
224 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2024-12-30 03:03:47 | 2024-12-30 03:37:17 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
225 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2024-12-30 03:37:17 | 2024-12-30 03:58:09 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/master: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
10 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2024-12-30 03:59:21 | 2024-12-30 04:13:22 |
40 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2024-12-30 04:14:49 | 2024-12-30 04:28:50 |
45 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2024-12-30 04:28:52 | 2024-12-30 04:42:53 |
48 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2024-12-30 04:44:53 | 2024-12-30 04:44:53 |
79 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2024-12-30 04:49:42 | 2024-12-30 05:03:42 |
80 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2024-12-30 05:03:42 | 2024-12-30 05:07:12 |
osm-stage_3-merge/v16.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
320 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2024-12-30 02:02:11 | 2024-12-30 02:16:12 |
350 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2024-12-30 02:17:39 | 2024-12-30 02:31:40 |
355 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2024-12-30 02:31:42 | 2024-12-30 02:45:43 |
358 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2024-12-30 02:47:43 | 2024-12-30 02:47:43 |
389 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2024-12-30 02:52:30 | 2024-12-30 03:06:31 |
390 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2024-12-30 03:06:31 | 2024-12-30 03:10:01 |
osm-stage_3-merge/v15.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
192 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2024-12-30 05:29:52 | 2024-12-30 05:43:52 |
222 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2024-12-30 05:45:50 | 2024-12-30 05:59:50 |
227 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2024-12-30 05:59:53 | 2024-12-30 06:13:53 |
230 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2024-12-30 06:15:53 | 2024-12-30 06:15:53 |
261 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2024-12-30 06:21:44 | 2024-12-30 06:35:45 |
262 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2024-12-30 06:35:45 | 2024-12-30 06:39:16 |
275 | Slice 02-Shared Network Slicing | Network Slice First Instance | Create Network Slice | FAIL | 2024-12-30 06:48:39 | 2024-12-30 06:49:44 |
286 | Slice 02-Shared Network Slicing | Get First Slice Vnf IPs | Get Slice Vnf Ip Addresses | FAIL | 2024-12-30 06:50:56 | 2024-12-30 06:51:00 |
293 | Slice 02-Shared Network Slicing | Test First Slice Connectivity | Ping Many | FAIL | 2024-12-30 06:52:34 | 2024-12-30 06:52:34 |
osm-stage_3-merge/v14.0: (full report)
suite_name | test_name | keyword_name | status | starttime | endtime | |
---|---|---|---|---|---|---|
98 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2024-12-30 02:49:11 | 2024-12-30 03:03:11 |
131 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2024-12-30 03:04:37 | 2024-12-30 03:18:37 |
138 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2024-12-30 03:18:39 | 2024-12-30 03:32:40 |
141 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2024-12-30 03:34:40 | 2024-12-30 03:34:40 |
146 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VDUs After Auto-scaling | Run Keyword If | FAIL | 2024-12-30 03:36:40 | 2024-12-30 03:36:40 |
173 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2024-12-30 03:39:25 | 2024-12-30 03:53:25 |
175 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2024-12-30 03:53:25 | 2024-12-30 03:56:55 |
Powered by osm-analytics