Analysis of Robot reports from OSM Jenkins¶

Date and time of the report: 2026-08-23 08:45:04.247103

Analysed period: 2026-08-02 to 2026-08-23.

1. Retrieval of all currrent data for aggregate analytics¶

Database update:

Retrieving new jobs...

Getting new builds from: osm-stage_3-merge/master, osm-stage_3-merge/v19.0, osm-stage_3-merge/v18.0, osm-stage_3-merge/v17.0, osm-stage_3-merge/v16.0, osm-stage_3-merge/v14.0

Retrieving build 172 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 173 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 174 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 175 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 176 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 177 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 178 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 179 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 180 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 181 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 182 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 183 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 184 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 185 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 186 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 187 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 188 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 189 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 190 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 191 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 192 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 193 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 194 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 195 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 196 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 197 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 198 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 199 from "osm-stage_3-merge/v19.0"...	Build: FAILURE	Report unavailable

Retrieving build 335 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 336 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 337 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 338 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 339 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 340 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 341 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 342 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 343 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 344 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 345 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 346 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 347 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 348 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 349 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 350 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 351 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 352 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 353 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 354 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 355 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 356 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 357 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 358 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 359 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 360 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 361 from "osm-stage_3-merge/v18.0"...	Build: FAILURE	Report unavailable

Retrieving build 560 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 561 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 562 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 563 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 564 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 565 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 566 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 567 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 568 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 569 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 570 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 571 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 572 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 573 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 574 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 575 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 576 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 577 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 578 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 579 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 580 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 581 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 582 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 583 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 584 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 585 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 586 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 587 from "osm-stage_3-merge/v17.0"...	Build: FAILURE	Report unavailable

Retrieving build 669 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 670 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 671 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 672 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 673 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 674 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 675 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 676 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 677 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 678 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 679 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 680 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 681 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 682 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 683 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 684 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 685 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 686 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 687 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 688 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 689 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 690 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 691 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 692 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 693 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 694 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 695 from "osm-stage_3-merge/v16.0"...	Build: FAILURE	Report unavailable

Retrieving build 1087 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1088 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1089 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1090 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1091 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1092 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1093 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1094 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1095 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1096 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1097 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1098 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1099 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1100 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1101 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1102 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1103 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1104 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1105 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1106 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1107 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1108 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1109 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1110 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1111 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1112 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1113 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

Retrieving build 1114 from "osm-stage_3-merge/v14.0"...	Build: FAILURE	Report unavailable

DONE

Retrieving from database...

DONE

2. Aggregated analysis of stability¶

2.1 Restricts data to time window for analysis¶

2.2 Finding sequences of successful builds and Robot reports¶

Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:

build_result is the outcome of the build, reported by Jenkins. It can be: SUCCESS, FAILURE, UNSTABLE or ABORTED.
test_result is the summary of the concerned Robot tests. It can be: FAIL, UNAVAILABLE or PASS.

Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:

Successful builds/failed builds in a row: grp_build_result.
Successful test reports vs. test reports with fails in a row: grp_test_result.
Clean builds and tests vs. failures (of any kind) in a row: grp_success_fail.

For the identification of these sequences, the following mapping applies:

Type of sequence	Relevant state	OK sequence contains	NOK sequence contains	Ignore
`grp_build_result`	`build_result`	`SUCCESS` or `UNSTABLE`	`FAILURE`	`ABORTED`
`grp_test_result`	`test_result`	`PASS`	`FAIL`	`UNAVAILABLE`
`grp_success_fail`	`test_result`	`PASS`	`FAIL` or `UNAVAILABLE`	N/A

The different groupings of segments are detected and a label is added to each sample...

DONE

Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...

DONE

2.3 Finding sequences of pass/fails per test suite¶

DONE

3. Reports¶

3.1 Aggregated success rate per test step¶

Empty dataframe. Skipping.

No description has been provided for this image

3.2 Overall success of Jenkins builds and Robot tests¶

3.3 Sequences of pass/fails per test suites¶

Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.
Empty dataframe. Skipping.

3.4 Failing days per test suite¶

Out[58]:

job	Master branch				Release EIGHTEEN				Release FOURTEEN		...	Release NINETEEN		Release SEVENTEEN				Release SIXTEEN
	days_failing	days_passing	days_run	error_rate	days_failing	days_passing	days_run	error_rate	days_failing	days_passing	...	days_run	error_rate	days_failing	days_passing	days_run	error_rate	days_failing	days_passing	days_run	error_rate
suites
Basic 01-Crud Operations On Vim Targets	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Basic 05-Instantiation Parameters In Cloud Init	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Basic 06-Vnf With Charm	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Basic 07-Secure Key Management	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Basic 08-Disable Port Security Network Level	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
Sa 08-Vnf With Vnf Indicators Snmp	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Slice 01-Network Slicing	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Slice 02-Shared Network Slicing	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Sol003 01-Vnf-Lifecycle-Management	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Sol003 02-Dualstack Ip Vnfm	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

62 rows × 24 columns

4. Information about the latest builds of relevant jobs¶

Latest build of each job¶

	auto_id	job	build	timestamp	duration	build_result	test_result	pass_count	fail_count
0	9680	osm-stage_3-merge/master	3742	2026-05-04 23:06:08	15008	FAILURE	UNAVAILABLE	NaN	NaN
1	10091	osm-stage_3-merge/v19.0	199	2026-08-18 13:11:11	146165	FAILURE	UNAVAILABLE	NaN	NaN
2	10118	osm-stage_3-merge/v18.0	361	2026-08-17 21:12:09	65380	FAILURE	UNAVAILABLE	NaN	NaN
3	10146	osm-stage_3-merge/v17.0	587	2026-08-18 16:49:07	22389	FAILURE	UNAVAILABLE	NaN	NaN
4	10173	osm-stage_3-merge/v16.0	695	2026-08-18 01:41:07	288225	FAILURE	UNAVAILABLE	NaN	NaN
5	10201	osm-stage_3-merge/v14.0	1114	2026-08-18 18:08:08	0	FAILURE	UNAVAILABLE	NaN	NaN

Click to see the details of build 3742 of osm-stage_3-merge/master

Click to see the details of build 199 of osm-stage_3-merge/v19.0

Click to see the details of build 361 of osm-stage_3-merge/v18.0

Click to see the details of build 587 of osm-stage_3-merge/v17.0

Click to see the details of build 695 of osm-stage_3-merge/v16.0

Click to see the details of build 1114 of osm-stage_3-merge/v14.0

Failed test suites per job (if any):¶

osm-stage_3-merge/master: (full report)

	auto_id	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

osm-stage_3-merge/v19.0: (full report)

	auto_id	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

osm-stage_3-merge/v18.0: (full report)

	auto_id	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

osm-stage_3-merge/v17.0: (full report)

	auto_id	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

osm-stage_3-merge/v16.0: (full report)

	auto_id	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

osm-stage_3-merge/v14.0: (full report)

	auto_id	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

Details of failed tests into failing test suites (if any):¶