Analysis of Robot reports from OSM Jenkins

Date and time of the report: 2023-01-13 08:20:31.746870

Analysed period: 2022-12-23 to 2023-01-13.

1. Retrieval of all currrent data for aggregate analytics

Database update:

Retrieving from database...	

DONE

2. Aggregated analysis of stability

2.1 Restricts data to time window for analysis

2.2 Finding sequences of successful builds and Robot reports

Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:

• build_result is the outcome of the build, reported by Jenkins. It can be: SUCCESS, FAILURE, UNSTABLE or ABORTED.
• test_result is the summary of the concerned Robot tests. It can be: FAIL, UNAVAILABLE or PASS.

Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:

1. Successful builds/failed builds in a row: grp_build_result.
2. Successful test reports vs. test reports with fails in a row: grp_test_result.
3. Clean builds and tests vs. failures (of any kind) in a row: grp_success_fail.

For the identification of these sequences, the following mapping applies:

Type of sequence	Relevant state	OK sequence contains	NOK sequence contains	Ignore
grp_build_result	build_result	SUCCESS or UNSTABLE	FAILURE	ABORTED
grp_test_result	test_result	PASS	FAIL	UNAVAILABLE
grp_success_fail	test_result	PASS	FAIL or UNAVAILABLE	N/A

The different groupings of segments are detected and a label is added to each sample...

DONE

Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...

DONE

2.3 Finding sequences of pass/fails per test suite

DONE

3. Reports

3.1 Aggregated success rate per test step

3.2 Overall success of Jenkins builds and Robot tests

3.3 Sequences of pass/fails per test suites

3.4 Failing days per test suite

job	Master branch				Release TEN				Release THIRTEEN				Release TWELVE
	days_failing	days_passing	days_run	error_rate	days_failing	days_passing	days_run	error_rate	days_failing	days_passing	days_run	error_rate	days_failing	days_passing	days_run	error_rate
suites
Basic 01-Crud Operations On Vim Targets	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
Basic 05-Instantiation Parameters In Cloud Init	7.999907	13.456968	21.456875	0.372837	7.999907	13.372245	21.372153	0.374315	0.999988	19.581204	20.581192	0.048587	5.999907	15.500035	21.499942	0.279066
Basic 06-Vnf With Charm	8.999907	12.456968	21.456875	0.419442	8.999896	12.372257	21.372153	0.421104	2.999919	17.581273	20.581192	0.145760	7.499907	14.000035	21.499942	0.348834
Basic 07-Secure Key Management	8.999907	12.456968	21.456875	0.419442	9.999896	11.372257	21.372153	0.467894	2.999919	17.581273	20.581192	0.145760	7.499907	14.000035	21.499942	0.348834
Basic 08-Disable Port Security Network Level	7.999896	13.456979	21.456875	0.372836	9.999896	11.372257	21.372153	0.467894	2.999919	17.581273	20.581192	0.145760	7.999907	13.500035	21.499942	0.372090
Basic 09-Manual Vdu Scaling	6.999896	14.456979	21.456875	0.326231	7.999907	13.372245	21.372153	0.374315	2.999919	17.581273	20.581192	0.145760	6.999907	14.500035	21.499942	0.325578
Basic 11-Native Charms	6.999896	14.456979	21.456875	0.326231	7.999907	13.372245	21.372153	0.374315	2.999919	17.581273	20.581192	0.145760	7.499907	14.000035	21.499942	0.348834
Basic 12-Ns Primitives	5.456933	15.999942	21.456875	0.254321	7.999907	13.372245	21.372153	0.374315	2.999919	17.581273	20.581192	0.145760	1.000000	20.499942	21.499942	0.046512
Basic 13-Ns Relations	8.999896	12.456979	21.456875	0.419441	8.999896	12.372257	21.372153	0.421104	2.999919	17.581273	20.581192	0.145760	0.500000	20.999942	21.499942	0.023256
Basic 14-Vnf Relations	6.999896	14.456979	21.456875	0.326231	7.999907	13.372245	21.372153	0.374315	3.581215	16.999977	20.581192	0.174004	0.500000	20.999942	21.499942	0.023256
Basic 15-Rbac Configurations	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
Basic 16-Advanced Onboarding And Scaling	7.999907	13.456968	21.456875	0.372837	9.999896	11.372257	21.372153	0.467894	4.999803	15.581389	20.581192	0.242931	6.999907	14.500035	21.499942	0.325578
Basic 17-Delete Vnf Package	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
Basic 18-Ns Ip Profile	6.999896	14.456979	21.456875	0.326231	7.999907	13.372245	21.372153	0.374315	2.999919	17.581273	20.581192	0.145760	0.000000	21.499942	21.499942	0.000000
Basic 19-Vnf Ip Profile	1.999988	19.456887	21.456875	0.093210	8.999907	12.372245	21.372153	0.421104	2.999919	17.581273	20.581192	0.145760	0.000000	21.499942	21.499942	0.000000
Basic 20-Manual Native Charm Vdu Scaling	6.999896	14.456979	21.456875	0.326231	9.999896	11.372257	21.372153	0.467894	3.581215	16.999977	20.581192	0.174004	6.499907	15.000035	21.499942	0.302322
Basic 21-Persistent Volumes	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Basic 21-Support Of Volumes	1.999988	19.456887	21.456875	0.093210	0.000000	0.000000	0.000000	0.000000	0.999988	19.581204	20.581192	0.048587	0.000000	21.499942	21.499942	0.000000
Basic 22-Cross Model Relations	8.999907	12.456968	21.456875	0.419442	0.000000	0.000000	0.000000	0.000000	7.581181	13.000012	20.581192	0.368355	9.499907	12.000035	21.499942	0.441857
Basic 23-Sol004 Sol007 Packages	0.000000	21.456875	21.456875	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
Basic 24-Affinity Groups	3.000000	18.456875	21.456875	0.139815	8.999907	12.372245	21.372153	0.421104	2.999919	17.581273	20.581192	0.145760	5.999907	15.500035	21.499942	0.279066
Basic 25-Update Charm In Running Vnf Instance	1.999988	19.456887	21.456875	0.093210	0.000000	0.000000	0.000000	0.000000	2.999919	17.581273	20.581192	0.145760	5.999907	15.500035	21.499942	0.279066
Basic 26-Secure Helm Execution Environment	1.999988	19.456887	21.456875	0.093210	0.000000	0.000000	0.000000	0.000000	0.999988	19.581204	20.581192	0.048587	0.000000	0.000000	0.000000	0.000000
Basic 27-Update Helm Ee In Running Vnf Instance	9.999931	11.456944	21.456875	0.466048	0.000000	0.000000	0.000000	0.000000	6.000035	14.581157	20.581192	0.291530	0.000000	0.000000	0.000000	0.000000
Basic 28-Keep Persistent Volumes	0.000000	8.457002	8.457002	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Epa 01-Epa Sriov	6.999896	14.456979	21.456875	0.326231	3.999988	17.372164	21.372153	0.187159	0.999988	19.581204	20.581192	0.048587	5.999907	15.500035	21.499942	0.279066
Epa 02-Additional Capabilities	1.999988	19.456887	21.456875	0.093210	3.999988	17.372164	21.372153	0.187159	0.999988	19.581204	20.581192	0.048587	5.999907	15.500035	21.499942	0.279066
Epa 03-Crud Operations On Sdnc	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
Epa 04-Epa Underlay Sriov	4.999988	16.456887	21.456875	0.233025	11.999931	9.372222	21.372153	0.561475	2.999919	17.581273	20.581192	0.145760	7.499907	14.000035	21.499942	0.348834
Epa 05-Epa Underlay Passthrough	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Fail 01-Insufficient Resources	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
Hackfest Basic	0.000000	21.456875	21.456875	0.000000	7.999919	13.372234	21.372153	0.374315	0.999988	19.581204	20.581192	0.048587	0.000000	21.499942	21.499942	0.000000
Hackfest Cloudinit	7.999907	13.456968	21.456875	0.372837	9.999896	11.372257	21.372153	0.467894	4.999803	15.581389	20.581192	0.242931	7.999907	13.500035	21.499942	0.372090
Hackfest Multivdu	3.000000	18.456875	21.456875	0.139815	9.999896	11.372257	21.372153	0.467894	4.999803	15.581389	20.581192	0.242931	8.999907	12.500035	21.499942	0.418601
Heal 01-Volume Vdu Healing	3.000000	18.456875	21.456875	0.139815	0.000000	0.000000	0.000000	0.000000	5.581100	15.000093	20.581192	0.271175	8.499907	13.000035	21.499942	0.395346
Heal 02-Scale Vdu Healing	7.999907	13.456968	21.456875	0.372837	0.000000	0.000000	0.000000	0.000000	5.581100	15.000093	20.581192	0.271175	8.499907	13.000035	21.499942	0.395346
Heal 03-Multiple Healing	7.999907	13.456968	21.456875	0.372837	0.000000	0.000000	0.000000	0.000000	6.581192	14.000000	20.581192	0.319767	8.499907	13.000035	21.499942	0.395346
Heal 04-Autohealing	3.000000	18.456875	21.456875	0.139815	0.000000	0.000000	0.000000	0.000000	3.999803	16.581389	20.581192	0.194343	1.000000	20.499942	21.499942	0.046512
K8S 01-Create K8S Cluster	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
K8S 02-K8Scluster Creation	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
K8S 03-Simple K8S	1.999988	19.456887	21.456875	0.093210	4.999977	16.372176	21.372153	0.233948	1.999988	18.581204	20.581192	0.097176	0.000000	21.499942	21.499942	0.000000
K8S 04-Openldap Helm	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
K8S 05-K8S Proxy Charms	3.000000	18.456875	21.456875	0.139815	4.999977	16.372176	21.372153	0.233948	4.999803	15.581389	20.581192	0.242931	2.000000	19.499942	21.499942	0.093024
K8S 06-K8S Secure Key Management	7.999907	13.456968	21.456875	0.372837	5.999977	15.372176	21.372153	0.280738	5.999896	14.581296	20.581192	0.291523	7.999907	13.500035	21.499942	0.372090
K8S 07-Dummy Helm	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
K8S 08-Simple K8S Scaling	1.999988	19.456887	21.456875	0.093210	4.999977	16.372176	21.372153	0.233948	1.999988	18.581204	20.581192	0.097176	0.000000	21.499942	21.499942	0.000000
K8S 09-Pebble Charm K8S	1.999988	19.456887	21.456875	0.093210	0.000000	0.000000	0.000000	0.000000	1.999988	18.581204	20.581192	0.097176	0.000000	21.499942	21.499942	0.000000
K8S 10-Sol004 Sol007 With K8S Proxy Charms	1.999988	19.456887	21.456875	0.093210	0.000000	0.000000	0.000000	0.000000	5.999896	14.581296	20.581192	0.291523	7.999907	13.500035	21.499942	0.372090
K8S 11-Simple Helm K8S Scaling	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
K8S 12-Openldap Helm Day-2	1.999988	19.456887	21.456875	0.093210	0.000000	0.000000	0.000000	0.000000	2.581285	17.999907	20.581192	0.125420	0.500000	20.999942	21.499942	0.023256
Quotas 01-Quota Enforcement	0.000000	21.456875	21.456875	0.000000	0.000000	21.372153	21.372153	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
Sa 01-Vnf With Vim Metrics	11.999873	9.457002	21.456875	0.559255	9.999896	11.372257	21.372153	0.467894	11.999931	8.581262	20.581192	0.583053	11.999873	9.500069	21.499942	0.558135
Sa 02-Vnf With Vim Metrics And Autoscaling	11.999873	9.457002	21.456875	0.559255	9.999896	11.372257	21.372153	0.467894	11.999931	8.581262	20.581192	0.583053	11.999873	9.500069	21.499942	0.558135
Sa 07-Alarms From Sa-Related Vnfs	11.999873	9.457002	21.456875	0.559255	9.999896	11.372257	21.372153	0.467894	11.999931	8.581262	20.581192	0.583053	11.999873	9.500069	21.499942	0.558135
Sa 08-Vnf With Vnf Indicators Snmp	21.456875	0.000000	21.456875	1.000000	0.000000	21.372153	21.372153	0.000000	3.999873	16.581319	20.581192	0.194346	0.500000	20.999942	21.499942	0.023256
Slice 01-Network Slicing	1.999988	19.456887	21.456875	0.093210	7.999907	13.372245	21.372153	0.374315	7.999896	12.581296	20.581192	0.388699	6.999907	14.500035	21.499942	0.325578
Slice 02-Shared Network Slicing	7.999907	13.456968	21.456875	0.372837	7.999907	13.372245	21.372153	0.374315	6.999907	13.581285	20.581192	0.340112	7.999907	13.500035	21.499942	0.372090
Sol003 01-Vnf-Lifecycle-Management	0.000000	21.456875	21.456875	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	20.581192	20.581192	0.000000	0.000000	21.499942	21.499942	0.000000
basic 23-delete vnf package SOL004 SOL007	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
k8s 10-k8s proxy charms SOL004 SOL007	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000

4. Information about the latest builds of relevant jobs

Latest build of each job

	job	build	timestamp	duration	build_result	test_result	pass_count	fail_count
0	osm-stage_3-merge/master	2452	2023-01-13 02:06:13	334531	FAILURE	UNAVAILABLE	NaN	NaN
1	osm-stage_3-merge/v13.0	75	2023-01-12 23:07:09	16880294	UNSTABLE	FAIL	517.0	41.0
2	osm-stage_3-merge/v12.0	270	2023-01-13 01:04:13	21672576	UNSTABLE	FAIL	468.0	62.0
3	osm-stage_3-merge/v10.0	686	2023-01-13 04:08:13	0	FAILURE	UNAVAILABLE	NaN	NaN

Click to see the details of build 2452 of osm-stage_3-merge/master

Click to see the details of build 75 of osm-stage_3-merge/v13.0

Click to see the details of build 270 of osm-stage_3-merge/v12.0

Click to see the details of build 686 of osm-stage_3-merge/v10.0

Failed test suites per job (if any):

osm-stage_3-merge/master: (full report)

	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

osm-stage_3-merge/v13.0: (full report)

	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword
136	Basic 14-Vnf Relations	FAIL	2023-01-13 00:01:33	2023-01-13 00:18:02	5	1	Instantiate Charm Network Service	Create Network Service
142	Basic 20-Manual Native Charm Vdu Scaling	FAIL	2023-01-13 00:27:37	2023-01-13 00:34:17	10	8	Instantiate Network Service	Create Network Service
144	Basic 22-Cross Model Relations	FAIL	2023-01-13 00:36:40	2023-01-13 00:55:25	9	1	Instantiate Charm Network Service	Create Network Service
158	Heal 01-Volume Vdu Healing	FAIL	2023-01-13 01:35:50	2023-01-13 01:42:06	6	10	Network Service Instance Test	Create Network Service
159	Heal 02-Scale Vdu Healing	FAIL	2023-01-13 01:42:06	2023-01-13 01:48:21	6	9	Network Service Instance Test	Create Network Service
160	Heal 03-Multiple Healing	FAIL	2023-01-13 01:48:21	2023-01-13 01:55:06	6	10	Network Service Instance Test	Create Network Service
171	K8S 12-Openldap Helm Day-2	FAIL	2023-01-13 02:55:24	2023-01-13 03:05:01	13	2	Create Network Service Instance	Create Network Service

osm-stage_3-merge/v12.0: (full report)

	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword
78	Basic 06-Vnf With Charm	FAIL	2023-01-13 01:43:13	2023-01-13 02:01:24	5	6	Instantiate Charm Network Service	Create Network Service
79	Basic 07-Secure Key Management	FAIL	2023-01-13 02:01:24	2023-01-13 02:21:12	8	4	Instantiate Nopasswd Charm Network Service	Create Network Service
82	Basic 11-Native Charms	FAIL	2023-01-13 02:26:35	2023-01-13 02:43:26	5	4	Instantiate Charm Network Service	Create Network Service
84	Basic 13-Ns Relations	FAIL	2023-01-13 02:56:41	2023-01-13 03:12:56	7	1	Instantiate Charm Network Service	Create Network Service
85	Basic 14-Vnf Relations	FAIL	2023-01-13 03:12:56	2023-01-13 03:28:43	5	1	Instantiate Charm Network Service	Create Network Service
91	Basic 20-Manual Native Charm Vdu Scaling	FAIL	2023-01-13 03:38:31	2023-01-13 03:44:58	10	8	Instantiate Network Service	Create Network Service
93	Basic 22-Cross Model Relations	FAIL	2023-01-13 03:47:19	2023-01-13 04:06:19	9	1	Instantiate Charm Network Service	Create Network Service
100	Epa 04-Epa Underlay Sriov	FAIL	2023-01-13 04:21:23	2023-01-13 04:25:06	8	1	Ping from Vnf1 to Vnf2	Execute Remote Command Check Rc Return Output
105	Heal 01-Volume Vdu Healing	FAIL	2023-01-13 04:34:29	2023-01-13 04:41:16	6	10	Network Service Instance Test	Create Network Service
106	Heal 02-Scale Vdu Healing	FAIL	2023-01-13 04:41:16	2023-01-13 04:47:30	6	9	Network Service Instance Test	Create Network Service
107	Heal 03-Multiple Healing	FAIL	2023-01-13 04:47:30	2023-01-13 04:54:15	6	10	Network Service Instance Test	Create Network Service
118	K8S 12-Openldap Helm Day-2	FAIL	2023-01-13 06:02:05	2023-01-13 06:14:04	10	5	Create Network Service Instance	Create Network Service
123	Sa 08-Vnf With Vnf Indicators Snmp	FAIL	2023-01-13 06:34:33	2023-01-13 06:49:39	5	2	Instantiate Network Service	Create Network Service

osm-stage_3-merge/v10.0: (full report)

	name	status	starttime	endtime	pass	fail	failed_test_name	failed_keyword

Details of failed tests into failing test suites (if any):

osm-stage_3-merge/master: (full report)

	suite_name	test_name	keyword_name	status	starttime	endtime

osm-stage_3-merge/v13.0: (full report)

	suite_name	test_name	keyword_name	status	starttime	endtime
397	Basic 14-Vnf Relations	Instantiate Charm Network Service	Create Network Service	FAIL	2023-01-13 00:01:38	2023-01-13 00:17:04
404	Basic 20-Manual Native Charm Vdu Scaling	Instantiate Network Service	Create Network Service	FAIL	2023-01-13 00:27:41	2023-01-13 00:33:14
406	Basic 20-Manual Native Charm Vdu Scaling	Get Vnf Id	Variable Should Exist	FAIL	2023-01-13 00:33:14	2023-01-13 00:33:14
410	Basic 20-Manual Native Charm Vdu Scaling	Get Vdus Before Scale Out	Get Vnf Vdur Names	FAIL	2023-01-13 00:33:14	2023-01-13 00:33:14
419	Basic 20-Manual Native Charm Vdu Scaling	Perform Manual Vdu Scale Out	Variable Should Exist	FAIL	2023-01-13 00:33:16	2023-01-13 00:33:16
421	Basic 20-Manual Native Charm Vdu Scaling	Check Vdus After Scale Out	Variable Should Exist	FAIL	2023-01-13 00:33:16	2023-01-13 00:33:16
430	Basic 20-Manual Native Charm Vdu Scaling	Check Number of Duplicated Application Name Af...	Run Keyword If	FAIL	2023-01-13 00:33:18	2023-01-13 00:33:18
431	Basic 20-Manual Native Charm Vdu Scaling	Perform Manual Vdu Scale In	Variable Should Exist	FAIL	2023-01-13 00:33:18	2023-01-13 00:33:18
433	Basic 20-Manual Native Charm Vdu Scaling	Check Vdus After Scaling In	Variable Should Exist	FAIL	2023-01-13 00:33:18	2023-01-13 00:33:18
450	Basic 22-Cross Model Relations	Instantiate Charm Network Service	Create Network Service	FAIL	2023-01-13 00:37:23	2023-01-13 00:52:53
460	Heal 01-Volume Vdu Healing	Network Service Instance Test	Create Network Service	FAIL	2023-01-13 01:35:57	2023-01-13 01:40:59
463	Heal 01-Volume Vdu Healing	Get VIM Objects	Variable Should Exist	FAIL	2023-01-13 01:40:59	2023-01-13 01:40:59
468	Heal 01-Volume Vdu Healing	Get Volume VNF Info	Variable Should Exist	FAIL	2023-01-13 01:40:59	2023-01-13 01:40:59
485	Heal 01-Volume Vdu Healing	Get Volumes Info	Get Server Property	FAIL	2023-01-13 01:41:01	2023-01-13 01:41:01
488	Heal 01-Volume Vdu Healing	Check VDU Disks	Variable Should Exist	FAIL	2023-01-13 01:41:01	2023-01-13 01:41:01
495	Heal 01-Volume Vdu Healing	Delete Persistent Volume VDU	Variable Should Exist	FAIL	2023-01-13 01:41:01	2023-01-13 01:41:01
498	Heal 01-Volume Vdu Healing	Heal Persistent Volume VDU	Variable Should Exist	FAIL	2023-01-13 01:41:01	2023-01-13 01:41:01
500	Heal 01-Volume Vdu Healing	Check VNF After Healing	Variable Should Exist	FAIL	2023-01-13 01:41:01	2023-01-13 01:41:01
516	Heal 01-Volume Vdu Healing	Update VIM Objects	Variable Should Exist	FAIL	2023-01-13 01:41:01	2023-01-13 01:41:01
529	Heal 02-Scale Vdu Healing	Network Service Instance Test	Create Network Service	FAIL	2023-01-13 01:42:12	2023-01-13 01:47:16
532	Heal 02-Scale Vdu Healing	Scale Out Charm VNF	Get Vnf Id	FAIL	2023-01-13 01:47:16	2023-01-13 01:47:16
541	Heal 02-Scale Vdu Healing	Get VIM Objects	Variable Should Exist	FAIL	2023-01-13 01:47:16	2023-01-13 01:47:16
546	Heal 02-Scale Vdu Healing	Get Charm VNF Info	Variable Should Exist	FAIL	2023-01-13 01:47:16	2023-01-13 01:47:16
554	Heal 02-Scale Vdu Healing	Halt Charm VDU	Variable Should Exist	FAIL	2023-01-13 01:47:16	2023-01-13 01:47:16
557	Heal 02-Scale Vdu Healing	Heal Charm VDU	Variable Should Exist	FAIL	2023-01-13 01:47:16	2023-01-13 01:47:16
559	Heal 02-Scale Vdu Healing	Check VNF After Healing	Variable Should Exist	FAIL	2023-01-13 01:47:16	2023-01-13 01:47:16
571	Heal 02-Scale Vdu Healing	Update VIM Objects	Variable Should Exist	FAIL	2023-01-13 01:47:16	2023-01-13 01:47:16
584	Heal 03-Multiple Healing	Network Service Instance Test	Create Network Service	FAIL	2023-01-13 01:48:27	2023-01-13 01:54:01
587	Heal 03-Multiple Healing	Scale Out Charm VNF	Get Vnf Id	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
596	Heal 03-Multiple Healing	Get VIM Objects	Variable Should Exist	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
601	Heal 03-Multiple Healing	Get Charm VNF Info	Variable Should Exist	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
609	Heal 03-Multiple Healing	Get Volume VNF Info	Variable Should Exist	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
621	Heal 03-Multiple Healing	Stop VDUs	Variable Should Exist	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
627	Heal 03-Multiple Healing	Heal VDUs	Variable Should Exist	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
630	Heal 03-Multiple Healing	Check VNFs After Healing	Variable Should Exist	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
643	Heal 03-Multiple Healing	Update VIM Objects	Variable Should Exist	FAIL	2023-01-13 01:54:01	2023-01-13 01:54:01
656	K8S 12-Openldap Helm Day-2	Create Network Service Instance	Create Network Service	FAIL	2023-01-13 02:56:05	2023-01-13 03:01:08
664	K8S 12-Openldap Helm Day-2	Execute Day 2 Operations	Execute NS K8s Action	FAIL	2023-01-13 03:01:11	2023-01-13 03:02:17

osm-stage_3-merge/v12.0: (full report)

	suite_name	test_name	keyword_name	status	starttime	endtime
2	Basic 06-Vnf With Charm	Instantiate Charm Network Service	Create Network Service	FAIL	2023-01-13 01:43:18	2023-01-13 01:58:51
4	Basic 06-Vnf With Charm	Get Management Ip Addresses	Get Vnf Management Ip Address	FAIL	2023-01-13 01:58:51	2023-01-13 01:58:51
10	Basic 06-Vnf With Charm	Test SSH Access	Variable Should Exist	FAIL	2023-01-13 01:58:51	2023-01-13 01:58:51
15	Basic 06-Vnf With Charm	Check Remote Files Created Via Day 1 Operations	Check If remote File Exists	FAIL	2023-01-13 01:58:51	2023-01-13 01:58:51
17	Basic 06-Vnf With Charm	Execute Day 2 Operations	Variable Should Exist	FAIL	2023-01-13 01:58:51	2023-01-13 01:58:51
20	Basic 06-Vnf With Charm	Check Remote Files Created Via Day 2 Operations	Check If remote File Exists	FAIL	2023-01-13 01:58:51	2023-01-13 01:58:51
27	Basic 07-Secure Key Management	Instantiate Nopasswd Charm Network Service	Create Network Service	FAIL	2023-01-13 02:01:29	2023-01-13 02:17:01
36	Basic 07-Secure Key Management	Check Remote Files Created Via Day 1 Operations	Check If remote File Exists	FAIL	2023-01-13 02:17:36	2023-01-13 02:17:38
38	Basic 07-Secure Key Management	Execute Day 2 Operations	Execute NS Action	FAIL	2023-01-13 02:17:38	2023-01-13 02:18:43
39	Basic 07-Secure Key Management	Check Remote Files Created Via Day 2 Operations	Check If remote File Exists	FAIL	2023-01-13 02:18:43	2023-01-13 02:18:45
45	Basic 11-Native Charms	Instantiate Charm Network Service	Create Network Service	FAIL	2023-01-13 02:26:40	2023-01-13 02:42:10
47	Basic 11-Native Charms	Get Management Ip Addresses	Get Vnf Management Ip Address	FAIL	2023-01-13 02:42:10	2023-01-13 02:42:10
53	Basic 11-Native Charms	Test SSH Access	Variable Should Exist	FAIL	2023-01-13 02:42:10	2023-01-13 02:42:10
58	Basic 11-Native Charms	Check Remote Files Created Via Day 1 Operations	Check If remote File Exists	FAIL	2023-01-13 02:42:10	2023-01-13 02:42:10
66	Basic 13-Ns Relations	Instantiate Charm Network Service	Create Network Service	FAIL	2023-01-13 02:56:48	2023-01-13 03:12:16
74	Basic 14-Vnf Relations	Instantiate Charm Network Service	Create Network Service	FAIL	2023-01-13 03:13:01	2023-01-13 03:28:03
81	Basic 20-Manual Native Charm Vdu Scaling	Instantiate Network Service	Create Network Service	FAIL	2023-01-13 03:38:35	2023-01-13 03:44:09
83	Basic 20-Manual Native Charm Vdu Scaling	Get Vnf Id	Variable Should Exist	FAIL	2023-01-13 03:44:09	2023-01-13 03:44:09
87	Basic 20-Manual Native Charm Vdu Scaling	Get Vdus Before Scale Out	Get Vnf Vdur Names	FAIL	2023-01-13 03:44:09	2023-01-13 03:44:09
96	Basic 20-Manual Native Charm Vdu Scaling	Perform Manual Vdu Scale Out	Variable Should Exist	FAIL	2023-01-13 03:44:11	2023-01-13 03:44:11
98	Basic 20-Manual Native Charm Vdu Scaling	Check Vdus After Scale Out	Variable Should Exist	FAIL	2023-01-13 03:44:11	2023-01-13 03:44:11
107	Basic 20-Manual Native Charm Vdu Scaling	Check Number of Duplicated Application Name Af...	Run Keyword If	FAIL	2023-01-13 03:44:13	2023-01-13 03:44:13
108	Basic 20-Manual Native Charm Vdu Scaling	Perform Manual Vdu Scale In	Variable Should Exist	FAIL	2023-01-13 03:44:13	2023-01-13 03:44:13
110	Basic 20-Manual Native Charm Vdu Scaling	Check Vdus After Scaling In	Variable Should Exist	FAIL	2023-01-13 03:44:13	2023-01-13 03:44:13
127	Basic 22-Cross Model Relations	Instantiate Charm Network Service	Create Network Service	FAIL	2023-01-13 03:48:02	2023-01-13 04:03:32
148	Epa 04-Epa Underlay Sriov	Ping from Vnf1 to Vnf2	Execute Remote Command Check Rc Return Output	FAIL	2023-01-13 04:23:46	2023-01-13 04:24:02
159	Heal 01-Volume Vdu Healing	Network Service Instance Test	Create Network Service	FAIL	2023-01-13 04:34:35	2023-01-13 04:40:09
162	Heal 01-Volume Vdu Healing	Get VIM Objects	Variable Should Exist	FAIL	2023-01-13 04:40:09	2023-01-13 04:40:09
167	Heal 01-Volume Vdu Healing	Get Volume VNF Info	Variable Should Exist	FAIL	2023-01-13 04:40:09	2023-01-13 04:40:09
184	Heal 01-Volume Vdu Healing	Get Volumes Info	Get Server Property	FAIL	2023-01-13 04:40:10	2023-01-13 04:40:10
187	Heal 01-Volume Vdu Healing	Check VDU Disks	Variable Should Exist	FAIL	2023-01-13 04:40:10	2023-01-13 04:40:10
194	Heal 01-Volume Vdu Healing	Delete Persistent Volume VDU	Variable Should Exist	FAIL	2023-01-13 04:40:10	2023-01-13 04:40:10
197	Heal 01-Volume Vdu Healing	Heal Persistent Volume VDU	Variable Should Exist	FAIL	2023-01-13 04:40:10	2023-01-13 04:40:10
199	Heal 01-Volume Vdu Healing	Check VNF After Healing	Variable Should Exist	FAIL	2023-01-13 04:40:10	2023-01-13 04:40:10
215	Heal 01-Volume Vdu Healing	Update VIM Objects	Variable Should Exist	FAIL	2023-01-13 04:40:10	2023-01-13 04:40:10
228	Heal 02-Scale Vdu Healing	Network Service Instance Test	Create Network Service	FAIL	2023-01-13 04:41:23	2023-01-13 04:46:26
231	Heal 02-Scale Vdu Healing	Scale Out Charm VNF	Get Vnf Id	FAIL	2023-01-13 04:46:26	2023-01-13 04:46:26
240	Heal 02-Scale Vdu Healing	Get VIM Objects	Variable Should Exist	FAIL	2023-01-13 04:46:26	2023-01-13 04:46:26
245	Heal 02-Scale Vdu Healing	Get Charm VNF Info	Variable Should Exist	FAIL	2023-01-13 04:46:26	2023-01-13 04:46:26
253	Heal 02-Scale Vdu Healing	Halt Charm VDU	Variable Should Exist	FAIL	2023-01-13 04:46:26	2023-01-13 04:46:26
256	Heal 02-Scale Vdu Healing	Heal Charm VDU	Variable Should Exist	FAIL	2023-01-13 04:46:26	2023-01-13 04:46:26
258	Heal 02-Scale Vdu Healing	Check VNF After Healing	Variable Should Exist	FAIL	2023-01-13 04:46:26	2023-01-13 04:46:26
270	Heal 02-Scale Vdu Healing	Update VIM Objects	Variable Should Exist	FAIL	2023-01-13 04:46:26	2023-01-13 04:46:26
283	Heal 03-Multiple Healing	Network Service Instance Test	Create Network Service	FAIL	2023-01-13 04:47:36	2023-01-13 04:53:11
286	Heal 03-Multiple Healing	Scale Out Charm VNF	Get Vnf Id	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
295	Heal 03-Multiple Healing	Get VIM Objects	Variable Should Exist	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
300	Heal 03-Multiple Healing	Get Charm VNF Info	Variable Should Exist	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
308	Heal 03-Multiple Healing	Get Volume VNF Info	Variable Should Exist	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
320	Heal 03-Multiple Healing	Stop VDUs	Variable Should Exist	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
326	Heal 03-Multiple Healing	Heal VDUs	Variable Should Exist	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
329	Heal 03-Multiple Healing	Check VNFs After Healing	Variable Should Exist	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
342	Heal 03-Multiple Healing	Update VIM Objects	Variable Should Exist	FAIL	2023-01-13 04:53:11	2023-01-13 04:53:11
355	K8S 12-Openldap Helm Day-2	Create Network Service Instance	Create Network Service	FAIL	2023-01-13 06:02:46	2023-01-13 06:08:19
363	K8S 12-Openldap Helm Day-2	Execute Day 2 Operations	Execute NS K8s Action	FAIL	2023-01-13 06:08:23	2023-01-13 06:09:29
365	K8S 12-Openldap Helm Day-2	Execute Upgrade Operation	Execute NS K8s Action	FAIL	2023-01-13 06:09:29	2023-01-13 06:10:35
369	K8S 12-Openldap Helm Day-2	Check Replicas After Upgrade Operation	Should Be Equal As Integers	FAIL	2023-01-13 06:11:07	2023-01-13 06:11:07
371	K8S 12-Openldap Helm Day-2	Execute Rollback Operation	Execute NS K8s Action	FAIL	2023-01-13 06:11:07	2023-01-13 06:12:12
382	Sa 08-Vnf With Vnf Indicators Snmp	Instantiate Network Service	Create Network Service	FAIL	2023-01-13 06:34:37	2023-01-13 06:39:40
388	Sa 08-Vnf With Vnf Indicators Snmp	Get VNF SNMP Metrics	Wait Until Keyword Succeeds	FAIL	2023-01-13 06:39:40	2023-01-13 06:48:40

osm-stage_3-merge/v10.0: (full report)

	suite_name	test_name	keyword_name	status	starttime	endtime

---

---