Analysis of Robot reports from OSM Jenkins¶
Date and time of the report: 2025-09-21 22:33:10.584545
Analysed period: 2025-08-31 to 2025-09-21.
1. Retrieval of all currrent data for aggregate analytics¶
Database update:
Retrieving new jobs...
server=<jenkins.Jenkins object at 0x728d59954980> job='osm-stage_3-merge/master' engine=Engine(sqlite:///etl_outputs/test_executions.db) robot_report='inputs/output.xml'
Traceback (most recent call last): File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 465, in get_job_info response = self.jenkins_open(requests.Request( 'GET', self._build_url(JOB_INFO, locals()) )) File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 557, in jenkins_open return self.jenkins_request(req, add_crumb, resolve_auth).text ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 573, in jenkins_request self.maybe_add_crumb(req) ~~~~~~~~~~~~~~~~~~~~^^^^^ File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 370, in maybe_add_crumb response = self.jenkins_open(requests.Request( 'GET', self._build_url(CRUMB_URL)), add_crumb=False) File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 557, in jenkins_open return self.jenkins_request(req, add_crumb, resolve_auth).text ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 576, in jenkins_request self._request(req)) ~~~~~~~~~~~~~^^^^^ File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 550, in _request return self._session.send(r, **_settings) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.13/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/opt/conda/lib/python3.13/site-packages/requests/adapters.py", line 664, in send timeout = TimeoutSauce(connect=timeout, read=timeout) File "/opt/conda/lib/python3.13/site-packages/urllib3/util/timeout.py", line 115, in __init__ self._connect = self._validate_timeout(connect, "connect") ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.13/site-packages/urllib3/util/timeout.py", line 152, in _validate_timeout raise ValueError( ...<2 lines>... ) from None ValueError: Timeout value connect was <object object at 0x728d76487f20>, but it must be an int, float or None. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/workspaces/Analíticas OSM/osm-analytics/Jenkins/./00-script-jenkins_and_robot_etl.py", line 122, in <module> ingest_update_all_jenkins_job(jenkins_server=server, ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ job_name=job, ^^^^^^^^^^^^^ database_engine=engine, ^^^^^^^^^^^^^^^^^^^^^^^ robot_report=os.path.join(inputs_folder, input_robot_file)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/Analíticas OSM/osm-analytics/Jenkins/jenkins_robot_etl.py", line 25, in ingest_update_all_jenkins_job df_builds_of_job = get_all_job_builds(jenkins_server, job_name) File "/workspaces/Analíticas OSM/osm-analytics/Jenkins/jenkins_lib.py", line 79, in get_all_job_builds my_job = server.get_job_info(job_name, 0, True) File "/opt/conda/lib/python3.13/site-packages/jenkins/__init__.py", line 478, in get_job_info raise JenkinsException( "Could not parse JSON info for job[%s]" % name) jenkins.JenkinsException: Could not parse JSON info for job[osm-stage_3-merge/master]
Retrieving from database...
DONE
2. Aggregated analysis of stability¶
2.1 Restricts data to time window for analysis¶
2.2 Finding sequences of successful builds and Robot reports¶
Two values retrieved from the build are key to determine the state of the jubs and test suites at a given moment:
build_resultis the outcome of the build, reported by Jenkins. It can be:SUCCESS,FAILURE,UNSTABLEorABORTED.test_resultis the summary of the concerned Robot tests. It can be:FAIL,UNAVAILABLEorPASS.
Based on these two states, 3 types of temporal sequences of success/failure are identified per builds and test suites:
- Successful builds/failed builds in a row:
grp_build_result. - Successful test reports vs. test reports with fails in a row:
grp_test_result. - Clean builds and tests vs. failures (of any kind) in a row:
grp_success_fail.
For the identification of these sequences, the following mapping applies:
| Type of sequence | Relevant state | OK sequence contains | NOK sequence contains | Ignore |
|---|---|---|---|---|
grp_build_result |
build_result |
SUCCESS or UNSTABLE |
FAILURE |
ABORTED |
grp_test_result |
test_result |
PASS |
FAIL |
UNAVAILABLE |
grp_success_fail |
test_result |
PASS |
FAIL or UNAVAILABLE |
N/A |
The different groupings of segments are detected and a label is added to each sample...
/tmp/ipykernel_5651/285049919.py:8: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. return df.groupby(grouping)[relevant_col].transform( /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:8: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. return df.groupby(grouping)[relevant_col].transform( /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:8: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. return df.groupby(grouping)[relevant_col].transform( /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored) /tmp/ipykernel_5651/285049919.py:12: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='ffill') /tmp/ipykernel_5651/285049919.py:13: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead. .fillna(method='bfill') # Extrapolation if first samples are inconclusive (i.e. should be ignored)
DONE
Then, the duration of each of the sequences of success/failure is determined and a specific dataframe is built summarizing such sequences, to ease their representation and analysis...
/tmp/ipykernel_5651/2631965633.py:75: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
.groupby(by=grp_cols)
/tmp/ipykernel_5651/2631965633.py:18: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
left_shifted = df.groupby(agg).min_timestamp.shift(-1)
/tmp/ipykernel_5651/2631965633.py:30: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
last_item_indexes = df.groupby(agg).tail(1).index
/tmp/ipykernel_5651/2631965633.py:40: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
df.groupby('job')
/tmp/ipykernel_5651/2631965633.py:75: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
.groupby(by=grp_cols)
/tmp/ipykernel_5651/2631965633.py:18: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
left_shifted = df.groupby(agg).min_timestamp.shift(-1)
/tmp/ipykernel_5651/2631965633.py:30: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
last_item_indexes = df.groupby(agg).tail(1).index
/tmp/ipykernel_5651/2631965633.py:40: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
df.groupby('job')
/tmp/ipykernel_5651/2631965633.py:75: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
.groupby(by=grp_cols)
/tmp/ipykernel_5651/2631965633.py:18: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
left_shifted = df.groupby(agg).min_timestamp.shift(-1)
/tmp/ipykernel_5651/2631965633.py:30: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
last_item_indexes = df.groupby(agg).tail(1).index
/tmp/ipykernel_5651/2631965633.py:40: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
df.groupby('job')
/tmp/ipykernel_5651/2631965633.py:75: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
.groupby(by=grp_cols)
/tmp/ipykernel_5651/2631965633.py:18: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
left_shifted = df.groupby(agg).min_timestamp.shift(-1)
/tmp/ipykernel_5651/2631965633.py:30: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
last_item_indexes = df.groupby(agg).tail(1).index
/tmp/ipykernel_5651/2631965633.py:40: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
df.groupby('job')
/tmp/ipykernel_5651/2631965633.py:75: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
.groupby(by=grp_cols)
/tmp/ipykernel_5651/2631965633.py:18: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
left_shifted = df.groupby(agg).min_timestamp.shift(-1)
/tmp/ipykernel_5651/2631965633.py:30: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
last_item_indexes = df.groupby(agg).tail(1).index
/tmp/ipykernel_5651/2631965633.py:40: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
df.groupby('job')
DONE
2.3 Finding sequences of pass/fails per test suite¶
/tmp/ipykernel_5651/285049919.py:8: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. return df.groupby(grouping)[relevant_col].transform(
/tmp/ipykernel_5651/2631965633.py:75: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
.groupby(by=grp_cols)
/tmp/ipykernel_5651/2631965633.py:18: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
left_shifted = df.groupby(agg).min_timestamp.shift(-1)
/tmp/ipykernel_5651/2631965633.py:30: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
last_item_indexes = df.groupby(agg).tail(1).index
/tmp/ipykernel_5651/2631965633.py:40: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
df.groupby('job')
DONE
3. Reports¶
3.1 Aggregated success rate per test step¶
Empty dataframe. Skipping. Empty dataframe. Skipping. Empty dataframe. Skipping. Empty dataframe. Skipping. Empty dataframe. Skipping.
3.2 Overall success of Jenkins builds and Robot tests¶
3.3 Sequences of pass/fails per test suites¶
Empty dataframe. Skipping. Empty dataframe. Skipping. Empty dataframe. Skipping. Empty dataframe. Skipping. Empty dataframe. Skipping.
/tmp/ipykernel_5651/2250086286.py:7: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. for name, group in df_suites.groupby(['job', 'name']): /tmp/ipykernel_5651/2250086286.py:7: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. for name, group in df_suites.groupby(['job', 'name']): /tmp/ipykernel_5651/2250086286.py:7: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. for name, group in df_suites.groupby(['job', 'name']): /tmp/ipykernel_5651/2250086286.py:7: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. for name, group in df_suites.groupby(['job', 'name']): /tmp/ipykernel_5651/2250086286.py:7: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. for name, group in df_suites.groupby(['job', 'name']):
3.4 Failing days per test suite¶
/tmp/ipykernel_5651/771943242.py:14: FutureWarning: The default value of observed=False is deprecated and will change to observed=True in a future version of pandas. Specify observed=False to silence this warning and retain the current behavior .pivot_table(
/tmp/ipykernel_5651/771943242.py:14: FutureWarning: The default value of observed=False is deprecated and will change to observed=True in a future version of pandas. Specify observed=False to silence this warning and retain the current behavior .pivot_table(
/tmp/ipykernel_5651/771943242.py:14: FutureWarning: The default value of observed=False is deprecated and will change to observed=True in a future version of pandas. Specify observed=False to silence this warning and retain the current behavior .pivot_table(
/tmp/ipykernel_5651/771943242.py:14: FutureWarning: The default value of observed=False is deprecated and will change to observed=True in a future version of pandas. Specify observed=False to silence this warning and retain the current behavior .pivot_table(
/tmp/ipykernel_5651/771943242.py:14: FutureWarning: The default value of observed=False is deprecated and will change to observed=True in a future version of pandas. Specify observed=False to silence this warning and retain the current behavior .pivot_table(
Out[58]:
| job | Master branch | Release FIFTEEN | Release FOURTEEN | Release SEVENTEEN | Release SIXTEEN | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | days_failing | days_passing | days_run | error_rate | |
| suites | ||||||||||||||||||||
| Basic 01-Crud Operations On Vim Targets | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Basic 05-Instantiation Parameters In Cloud Init | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Basic 06-Vnf With Charm | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Basic 07-Secure Key Management | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Basic 08-Disable Port Security Network Level | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| Sa 08-Vnf With Vnf Indicators Snmp | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Slice 01-Network Slicing | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Slice 02-Shared Network Slicing | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Sol003 01-Vnf-Lifecycle-Management | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Sol003 02-Dualstack Ip Vnfm | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
61 rows × 20 columns
4. Information about the latest builds of relevant jobs¶
Latest build of each job¶
| job | build | timestamp | duration | build_result | test_result | pass_count | fail_count | |
|---|---|---|---|---|---|---|---|---|
| 0 | osm-stage_3-merge/master | 3397 | 2025-06-22 01:06:41.707 | 12623573 | UNSTABLE | FAIL | 465.0 | 7.0 |
| 1 | osm-stage_3-merge/v17.0 | 166 | 2025-06-22 02:49:42.092 | 12347487 | UNSTABLE | FAIL | 465.0 | 7.0 |
| 2 | osm-stage_3-merge/v16.0 | 279 | 2025-06-21 23:01:41.225 | 12448708 | UNSTABLE | FAIL | 465.0 | 7.0 |
| 3 | osm-stage_3-merge/v15.0 | 446 | 2025-03-21 03:34:47.201 | 12254896 | UNSTABLE | FAIL | 448.0 | 7.0 |
| 4 | osm-stage_3-merge/v14.0 | 696 | 2025-06-21 22:08:40.749 | 0 | FAILURE | UNAVAILABLE | NaN | NaN |
Failed test suites per job (if any):¶
osm-stage_3-merge/master: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|
| 132 | Sa 01-Vnf With Vim Metrics | FAIL | 2025-06-22 03:06:22.104 | 2025-06-22 03:21:48.272 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
| 133 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2025-06-22 03:21:48.273 | 2025-06-22 03:56:01.355 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
| 134 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2025-06-22 03:56:01.357 | 2025-06-22 04:17:08.606 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v17.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|
| 220 | Sa 01-Vnf With Vim Metrics | FAIL | 2025-06-22 04:45:54.942 | 2025-06-22 05:01:14.244 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
| 221 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2025-06-22 05:01:14.245 | 2025-06-22 05:34:45.064 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
| 222 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2025-06-22 05:34:45.065 | 2025-06-22 05:55:36.953 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v16.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|
| 176 | Sa 01-Vnf With Vim Metrics | FAIL | 2025-06-22 00:58:48.380 | 2025-06-22 01:14:44.050 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
| 177 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2025-06-22 01:14:44.051 | 2025-06-22 01:48:17.351 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
| 178 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2025-06-22 01:48:17.353 | 2025-06-22 02:09:15.257 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v15.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword | |
|---|---|---|---|---|---|---|---|---|
| 88 | Sa 01-Vnf With Vim Metrics | FAIL | 2025-03-21 05:28:38.675 | 2025-03-21 05:43:57.785 | 6 | 1 | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds |
| 89 | Sa 02-Vnf With Vim Metrics And Autoscaling | FAIL | 2025-03-21 05:43:57.789 | 2025-03-21 06:17:27.173 | 11 | 4 | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds |
| 90 | Sa 07-Alarms From Sa-Related Vnfs | FAIL | 2025-03-21 06:17:27.175 | 2025-03-21 06:38:22.438 | 14 | 2 | Get Alarm Metric | Wait Until Keyword Succeeds |
osm-stage_3-merge/v14.0: (full report)
| name | status | starttime | endtime | pass | fail | failed_test_name | failed_keyword |
|---|
Details of failed tests into failing test suites (if any):¶
osm-stage_3-merge/master: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|
| 97 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2025-06-22 03:07:06.261 | 2025-06-22 03:21:07.947 |
| 127 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2025-06-22 03:23:12.258 | 2025-06-22 03:37:13.549 |
| 132 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2025-06-22 03:37:16.124 | 2025-06-22 03:51:18.545 |
| 135 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2025-06-22 03:53:18.549 | 2025-06-22 03:53:18.628 |
| 166 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2025-06-22 03:58:13.238 | 2025-06-22 04:12:14.967 |
| 167 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2025-06-22 04:12:14.969 | 2025-06-22 04:15:45.407 |
osm-stage_3-merge/v17.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|
| 271 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2025-06-22 04:46:36.443 | 2025-06-22 05:00:38.115 |
| 301 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2025-06-22 05:02:01.809 | 2025-06-22 05:16:03.387 |
| 306 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2025-06-22 05:16:05.676 | 2025-06-22 05:30:07.052 |
| 309 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2025-06-22 05:32:07.055 | 2025-06-22 05:32:07.105 |
| 340 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2025-06-22 05:36:52.182 | 2025-06-22 05:50:53.846 |
| 341 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2025-06-22 05:50:53.847 | 2025-06-22 05:54:23.857 |
osm-stage_3-merge/v16.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|
| 184 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2025-06-22 01:00:03.399 | 2025-06-22 01:14:06.018 |
| 214 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2025-06-22 01:15:32.476 | 2025-06-22 01:29:33.671 |
| 219 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2025-06-22 01:29:36.027 | 2025-06-22 01:43:37.313 |
| 222 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2025-06-22 01:45:37.317 | 2025-06-22 01:45:37.400 |
| 253 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2025-06-22 01:50:25.082 | 2025-06-22 02:04:26.527 |
| 254 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2025-06-22 02:04:26.529 | 2025-06-22 02:07:57.076 |
osm-stage_3-merge/v15.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime | |
|---|---|---|---|---|---|---|
| 10 | Sa 01-Vnf With Vim Metrics | Get VNF VIM-based Metrics | Wait Until Keyword Succeeds | FAIL | 2025-03-21 05:29:20.827 | 2025-03-21 05:43:21.137 |
| 40 | Sa 02-Vnf With Vim Metrics And Autoscaling | Get VNF VIM-based Metric Before Auto-scaling | Wait Until Keyword Succeeds | FAIL | 2025-03-21 05:44:45.904 | 2025-03-21 05:58:46.223 |
| 45 | Sa 02-Vnf With Vim Metrics And Autoscaling | Wait VIM-based Metric To Exceed Threshold | Wait Until Keyword Succeeds | FAIL | 2025-03-21 05:58:48.617 | 2025-03-21 06:12:48.958 |
| 48 | Sa 02-Vnf With Vim Metrics And Autoscaling | Check VIM-based Metric Exceeds Threshold After... | Check VIM-based Metric Exceeds Threshold | FAIL | 2025-03-21 06:14:48.962 | 2025-03-21 06:14:48.981 |
| 79 | Sa 07-Alarms From Sa-Related Vnfs | Get Alarm Metric | Wait Until Keyword Succeeds | FAIL | 2025-03-21 06:19:34.795 | 2025-03-21 06:33:35.126 |
| 80 | Sa 07-Alarms From Sa-Related Vnfs | Check Alarms Were Received | Wait Until Keyword Succeeds | FAIL | 2025-03-21 06:33:35.127 | 2025-03-21 06:37:05.912 |
osm-stage_3-merge/v14.0: (full report)
| suite_name | test_name | keyword_name | status | starttime | endtime |
|---|
Powered by osm-analytics