Check and Retrieve Data

In addition to process logs, 4P offers additional features to monitor and determine whether a process and its generated data are valid. This part presents three of these features.

Check the process validation report

Note

This feature is not yet available but is scheduled for a future release of 4P.

Check the process success rate

Some processes are executed in parallel (for example, Phenomobile, or Literal data extractions). In that case each input data item is being processed by a separate and independant execution thread. When some of these parallel executions fail (due to corrupted input data), the overall process doesn’t fail but continues executing the other data items. Eventually, if some data items are processed successfully, the overall process will be considered as a success (as illustrated in the picture below).

../_images/process-data-overall-process-success.png

A detailed report of the overall parallel process is available in the Success rate box of the process detail page.

../_images/process-data-success-rate-box.png

This report gives the following information:

  • The global process success rate indicates in percent how many items have been processed successfully considering all the processing tasks (92% in the example above, corresponding to 11 success on 12 items).

  • The success rate by task indicates how many items have been processed successfully for each processing task (here 100%, 75% and 100%)

  • The table underneath indicates which input data items are in success (OK) or in error (KO) for each processing task.

../_images/process-data-item-ko.png

When a data item is in error for a specific task it is possible to find out more about the failure by clicking the exclamation mark next to the KO symbol.

Important

When the global process success rate is 0 the process is considered a failure.

../_images/process-data-success-rate-status-warning.png

When the process success rate is not 100%, the status label of this process will display a warning icon to let the user know that something went wrong for some data, even though the overall process is considered successful. This icon will be present wherever the process status is shown (ie process page, task history, monitoring and data page). If such an icon is visible, it is up to the user to check what went wrong and decide whether to invalidate the generated data or not.

Retrieve Temporary Data Files

4P also offers the possibility to retrieve temporary data from a process still running or which has failed.

../_images/retrieve-temp-data.png

In the data detail page the button Retrieve data might be available. This button attempts to download data from the running temporary folder of the 4P drive.

Important

Notice that it is not garanteed to obtain the desired data as the process might be running or in an unstable state. If no data is found, the downloaded archive will be empty.