Create a Macro
4P offers effective and user-friendly tools to create macros. In the list of available macros, click the button Create macro.
Note
Users with no PHIS account cannot create macros.
A window shows up requiring to fill in several general information:
Name: fill in the name of the macro. Make sure to choose a unique, short and explicit name. A version number can be included in the name to follow version changes more easily.
Compatible vectors: select the vector(s) which this macro is compatible with.
Description: enter a short description of the macro’s purpose and specificities.
Technical guide: optionally, attach a document describing the macro’s features more accurately.
Click the button Validate to create the macro with the given information. Then the details page of the macro shows up.
Note
At anytime, and as long as the macro is not published, it is possible to modify its general information. To do so, click on the button Edit, next to the name of the macro.
Hint
At that moment, the macro is invalid because no data type nor any module was added to the workflow. We need to create the workflow to be able to use that macro in a process.
Create the workflow
Open the Workflow tab to start defining the steps of the computation chain as well as the input data types. For now the workflow is incomplete.
Add input data types
The first step consists in defining the types of data that the macro will take as inputs. On the right-hand side of the workflow editor, the Add a data selector contains a list of data types compatible with the defined Vector. Select a data type (here “L0 Drone - RGB Images”) and click the Add button.
The chosen data type is now added to the macro inputs. A macro accepts one or several input data types.
It is possible to remove a data type from the macro inputs by simply clicking the cross icon on the data type.
Add modules
The second step consists in composing the chain of computation modules. Only the modules compatible with the selected input data types are proposed. To add a new module to the workflow click the Add button on the left-hand side of the workflow editor at the desired position in the chain.
A list of available modules shows up. Pick any module by clicking the Add button next to it.
This action adds the module in the workflow. To remove a module from the workflow, simply click the cross icon on it.
Note
It is possible to add as many modules as necessary for the computation chain to be complete.
Set input parameters
Each module of the chain must be carefully configured by providing each of its input parameters with a value. To do so, select the module in the workflow to open the configuration panel. On the right-hand side of the workflow editor a list of input parameters is displayed.
Hint
4P automatically selects the best match for each input parameters, but make sure to always check each parameter to avoid process issues.
Make sure to select the right source for each input parameter. Sources are of different types:
Data: the value is provided by one of the parameters contained in the input data types. In the example above, the parameter Plots coordinates value will be provided by the parameter Plots coordinates contained in the RGB Images data type.
Module: the value is provided by one of the preceding modules outputs. In the example above, the parameter Photoscan processed data value will be provided by the output Photoscan data of the Process module.
Default value: the value is set to the default value of this parameter if it exists. In the example above, the parameter Extract samples is a boolean set to its default value true.
Manual: the value is a constant manually set in the macro configuration. In the example above, the parameter Minimum pixel percent is manually set to 33.33.
Process: this option means that the user will be asked to set this value while starting a new process. This new parameter will then be required every time the macro is used (see the picture below).
Input not defined: this option is available for optional parameter and indicates that the macro does not use the parameter.
Output data types
When the macro is consistent and generates referenced data, 4P automatically detects what will be the output data types. These are displayed in the box Output dataset in the left-hand side part of the workflow editor.
Note
For some reasons (for example when a new module has been added to the platform), it may happen that a data type containing the desired outputs does not exist in 4P. In this case, it is possible to create a new data type. Please refer to the section Create new data types for more details.
Publish the macro
Note
Only Super-Administrators can Publish a macro. After testing the macro contact one of them to make it published.
Resolve errors in a workflow
In certain circumstances, a macro might be invalid due to unresolved errors in the workflow. Problems that need to be solved are highlighted in red in the workflow editor. We’ll consider a few common errors.
Parameter not found
In this situation, the workflow editor highlights an error with a certain module (the module Phenoscript - Process in our example). When opening the module configuration panel, one or several parameters are not properly connected to their source, either because the wrong source was chosen by 4P or by the user, or because a compatible input data type is missing.
To solve this problem, try one of the following actions:
If the source is Manual make sure to fill in the value field
Browse the source selector and make sure to choose the most appropriate source
If the source selector is empty, make sur to use the proper input data type
Contact the support if no solution can be found
Output data types not identified
In this situation, 4P is not able to identify valid output data types because the output parameters of each module don’t match any referenced data type in the database.
To solve this problem, try one of the following actions:
One or several modules may be missing to obtain a complete output. Make sure to add the proper modules and in the correct order. In the example above, the module Phenoscript - Process is mandatory to obtain an interesting output. Adding that module will solve the problem.
The last module of the chain is a new module on 4P and no output data type as been defined. Try to create a new data type (see the section Create new data types)
If any problem persists, please contact the support.
Case of parallel processes
Some computation modules are designed to process only one unit of a dataset, whereas the dataset itself contains a list of units to be processed. In this situation, 4P is able to execute a workflow in a parallel loop on these input data. Each task will then be executed independently so that computation time is kept low.
Hint
4P is backed up by a cluster of virtual machines that are able to distribute computation tasks on the available resources.
Enter into a parallel loop
A module might take a single file as input parameter. However, 4P may propose sources of types file and list of files. When the user selects a source of type file the workflow is executed in a normal mode. However, when the user chooses a source of type list of files, the distribute icon appears.
As shown in the picture above, if that case occurs, 2 modes are available:
Distributed mode: when the distribute icon is ON (which is the default behavior), the process will execute the module in parallel for each file of the list. Consequently, the ouput will also be a list of objects.
Normal mode: when the distribute icon is OFF, the process will execute the module on the first file of the list only. The other files of the list will be ignored.
When a module opens a parallel loop in the workflow, it is displayed in the left-hand side of the workflow editor.
Add a module into the parallel loop
The next modules in the workflow will automatically be added to the parallel loop if they take only one file as an input, coming from the preceding module.
The module’s configuration panel is flagged with the distribute icon to indicate that it belongs to a parallel loop.
Exit the parallel loop
When a module takes a list of files as an input parameter, and when that parameter is connected to the ouput of a module in the parallel loop, 4P understands that the parallel loop is over and must be exited.
