Manage Data Types
4P features specific entities to categorize data and results called data types. Data types are available from the main left menu. The current section describes how to view, create and edit data types in 4P.
Note
A data type is a typed set of data items or artifacts (files, metadata, images, etc.) defining an independant processable entity. It is used as macro inputs and outputs, as well as for cutting up data groups.
Note
Users with no PHIS account cannot manage data types.
For example a pre-processed Phenomobile data group can be cut up in 3 different entities: RGB Images, Lidar Data and Multispectral Images. These entities can be processed seperately. That’s why 4P defines a data type for each of these sets of data. These data types contain all the data and metadata needed to be processed independantly.
View data types
Click the Data types main menu item to visualize the list of existing data types in 4P.
For each data type, the table displays the following attributes:
Name: the name used for the data type in 4P. Several data types may have the same name if the vector is different.
Data level: the level of transformation for this data:
L0 = Raw data: immediately extracted from acquisition with no transformation of any kind;
L1 = Intermediate data: obtained by a first transformation consisting in extracting, correcting or preparing data for advanced treatment;
L2 = Final data: obtained by transformation and interpretation of pre-processed data to a measurable biological or physical quantity.
Vector: the vector used for acquiring data: Drone or Phenomobile
Description: the data type description
Metadata: the list of metadata available in this data type
Status: the publication status which can be published, archived or draft
Hint
The list of data types can be filtered by name, data level, vector, metadata and status.
Click one data type in the list to open its details page.
The information previously displayed in the list are displayed here also plus a few more details about the list of included metadata:
The metadata label
The metadata ID, which is the internal ID 4P uses to access it
The metadata Type which can be a file, a folder, a string, an integer, a float, a date, a list, etc.
The metadata Description
The metadata Value which is the value this parameter will be set to by default
Create new data types
Caution
Creating a data type is an advanced operation and must be done by advanced users only! Before creating a new one, make sure a similar data type doesn’t exist yet and consult the other users and super-administrators to ensure it will meet a real need.
To create a new data type, click the Create data type button on the data types list page. This action opens the form for creating the data type.
Fill in general information
First, fill in the general information of the new data type:
Name: Give a unique, short (< 24 characters) and explicit name describing the content of this data type. Preferably use a Title Case style (capitalize each word) with correct spelling in English.
Description: Give a concise description of the content of the data type. The vector should be added to the description as well.
Data level: Select the data level of transformation:
L0 = Raw data
L1 = Intermediate data
L2 = Final data
Vector: Select the acquisition vector chain which this data type is part of : Drone or Phenomobile.
Computed from a parallelized process: Check this box if the data type will be an output of a parallel process (typically that is usually the case for Literal and Phenomobile processes which run in parallel over the plots).
Once general information is set, add all the metadata that will be available in this data type.
Add existing metadata (L1/L2)
For L1/L2 data types, it is possible to add existing metadata which are metadata already available in 4P and coming from other data types, modules or the system itself.
Use the Add an existing metadata button to open the list of available metadata in a popup. Then use filters to reduce the list of metadata. Finally click the Add button.
Adjust the metadata value, particularly the relative paths of resulted files. Use the wildcard symbol * when referring to list of files respecting a certain pattern.
Hint
How to identify the metadata? Determine the metadata that should be included in this data type so that a process will have every needed information for running successfully.
Note
If the data type is computed from a parallelized process the type of an existing metadata will automatically be converted to a compatible type (for example, Files will become List of files).
Add new metadata (L0)
For L0 data types, it is possible to add new metadata which value will be set during the loading process.
Note
Please refer to the Add Raw Data section to learn more about how to load raw acquisition data on 4P.
Use the Add a new metadata button to open the metadata configuration popup.
Then fill in the required information:
Label: The name of the metadata. Keep it short (maximum 30 characters) and meaningful.
ID: The 4P internal reference for this metadata. Use the camelCase format with no space, nor special characters.
Description: Explain the metadata content. Keep it concise.
Type: Can be a file, folder, string, integer, float, boolean or a list.
Required: If checked, the metadata will be mandatory when loading the dataset. If the type is a file or a folder, 4P will check if the targeted elements are present in the loaded archive.
Editable: If checked, the metadata value will be editable when loading the dataset.
- Validation: The kind of validation performed by 4P when loading the dataset:
The Standard validation will only make sure the metadata meets the required option and corresponds to the type
The Min and Max validation will allow to enter a minimum and/or a maximum limit to the metadata numerical value
The Static list validation will make sure the metadata value can only take one of the given values
The Dynamic list validation will make sure the metadata value can only take one of the given values dynamically generated when loading the dataset. Typically, these values are provided by PHIS while connecting to an installation.
Value: The default value the metadata will take when loading the dataset.
Click the Add button to add the current metadata to the data type.
Create the data type
Finally, click the Add data type button to finalize the data type creation. It is then available in the data types list and can be used in macros.
Important
Once created the data type remains in draft mode and will be visible only by the users of the current installation. A data type needs to be published do be visible by all users and usable in valid macros.
Edit and delete data types
It is possible to modify or delete a data type by using the corresponding buttons.
Important
A data type can be modified or deleted as long as it is in draft mode.
In any case it is possible to duplicate a data type, to create an exact copy of it, and then adjust the information from there in order to create a new and unique data type.
Publish a data type
Only users with super-administrator privileges can publish a data type. Once published, it is visible and usable by all users of 4P and can not be deleted nor modified. Super-administrators have access to the following advanced actions:
Publish: Publish an element by making it final and visible by all users of the platform
Archive: Archive an element by hiding it from all users of the platform
Unpublish: Unpublish an element by making it back to a draft state
Unarchive: Unarchive an element by making it back to a published state
Hint
Modules, Data types and Macros offer the same publication control by super-administrators