Data Catalog
This module allows the consultation and enrichment of metadata obtained from the organization's systems. For this, search, filtering and browsing options are provided through the catalog. Additionally, this module connects with both the glossary of concepts and the data quality glossary to allow navigation between the different modules.
When accessing the Catalog we will obtain a list of the available systems, showing for each one of them the volume of structures that have been imported as well as the count by type of structures when hovering over the number.
From this point on we will have several options to find the structure that we want to consult.
Discover your system metadata: by clicking on the system we will find a tree-like navigation that you can navigate in order to find your metadata. In this type of navigation we will also have a search box that will allow us to filter the list of current structures to easily find the one that we are looking for.
Filtering: You will be able to filter using information discovered from the data source and aditional information documented in your installation using the data catalog templates. Some filter examples: structure type, last changed date, linked to business concept, has aditional information.
Search: We will be able to search inside the complete matadata, including any manual fields, structure path and description.
Save filters: User will have the option to save his most used filters. In order to do that user will need to provide a name to the combination of filters and will be provided with this as an aditional quickfilter every time he/she enters the data catalog.
Structures
When viewing the detail of a structure we can see the following information:
Domain: the data domain to which the structure belongs.
System: is the metadata repository from which the information must be obtained
Group: functional division of the system.
Imported: date the structure was included in the catalog for the first time.
Updated: date the structure information was last modified in the catalog.
The detail of structures and fields will make a series of tabs available to us, showing or not depending on the casuistry.
Fields: Those structures that are composed of fields will have a tab that shows them. If it is a structure that does not have fields, this tab will not be displayed.
Notes: In the event that the type of structure has an associated template (see the corresponding administration chapter), a tab will be displayed to manage the notes information. Notes are manually managed metadata for a data structure. We include a workflow in case that this information needs to be reviewed before approval.
Profiling: It will be shown if we have profiling information for the children fields of the structure or data distribution information for the structure we are viewing.
Quality: The list of implementations / quality rules in which the structure we are consulting is being validated will be displayed.
Linkage: Tab that will always be shown showing existing links and giving the option to authorized users to generate new links.
Versions: The different versions available for the structure will be displayed with the option of being able to browse to consult them.
Audit: Details of all manual changes that have been made to the structure are displayed.
Domain modification
When metadata are loaded into truedat a domain is assigned by the connector as specified in the data source. In the data catalog you will be able to reassign the domain of a specific data structure. This is specially important if you are planning to manage diferent permissions to data structures depending on the data domain the are assigned.
Fields
It shows a list of the fields of the application if you have any. For these fields, it shows the name of the field, links to business concepts, link to the traceability of the field and other metadata columns that may have been loaded when obtaining the information from the system.
From this field listing screen we can perform the following actions:
Navigate to the detail of a field by clicking on the field name.
Navigate to the traceability of a field if you have traceability for that field available.
Navigate to the concept associated with our field if there is any (Terms Column)
Notes
This tab will be available in those structures whose type of structure is has been assigend a template. It will allow us to view the metadata entered in our tool to complement the metadata obtained automatically.
There are some permissions linked to this functionality which will allow the users to complete different steps of the process in order to publish a note. In case that you do not need an approval workflow you can just setupo a permission to directly publish for roles that are allowed to modify the notes.
Users will be able to perform different actions depending on permission and note status using following workflow to manage structures notes and depending on user permissions
Profiling
The profiling of a structure allows us to better understand our data, in such a way that we can better understand the structure that we have and the uses that can be given to it to obtain information within the organization. It will also help us to identify quality problems that we are having with our data in the first instance. In order to have the profiling information loaded, it will be necessary that the corresponding integration tasks have been carried out in order to obtain said profiling data from those systems for which we have such a need.
Two types of profiling display are currently available:
Outlined Summary of Fields of a Structure
The list of fields in the structure that have profiling information and the following information will be displayed, if available:
% unique values
% null values
Lowest data
Highest data
Fashion (most repeated value)
If there is a distribution of values, an icon will be displayed that will link to the outlined field.
Distribution of values for a field
In case of having information on the distribution of values, the profiling tab will be displayed, containing a graph with the different values that we have for said field.
Value patterns
If we have loaded pattern information using the profiling connector, we can also consult this distribution helping us to better understand our data.
This feature is not available for all data sources
Lineage and Impact
If traceability information is available for the selected structure, lineage and impact tabs will be displayed where both diagrams can be viewed quickly. It will also allow us to navigate through the graph as explained in the Data Lineage section
Lineage: It will inform the origin of the data. What information are we using to generate the data?
Impact: It will be shown in which other data structures the data we are consulting is being used. Where would it impact if we make a change in this data?
Quality
A list is shown with the quality implementations where the structure we are consulting is being referenced. Additionally, information on the rule to which this implementation belongs is displayed and navigation is allowed both for the quality rule and for the implementation to which the list refers. The last quality result will also be displayed if we have it.
Linkage
This tab will allow us to consult and manage the links that are available for a structure / field.
Versions
In case of having several versions loaded in the application, it will allow us to navigate between them. This navigation between versions will allow us to consult the fields that made up the structure in previous versions.
When consulting previous versions we will see marked those fields of our structure that have undergone modifications with respect to the current version. They will be marked in the event that these fields have been deleted or modified.
Audit
The data structure audit will show us all the manual actions carried out with said structures, being able to view the changes that have been made in the additional information of said structure as well as the people who have made said changes.
Structure confidentiality
For those users with the appropriate permissions, there is the option of marking a structure as confidential. A structure that is marked confidential will only be visible to people who have permission to manage confidential structures and view confidential structures about the domain in which the structure is located. If you do not want to use this functionality, you simply should not activate the permission in any of the defined roles.
Structure tagging
If your administrator has created structure tags and permissions have been assigned, you will be able to link a tag to structure including a description on why this tag has been assigned to the structure.
Once a tag has been linked to a structure any user with permission to see the given structure will also be able to see the tag and the description (making hover over the tag).
Sharing structures
You will be able to share your structures with others using the corresponding action on the right upper corner of the structure information. This will produce a notification that will be sent via email.
You will need to include SMTP server configuration in your installation in order to receive notifications via email
Bulk update of additional information
From the main screen of the data catalog, administrator users will have the option of performing a massive update of the additional information associated with the structures. In order to carry out this massive update, the following must be taken into account:
Additional information can only be massively updated for structures that have this option active, that is, have an associated template. Furthermore, only one type of structure can be updated in each action.
All fields are optional and only those for which a value is entered will be updated on the structure. All the fields of the template / type by which it has been filtered will appear.
Once the fields to be updated are filled in and when we press the "Save" button we will be asked for confirmation to carry out the update, informing us of the number of structures to be updated.
Catalog information export
Carrying out any search or filtering within the data catalog, we will have the option of exporting the metadata information contained in the catalog in csv format, so that these data can be processed in a third Excel type application.
Last updated
