# Data Catalog ## Retrieving batch of data structures `POST` `https://my-organization.truedat.io/api/data_structures/search` Retrieves a first batch of structures with all corresponding metadata #### Request Body | Name | Type | Description | | ---------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | page\* | Integer | Number of page to be retrieved. For first page this value needs to be 0 | | size\* | Integer | Number of records to be returned in each page | | scroll\* | String | The `scroll` parameter (passed to the `search` request and to every `scroll` request) tells Elasticsearch how long it should keep the search context alive. Its value (e.g. `1m`, see [Time units](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#time-units)) does not need to be long enough to process all data — it just needs to be long enough to process the previous batch of results. | | scroll\_id | String | This will be included starting on the second request, using the information provided by the response of the first call. | | filters | List |

There are a number of filters that can be included in this request, including custom fields from notes.

| | sort | list | We will be able to order results by various fields. For example by structure name. | {% tabs %} {% tab title="200: OK The response will provide a list of structures with all the information available for each of them and also the scroll\_id to be used in next calls." %} ```javascript { "data": [ { "class": "table", "classes": null, "confidential": false, "deleted_at": null, "description": null, "domain": { "external_id": "Risks", "id": 383, "name": "Risks" }, "domain_id": 383, "external_id": "postgres://Postgres-Test/postgres/public/city", "field_type": "Table", "group": "postgres", "id": 5375339, "inserted_at": "2021-11-15T15:27:42.042258Z", "latest_note": null, "linked_concepts": false, "metadata": { "alias": "postgres-temp", "database": "postgres", "host": "localhost", "schema": "public", "type": "BASE TABLE" }, "mutable_metadata": null, "name": "city", "path": [ "postgres", "public" ], "source": null, "source_id": 187, "system": { "external_id": "Postgres-Test", "id": 106, "name": "Postgres-Test" }, "system_id": 106, "tags": [], "type": "Table", "updated_at": "2022-02-07T16:19:01.385206Z", "version": 9 } ], "scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAADTgwWZ0g4MUo0VGRRZXFfUDQ5ZjQyZTRxdw==" } ``` {% endtab %} {% endtabs %} ## Load data catalog assets metadata using csv files `POST` `https://my-organization.truedat.io/api/systems/:external_id/metadata` Using this API we will be able to load metadata into truedat's data catalog. If the data asset which we are loading already exists, Truedat will analyze whether there are any changes and in that case a new version of the data asset will be created. See below the description of the files used to upload metadata into Truedat: #### Path Parameters | Name | Type | Description | | ---------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | external\_id\* | String | Indicates the external identifier of the system for which we are going to load metadata. The system must have been created prior to loading the metadata. | #### Request Body | Name | Type | Description | | -------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | data\_structures\* | File | [File containing the structures to be registered and / or modified.](https://docs.truedat.io/v6.7/api-integrations/files-description#structure-file) | | data\_structure\_relations | File | [Parent / child relationships between structures contained in the structure file.](https://docs.truedat.io/v6.7/api-integrations/files-description#data-structures-relations) | | job\_id | Integer | Identifier of the job related to this load in case that it exists. This is used by connectors mainly. | {% tabs %} {% tab title="401: Unauthorized " %} ```javascript { // Response } ``` {% endtab %} {% tab title="202: Accepted Files have been accepted to be processed" %} ```javascript { // Response } ``` {% endtab %} {% endtabs %} ## Update metadata for a list of data structures `PATCH` `https://my-organization.truedat.io/api/systems/:external_id/metadata` Enables including aditional metadata for any given data structure or update current metadata. Currently only mutable metadata is supported. The request is processed asynchronously, the service will respond with **202 ACCEPTED** to an authenticated service account.
#### Path Parameters | Name | Type | Description | | ---------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | external\_id\* | String | Indicates the external identifier of the system for which we are going to load metadata. The system must have been created prior to loading the metadata. | #### Request Body | Name | Type | Description | | ------------------------------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------- | | op\* | String | replace / merge. Depends on whether we want to merge metadata with existing or just remove all metadata and include new information. | | values | Array | List of changes to be performed. For each element we will need to send the external\_id for the data structure and the metadata to be merged. | {% tabs %} {% tab title="202: Accepted Request is ok and will be processed asynchronously" %} ```javascript { // Response } ``` {% endtab %} {% tab title="401: Unauthorized " %} ```javascript { // Response } ``` {% endtab %} {% endtabs %}
Request Body Example ``` { "op": "replace", "values": [ { "external_id": "ns://foo/bar", "mutable_metadata": { "COLUMN_TYPE": "varchar(10)", "CHARACTER_MAXIMUM_LENGTH": "10" } }, { "external_id": "ns://foo/baz", "mutable_metadata": { "COLUMN_TYPE": "date" } }, { "external_id": "ns://foo/xyzzy", "mutable_metadata": {} } ] } ```