--- title: Transformation Framework description: "# Open-Source Code Repository links The release package of Transformation Framework is available on a dedicated Open-Source GitHub repository: - [https://github.com/DHS-TransformationFramework/esa_tf/releases](https://github.com/DHS-TransformationFramework/esa_tf/releases) # Public Docker Repository links The relevant Docker images are available on dedicated Public Docker Hub repositories: - [https://hub.docker.com/r/collaborativedhs/esa_tf_restapi/tags](https://hub.docker.com/r/collaborativedhs/esa_tf_restapi/tags)…" url: "https://collgs.esa.int/docs/transformation-framework/" updated: "2026-07-07" category: Software Guide --- # Transformation Framework # Open-Source Code Repository links The release package of Transformation Framework is available on a dedicated Open-Source GitHub repository: - [https://github.com/DHS-TransformationFramework/esa_tf/releases](https://github.com/DHS-TransformationFramework/esa_tf/releases) # Public Docker Repository links The relevant Docker images are available on dedicated Public Docker Hub repositories: - [https://hub.docker.com/r/collaborativedhs/esa_tf_restapi/tags](https://hub.docker.com/r/collaborativedhs/esa_tf_restapi/tags) - [https://hub.docker.com/r/collaborativedhs/esa_tf_worker/tags](https://hub.docker.com/r/collaborativedhs/esa_tf_worker/tags) # Installation and Configuration Guidelines Installation of the Transformation Framework takes place via docker compose. ## Software pre-requirements The following software must be pre-installed on the Virtual Machine: - Docker engine (>=20.10) - Docker compose (>=2.0) - make (>=3.81) - curl (>=7.75) - unzip (>=6.0) - tar (>=2.8) ## Installation The following commands will install a version of Transformation Framework software: ``` curl -LJO https://github.com/DHS- TransformationFramework/esa_tf/archive/refs/tags/v.zip unzip esa_tf-.zip cd esa_tf-/esa_tf make setup ``` In order to start the docker compose, it is then necessary to configure properly the following files: - .env - config/hubs_credentials.yaml - config/esa_tf.config Finally, the following command pulls the docker images from the Public Docker Repository and starts the docker compose: ``` nohup make up & ``` ## Configuration Please refer to the Installation and Configuration Manual for details about the configuration. Please find it here attached. [COPE-SERCO-TN-21-1218-Collaborative-Data-Hub-Software-Transformation-Framework-Installation-Configuration-Manual_v5.0](https://collgs.esa.int/wp-content/uploads/2026/07/COPE-SERCO-TN-21-1218-Collaborative-Data-Hub-Software-Transformation-Framework-Installation-Configuration-Manual_v5.0.pdf)[Download](https://collgs.esa.int/wp-content/uploads/2026/07/COPE-SERCO-TN-21-1218-Collaborative-Data-Hub-Software-Transformation-Framework-Installation-Configuration-Manual_v5.0.pdf) # API User Guide The Transformation Framework exposes a REST API, which stores and catalogues data. Through this interface, the Framework allows: - the retrieval of an input product to be transformed. - the submission of request for data transformation, including user defined parameters. - monitoring the progress of transformation processes. Please refer to [COPE-SERCO-IF-21-1178 - Collaborative Data Hub Software Transformation Framework ICD v5.0] for details about available APIs: [COPE-SERCO-IF-21-1178-Collaborative-Data-Hub-Software-Transformation-Framework-ICD_v5.0](https://collgs.esa.int/wp-content/uploads/2026/07/COPE-SERCO-IF-21-1178-Collaborative-Data-Hub-Software-Transformation-Framework-ICD_v5.0.pdf)[Download](https://collgs.esa.int/wp-content/uploads/2026/07/COPE-SERCO-IF-21-1178-Collaborative-Data-Hub-Software-Transformation-Framework-ICD_v5.0.pdf) ## Explore available Workflows ``` Method: HTTP GET {{SERVICE-ROOT-URI}}/Workflows ``` ## Monitor submitted Transformation Orders ``` Method: HTTP GET {{SERVICE-ROOT-URI}}/TransformationOrders ``` ## Monitor Transformation Orders with Manager user ``` Method: HTTP GET {{SERVICE-ROOT-URI}}/admin/TransformationOrders ``` ## Submit a Transformation Order #### "sen2cor" Plugin ``` Method: HTTP POST {{SERVICE-ROOT-URI}}/TransformationOrders Body: Content Type =application/json { "WorkflowId": "sen2cor_l1c_l2a", "InputProductReference": { "Reference": "S2A_MSIL1C_20240415T045911_N0510_R133_T43HDT_20240415T081534.zip" }, "WorkflowOptions":{ "Aerosol_Type": "RURAL", "Mid_Latitude": "AUTO", "Ozone_Content": 250, "Cirrus_Correction": false, "DEM_Terrain_Correction": false, "Resolution": 10 } } ``` #### "eopf" Plugin ``` Method: HTTP POST {{SERVICE-ROOT-URI}}/TransformationOrders Body: Content Type =application/json { "WorkflowId": "eopf_convert_to_zarr", "InputProductReference": { "Reference": "S1A_IW_GRDH_1SSH_20240415T074035_20240415T074100_053441_067BB8_CE1E .zip" }, "WorkflowOptions":{ "dask_compression": "zstd", "dask_comp_level": 1, "dask_shuffle": 2 } } ``` ## Request specific Transformation Order (by ID) ``` Method: HTTP GET {{SERVICE-ROOT-URI}}/TransformationOrders('') ``` ## Filter accessible Orders (by Status) ``` Method: HTTP GET {{SERVICE-ROOT-URI}}/TransformationOrders?$filter=Status eq 'completed' {{SERVICE-ROOT-URI}}/TransformationOrders?$filter=Status eq 'in_progress' {{SERVICE-ROOT-URI}}/TransformationOrders?$filter=Status eq 'failed' ``` ## Filter accessible Orders (by SubmissionDate) ``` Method: HTTP GET {{SERVICE-ROOT-URI}}/TransformationOrders?$filter=SubmissionDate ge '2022- 06-07T00:00:00' ``` ## Filter accessible Orders (by CompletedDate) ``` Method: HTTP GET {{SERVICE-ROOT-URI}}/TransformationOrders?$filter=CompletedDate le '2022- 06-07T00:00:00' ``` ## Keycloak Interface In case the Transformation Framework is configured to support Keycloak Interface for Authentication and Authorization, an Access Token from the Public Keycloak client is needed to be retrieved and used. Please refer to Installation and Configuration Manual [COPE-SERCO-TN-21-1218] and Keycloak Installation and Configuration Manual [COPE-SERCO-TN-21-1229] for details about the configuration involving Keycloak. Please find it here attached. [COPE-SERCO-TN-21-1229 - Keycloak Installation and Configuration Manual_v2.5](https://collgs.esa.int/wp-content/uploads/2025/02/COPE-SERCO-TN-21-1229 - Keycloak Installation and Configuration Manual_v2.5.pdf)[Download](https://collgs.esa.int/wp-content/uploads/2025/02/COPE-SERCO-TN-21-1229 - Keycloak Installation and Configuration Manual_v2.5.pdf) ## Access Token request ``` Method: HTTP POST {{KEYCLOAK-SERVICE-URI}}/auth/realms/{{KEYCLOAK-REALM}}/protocol/openid- connect/token Body: Content Type = application/x-www-form-urlencoded --data-urlencode "grant_type=password" --data-urlencode "client_id=" --data-urlencode "username=" --data-urlencode "password=" ``` Extract the "access_token" variable from obtained response. ## Queries Execution Queries towards the Transformation Framework instance should always provide an Authorization Header. ``` Method: HTTP GET Authorization: Bearer {{SERVICE-ROOT-URI}}/Workflows ``` ``` Method: HTTP GET Authorization: Bearer {{SERVICE-ROOT-URI}}/TransformationOrders ``` # Tutorials Please find available Tutorials at the following link: [Link](https://www.youtube.com/@collaborativedhsservice)