Transformation Management API - CData Sync Documentation

The Transformation Management API supports the following actions:

List All Transformations in a Specified Workspace
Create a Transformation in a Specified Workspace
Update a Transformation
Delete a Transformation
Execute a Transformation
Retrieve the Transformation History
Retrieve a Specified Transformation History
Retrieve the Transformation Status
List All Tasks
Retrieve Task Details
Add a Task
Update a Task
Delete a Task
Move a Task
Retrieve the Last Run of All Tasks
Retrieve the Task History

The sections below describe each of these actions.

List All Transformations in a Specified Workspace

The following command returns a list of all transformations within a workspace in Sync:

GET http://<SyncHost>/api.rsc/v2/workspaces/{workspace-id}/transformations

Path Parameter

Name	Description
workspace_id (required)	Identifies the specific workspace from which to retrieve the list of transformations.

Retrieve a Transformation

The following command retrieves a transformation:

GET http://<SyncHost>/api.rsc/v2/transformations/{transformations-id}

Path Parameter

Name	Description
transformation_id (required)	Identifies the transformation that you want to retrieve.

Create a Transformation in a Specified Workspace

The following command creates a transformation in a specific workspace:

POST http://<SyncHost>/api.rsc/v2/workspaces/{workspace-id}/transformations

Path Parameter

Name	Description
workspace_id (required)	Identifies the workspace in which to create the transformation.

Body Parameters

Name	Description
Name	Specifies the name of the transformation.
Type	Specifies the type of transformation. Values are SQL, DBT, and DBT-Cloud.
Destination-Connection.id	Specifies the identifier (Id) of the destination connection.
Destination-Connection.name	Specifies the name of the destination connection.
Schedule	Specifies whether to execute a job according to a specific scheduler. Set the `Enabled` subparameter to True to execute the job per a specific scheduler.
Trigger.Type	Specifies the trigger type.
Trigger.After_Job	Specifies the job (Id) after will the transformation will run.
Trigger.After_Job_Tasks	Specifies a list of tasks (Id) that trigger the job.
Trigger.Schedule_Cron	Specifies the cron expression for the schedule.
Alerts.Send_Email	Specifies whether to send email alerts after job completion. Set this parameter to True to send the email alerts.
Alerts.Email_To	Specifies the email address to which alerts should be sent.
Alerts.Email_Error_Only	Specifies to send an email only when an error occurs.
Settings.Enable_Parallel_Processing	Specifies whether parallel processing is enabled.
Settings.Worker_Pool	Specifies the number of parallel worker pools.
Settings.Verbosity	Specifies the log verbosity level of the job. Verbosity levels are None, Error, Info, Transfer, and Verbose
Settings.Git_Repository	Specifies the URL of the Git repository where project resides.
Settings.Git_Token	Specifies the token that is required for accessing Git.
Settings.DBT_Project_Path	Specifies the project path of DBT transformation.
Settings.DBT_Schema	Specifies the schema for the DBT transformation.
Settings.DBT_Threads	Specifies the number of threads to use in a DBT transformation.
Labels	Specifies an array of `label` strings.
Pipeline_Id	Specifies the identifier for the pipeline.
Queries	Specifies the queries to use in the job.

Update a Transformation

The following command updates a transformation:

PUT http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}

Path Parameter

Name	Description
transformation_id (required)	Identifies the transformation that you want to update.

Body Parameters

Name	Description
Name	Specifies the name of the transformation.
Destination-Connection.id	Specifies the identifier (Id) of the destination connection.
Destination-Connection.name	Specifies the name of the destination connection.
Trigger.Type	Specifies the trigger type. Types of triggers include None, Scheduled, and AfterJob.
Trigger.After_Job	Specifies the job (Id) after will the transformation will run.
Trigger.After_Job_Tasks	Specifies a list of tasks (Id) that trigger the job.
Trigger.Schedule_Cron	Specifies the cron expression for the schedule.
Alerts.Send_Email	Specifies whether to send email alerts after job completion. Set this parameter to True to send the email alerts.
Alerts.Email_To	Specifies the email address to which alerts should be sent.
Alerts.Email_Error_Only	Specifies to send an email only when an error occurs.
Settings.Enable_Parallel_Processing	Specifies whether parallel processing is enabled.
Settings.Worker_Pool	Specifies the number of parallel worker pools.
Settings.Verbosity	Specifies the log verbosity level of the job. Verbosity levels are None, Error, Info, Transfer, and Verbose
Settings.Git_Token	Specifies the token that is required for accessing Git.
Settings.DBT_Schema	Specifies the schema for the DBT transformation.
Settings.Git_Repository	Specifies the URL of the Git repository where project resides.
Settings.DBT_Threads	Specifies the number of threads to use in a DBT transformation.
Settings.DBT_Project_Path	Specifies the project path of DBT transformation.
Labels	Specifies an array of `label` strings.
Pipeline_Id	Specifies the identifier for the pipeline.

Delete a Transformation

The following command deletes a transformation:

DELETE http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}

Note: You cannot delete a transformation that is running. Path Parameter

Name	Description
transformation_id (required)	Identifies the transformation that you want to delete.

Execute a Transformation

The following command executes a transformation:

POST http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}

Note: You cannot delete a transformation that is running. Path Parameter

Name	Description
transformation_id (required)	Identifies the transformation that you want to delete.

Body Parameter

Name	Description
Tasks	Specifies a list of task Ids to execute. If the value is null, the entire transformation is executed.

Example Request

{
  tasks:[
    "870237973789151200",
    "870875808946851841",
    "870875808946851842"    
  ]
}

Retrieve the Transformation History

The following command lists the transformation history:

GET http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/history

Path Parameter

Name	Description
transformation_id (required)	Identifies the transformation for which you want to list the history.

Retrieve a Specified Transformation History

The following command retrieves a specified transformation history:

GET http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/history/{history_id}

Path Parameters

Name	Description
transformation_id (required)	Identifies the transformation for which you want to retrieve the history.
history_id (required)	Identifies the specific transformation history that you ant to retrieve.

Retrieve the Transformation Status

The following command retrieves the transformation status:

GET http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/status

Path Parameters

Name	Description
transformation_id (required)	Identifies the transformation for which you want to retrieve the status.

List All Tasks

The following command lists all tasks in a transformation:

GET http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/tasks

Path Parameter

Name	Description
transformation_id	Identifies the transformation for which you want to retrieve a list of tasks.

Retrieve Task Details

The following command retrieves the details for a specific task in a transformation:

GET http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/tasks/{task_id}

Path Parameters

Name	Description
transformation_id (required)	Identifies the transformation for which you want to retrieve a list of tasks.
task_id (required)	Identifies the specific task for which to retrieve details.

Add a Task

The following command adds a task to a transformation:

POSt http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/tasks

Path Parameter

Name	Description
transformation_id (required)	Identifies the transformation for which you want to add a task.

Body Parameters

Name	Description
Query	Specifies the SQL statement that defines the operation performed by the task.
Is_Active	Specifies whether the task is enabled and will be executed when the job runs.

Example Request

{
  "query": "",
  "id": null, /* dbt Cloud job Id. When the value for Id is not null, `query` corresponds to the job name of dbt Cloud. */
  "is_active": true
}

Update a Task

The following command updates an existing task in a transformation:

PUT http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/tasks/{task_id}

Path Parameters

Name	Description
transformation_id (required)	Identifies the job for which you want to add a task.
task_id (required)	Identifies the task that you want to update.

Body Parameters

Name	Description
Query	Specifies the SQL statement that defines the operation performed by the task.
Is_Active	Specifies whether the task is enabled and will be executed when the job runs.

Example Request

{
  "query": "",
  "is_active": true
}

Delete a Task

The following command deletes a task from a transformation:

DELETE http://<SyncHost>/api.rsc/v2/transformations/{transformation_id}/tasks/{task_id}

Note: You cannot delete a task that is running. Path Parameters

Name	Description
transformation_id (required)	Identifies the transformation for which you want to delete a task.
task_id (required)	Identifies the task that you want to delete.

Move a Task

The following command moves the position of a task within a transformation:

POST /transformations/{transformation_id}/tasks/{task_id}/move

Path Parameters

Name	Description
transformation_id (required)	Identifies the transformation that contains the task that you want to move.
task_id (required)	Specifies the Id of the task that you want to move.

Retrieve the Last Run of All Tasks

The following command retrieves the last run of all tasks within a transformation:

GET /transformations/{transformation_id}/tasks/last-run

Path Parameter

Name	Description
transformation_id (required)	Identifies the transformation for which you want to retrieve the status of the last run of all tasks.

Retrieve the Task History

The following command retrieves the task history for a job:

GET /transformations/{transformation_id}/tasks/{task_id}/history

Path Parameters

Name	Description
transformation_id (required)	Identifies the transformation that contains the task history that you want to retrieve.
task_id (required)	Specifies the Id of the task with the history you want to retrieve.

​List All Transformations in a Specified Workspace

​Retrieve a Transformation

​Create a Transformation in a Specified Workspace

​Update a Transformation

​Delete a Transformation

​Execute a Transformation

​Retrieve the Transformation History

​Retrieve a Specified Transformation History

​Retrieve the Transformation Status

​List All Tasks

​Retrieve Task Details

​Add a Task

​Update a Task

​Delete a Task

​Move a Task

​Retrieve the Last Run of All Tasks

​Retrieve the Task History

List All Transformations in a Specified Workspace

Retrieve a Transformation

Create a Transformation in a Specified Workspace

Update a Transformation

Delete a Transformation

Execute a Transformation

Retrieve the Transformation History

Retrieve a Specified Transformation History

Retrieve the Transformation Status

List All Tasks

Retrieve Task Details

Add a Task

Update a Task

Delete a Task

Move a Task

Retrieve the Last Run of All Tasks

Retrieve the Task History