> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sync.cdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# DBT Transformations

export const siteNameShort = "Sync";

export const siteName = "CData Sync";

{siteName} now integrates with *dbt Core* (a data building tool by dbt Labs™), which enables you to create powerful transformations. This integration enables you to develop, control, and run transformations to build Python models in a supported destination database.

You have complete control over your project repository and the ability to write SQL SELECT statements to transform your data and build models. {siteNameShort} simply creates a pipeline for you to run those transformations in your chosen destination.

If you only need to execute simple transformations, then basic SQL transformations are sufficient. However, for advanced modeling requirements, transformations that use dbt Core models offer more robust capabilities. For example, if your models contain dependencies, a transformation that uses a dbt Core project recognizes and honors those dependencies, enabling the models to build in the correct order.

## Supported Destinations

Transformations of type DBT are supported for the following destinations:

* [Snowflake](../../connections/destinations/snowflake-destination)
* [Amazon Redshift](../../connections/destinations/amazon-redshift-destination)
* [Databricks](../../connections/destinations/databricks-destination)
* [PostgreSQL](../../connections/destinations/postgresql-destination)
* [Google BigQuery](../../connections/destinations/google-bigquery-destination)

## dbt Core Details

### Setting Up Tools for DBT Transformations

You need to set up dbt Core and other tools first to enable DBT transformations, as follows:

1. Install Git for Windows and Python 3.7 or later for Microsoft Windows.

   <Note>An application like Git is not required. However, such a repository is recommended because it provides versioning, enables you to share projects, and gives you more control of your source code.</Note>

   In Git, locate and copy the following information for later use in the {siteNameShort} settings for a DBT transformation:

   * <a name="url" />**Git Repository URL:** The repository URL is available in your Git repository window. Click **Code** to open the **Clone** dialog. The URL is on the **HTTPS** tab in the dialog. Copy the URL by clicking the copy icon on the right end of the URL field. Save the icon for use later in {siteNameShort}.
   * <a name="pat" />**Personal Access Token:** To obtain your personal access token:

     <ol type="a">
       <li>Click your profile icon in the top right of the Git window and select <strong>Settings</strong> to open the <strong>Profile settings</strong> page.</li>
       <li>Select <strong>Developer Settings > Personal access tokens</strong>. This selection opens the <strong>New personal access token</strong> pane.</li>
       <li>Select the <strong>Repo</strong> check box in the <strong>Select scopes</strong> category. (You can set other criteria in this category as well, but it is not necessary.) For example, you might want to choose an expiration period (or no expiration) for your token under the <strong>Expiration</strong> category.</li>
       <li>Click <strong>Generate token</strong> at the bottom of the page. Copy and save the generated token for use later in {siteNameShort}.</li>
     </ol>

   For more details about obtaining a personal access token, see <a href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token" target="_blank" rel="noopener noreferrer">Creating a personal access token</a>.

<p>2. Install dbt Core and the adapter plug-in for your destination on the same machine where {siteNameShort} is installed. Plug-ins are available for each of the destinations that are <a href="#supported-destinations">listed earlier</a>.</p>

<p>3. Download your dbt projects locally by cloning your Git repository to your local machine. {siteNameShort} clones the repository upon setup and before each run to ensure that any changes to the project are reflected in the next run from {siteNameShort}.</p>

<p>4. Create a transformation in {siteNameShort} (using dbt projects that contain YML or SQL files), as described in the next section.</p>

### Creating a DBT Transformation

To create a dbt transformation:

1. Open {siteNameShort} and navigate to the **Transformations** page.
2. Select the job on which you want to run a transformation. Then, click **Add Transformations**.
3. Fill out the fields in the **Add Transformation** dialog box, as follows:
   * <strong>Name:</strong> Enter a name for your transformation.
   * <strong>Type:</strong> Select <strong>dbt Core</strong> as the transformation type.
   * <strong>Project Folder:</strong> Select one of the following options. The default option is <strong>Local</strong>.

     <p><strong>Local:</strong> Enter the path for your existing local project folder. For this option, you also need to enter your dbt project folder page in the <strong>Folder Path</strong> field.</p>

     * <strong>Github:</strong> Enter the URL for the project that you [created earlier in Git](#url). For this option, enter the following information:
       * <strong>Git Repository URL:</strong> Enter your Git repository URL in the form [https://github.com/](https://github.com/)<em>Owner</em>/<em>Repository</em>.git.
       * <strong>Token:</strong> Enter the personal access token that you [copied earlier in Git](#pat).
   * <strong>Destination Schema:</strong> Add a destination schema of your choice.
4. Click **Add Transformation**.

## dbt Cloud Details

### Connecting {siteName} to dbt Cloud

*dbt Cloud* is an integrated development environment (IDE) in which you can write, run, and test code as well as manage version control in your dbt project from your browser (no command-line use is required).

To connect to dbt Cloud:

1. Open the **Advanced** tab on the {siteNameShort} **Settings** page and navigate to the **dbt Cloud Settings** section.

2. Click the edit icon for that section to open the **dbt Cloud Settings** dialog box.

3. Enter your account identifier (Id) in the **dbt Cloud Settings** section. If you do not know your account Id, you can retrieve it as follows:
   1. Login to your dbt Cloud Account Information Center.
   2. Click your profile image and name at the bottom of the left menu pane. This action opens another menu.
   3. Select **Your Profile > Account**. Your account Id appears in the **Account ID** field (in the **Account information** section), as shown below:

      <img src="https://mintcdn.com/cdata-sync/TV-HFHPdmNi5_nZ9/images/dbt-Transformations_Connecting-CData-Sync-to-dbt-Cloud_1.png?fit=max&auto=format&n=TV-HFHPdmNi5_nZ9&q=85&s=256d79237217ebabf2c90c843f1f863a" width="600" data-path="images/dbt-Transformations_Connecting-CData-Sync-to-dbt-Cloud_1.png" />

4. Enter either a dbt Cloud personal access token (PAT) or a service token in the **Token** field.
   * If you do not have a PAT, you can create it by selecting **Account Settings > API Tokens > Personal Tokens** in your dbt Cloud Account Information Center. Copy the new token and return to {siteNameShort} to paste it into the **Token** field in the **dbt Cloud Settings** dialog box.
   * If you do not have service token, you can create it by selecting **Account Settings > Service Tokens > New Token**. Copy the new token and return to {siteNameShort} to paste it into the **Token** field in the **dbt Cloud Settings** dialog box.

5. Enter your base URL in the **dbt Cloud Settings** dialog box.

   <img src="https://mintcdn.com/cdata-sync/TV-HFHPdmNi5_nZ9/images/dbt-Transformations_Connecting-CData-Sync-to-dbt-Cloud_2.png?fit=max&auto=format&n=TV-HFHPdmNi5_nZ9&q=85&s=c1ee0a016aa4f902ad6b4ce71e8e2187" width="400" data-path="images/dbt-Transformations_Connecting-CData-Sync-to-dbt-Cloud_2.png" />

   <Note>To locate your base URL, open your dbt Cloud Account information Center and select **Your Profile > Account > Access URLs**. The base URL is listed in the **Access URL** text box.</Note>

6. Click **Test Connection** to confirm that you are connected. If the connection is successful, click **Save** to save your information.

### Creating a DBT Cloud Transformation

To create a dbt Cloud transformation:

1. Open {siteNameShort} and navigate to the **Transformations** page.
2. Select the job on which you want to run a transformation. Then, click **Add Transformations**.
3. Fill out the fields in the **Add Transformation** dialog box, as follows:
   * <strong>Name:</strong> Enter a name for your transformation.
   * <strong>Type:</strong> Select <strong>dbt Cloud</strong> as the transformation type.
   * <strong>dbt Job:</strong> Select a dbt job from the drop-down list.
4. Click **Add Transformation**.

## Specifying DBT Transformation Settings

To modify transformation settings, select **... > Edit** at the end of the transformation row. The settings page contains three tabs from which you can manage settings:

* <strong>Overview</strong>
* <strong>Transformations History</strong>
* <strong>Comments</strong>

### **Overview** Tab

The **Overview** tab enables you to manage general settings, triggers, and email alerts. You can also see details from the last run of your transformation on this page.

#### **Settings** Category

The **Settings** category enables you to specify automatic job retry, logfile verbosity, and command timeout, as follows.

1. Select **... > Edit** at the end of the transformation row. This action opens the **Edit Settings** dialog box.

   <img src="https://mintcdn.com/cdata-sync/ANiMw09F6wBBj5Tp/images/Transformations-dbt-Overview-Settings.png?fit=max&auto=format&n=ANiMw09F6wBBj5Tp&q=85&s=a5f9a583f65bb3a7755106efac7963df" width="400" data-path="images/Transformations-dbt-Overview-Settings.png" />

2. (Optional) Select **Enabled** if you want {siteNameShort} to retry the transformation if one or more tasks fail. Only failed tasks are re-executed.

3. Select a level for **Logfile Verbosity**.

4. Enter a timeout value in the **Command Timeout** field.

5. Click **Save**.

#### **Trigger** Category

The **Triggers** category enables you to set a specific schedule by which to run your transformation, as follows.

1. Click **Configure** in the **Triggers** category. This action opens the **Trigger** dialog box.

2. Specify the type of trigger that you want. You can select from the following types:
   * <strong>None:</strong> Turns off scheduling options, allowing you to start a transformation manually.

   * <strong>Scheduled:</strong> Enables you to set a specific schedule by which to run your transformation. This setting also activates the **Intervals** and **Minutes Past The Hour** fields. You can choose more granular settings for the **Intervals** setting:

     * <strong>Minute:</strong> Activates the **Every** option where you can specify increments in minutes (every 10, 15, 20, or 30 minutes) in which to run your transformation.
     * <strong>Hourly:</strong> Activates a **Minutes Past The Hour** field so you can set your transformation to run hourly at specific numbers of minutes past the hour (0-59). For example, you might want to run the transformation every hour at 15 minutes past the hour.
     * <strong>Daily:</strong> Activates a **Time of Day** field so you can set the specific time each day that you want your transformation to run.
     * <strong>Weekly:</strong> Activates two fields (**Days** and **Time of Day**) where you can specify a specific day of the week and time to run your transformation.
     * <strong>Monthly:</strong> Activates **Day** and **Time** fields. However, for **Day**, you choose a specific day of each month (values of 1 through 31 or Last Day) along with the time.
     * <strong>Cron:</strong> Activates a **Cron Statement** field where you can write a cron job to initiate a transformation run.

     The **Scheduled** type is useful when you run multiple jobs. You cannot use the **After Job** option for multiple jobs because you cannot specify after which job to run the transformation.

   * <strong>After Job:</strong> Activates a **Job** drop-down list that presents a list of all available jobs. You can choose the specific job after which you want the transformation to run.

     You can also specify various tasks to run on your data by clicking **Select Specific Tasks** under the **Job** field. Simply select the task or tasks that you want to run.

3. Click **Save**.

<Note>To disable schedule triggers, slide the **Trigger** toggle switch to the left (the switch turns gray when it is disabled). The toggle switch appears to the right of **Trigger** in the category box.</Note>

#### **Labels** Category

This category enables you to add a label:

1. Click **Add Labels** in the **Labels** category. This action opens the ***dbtTransformationName* - Add Label** dialog box. You can search and select a label currently available in the list or you can add a new label.

2. To add a new label, enter the label in the **Search or Add Label** text box and click **Add**.

   <img src="https://mintcdn.com/cdata-sync/I6gJ4qR9mocXqHNN/images/DBT-Transformations_labels-category_1.png?fit=max&auto=format&n=I6gJ4qR9mocXqHNN&q=85&s=67614bff4dbc2f98a349a061bb585951" width="300" data-path="images/DBT-Transformations_labels-category_1.png" />

   Then, the new label appears in the list of available labels, as shown below.

   <img src="https://mintcdn.com/cdata-sync/I6gJ4qR9mocXqHNN/images/DBT-Transformations_labels-category_2.png?fit=max&auto=format&n=I6gJ4qR9mocXqHNN&q=85&s=4d3782221c902be2acd27660749d6604" width="300" data-path="images/DBT-Transformations_labels-category_2.png" />

3. Click **Save**.

After you complete these steps, the new label appears in the **Labels** category box on the settings page.

#### **Notifications** Category

The **Notifications** category enables you to set email alerts for transformations, as follows.

1. Select **... > Edit** at the end of the transformation row for which you want to set alerts.
2. Click **Configure** in the **Notifications** category.
3. Set the following options in the **Notifications** dialog box:
   * <strong>To:</strong> The email to which to send alerts.
   * <strong>Subject:</strong> The subject for alert emails.
   * <strong>Errors Only:</strong> An option that you can select to specify that alert emails should be sent only when the job encounters an error. If you do not select this option, {siteNameShort} sends alerts after each job run.
4. Click **Save**.

<Note>To disable alerts, slide the **Notifications** toggle switch to the left (the switch turns gray when it is disabled). The toggle switch appears to the right of **Notifications** in the category box.</Note>

### **Transformations History** Tab

This tab provides details about your transformation job run. Details include **RUN DATE**, **STATUS**, **RECORDS AFFECTED**, **RUN TIME**, and **INSTANCE ID**. You can also filter your report to show transformations by status (**All Status**, **Success**, **Failed**, or **Error**). In addition, you can export the history by clicking **Export History**.

### Comments Tab

This tab records and tracks notes that are related to your transformation. Each comment appears with a timestamp. You can change the sort order based on timestamp by using the drop-down menu in the top-right corner, with options for **Newest First** (default) or **Oldest First**.
