Skip to main content

​
Supported File Formats

When writes data to , you can choose the file format for the exported data. The following file formats are supported for the source:
  • (Default) CSV — Plain text comma-separated values.
  • Avro — A row-based binary format that supports schema evolution.
  • Parquet — A columnar storage format that is optimized for analytics.

​
Add the Azure Data Lake Storage Connector

​
Authenticate to Azure Data Lake Storage

After you add the connector, you need to set the required properties.
  • File Format: Select the file format that you want to use: Delta Parquet, CSV (default), Avro, and Parquet.
    The Delta Parquet and Apache Iceberg file formats are not supported for source connectors. If you select either of these formats, generates an error.
  • Azure Storage Account: Enter the name of your Azure storage account.
  • URI: Enter the path of the file system and folder that contains your files (for example, abfss://MyFileSystem/FolderName).
supports authenticating to in several ways. Select your authentication method below to proceed to the relevant section that contains the authentication details.

​
Azure Active Directory

To connect with an Azure Active Directory (AD) user account, specify the following properties:
  • Auth Scheme: Select AzureAD.
  • Use Lake Formation: Select True if you want the AWS Lake Formation service to retrieve temporary credentials. These temporary credentials enforce access policies against the user based on the configured IAM role. You can use this service when you authenticate through AzureAD, Okta, ADFS, and PingFederate, while providing a Security Assertion Markup Language (SAML) assertion. The default setting for Use Lake Formation is False.

​
Azure Managed Service Identity

​
Azure Service Principal

​
Azure Service Principal Certificate

​
Azure Access Key

​
Azure Shared Access Signature

​
Complete Your Connection

To complete your connection:
  1. Specify the following properties: For the CSV file format:
    • FMT: Enter the format that you want to use to parse all text files. The default format is CsvDelimited.
    • Aggregate Files: Specify whether you want to aggregate all the files that are located in the URI directory and that have the same schema into a single table named AggregatedFiles. The default option is False.
    • Include Column Headers: Specify whether you want to obtain column headers from the first lines of the specified files. The default option is True.
    For the Avro and Parquet file formats:
    • Data Model: Select the data model that you want to use to parse documents for your format and to generate the database metadata. The default data model is Document.
    • Aggregate Files: Specify whether you want to aggregate all the files that are located in the URI directory and that have the same schema into a single table named AggregatedFiles. The default option is False.
  2. Define advanced connection settings on the Advanced tab. (In most cases, though, you should not need these settings.)
  3. If you authenticate with AzureAD, click Connect to Azure Data Lake Storage to connect to your Azure Data Lake Storage account.
  4. Click Create & Test to create your connection.