View schema and override data types

Bobsled allows you to observe the inferred schema and optionally overrides the data type of the source data to a Cloud Data Warehouse destination, mediated by Bobsled data types. It’s useful to get acquainted with how Bobsled maps the source data type to the destination to ensure your target data type meets your requirements.

This article will describe the steps to:

• Override a data type within a data transfer and,

• Observe the schema in a Share after setting up a data transfer

Bobsled introduces two columns to your schema, which are required to ensure your data is successfully loaded to the preferred Cloud Data Warehouse destination.

• _META_LOADED_AT: TIMESTAMP_TZ (UTC)

• _META_SOURCE_FILE_NAME: STRING

NOTE:
Bobsled does not allow you to override these data types, and will always inform you that they are Bobsled generated and required. For more information, contact your account team.

Prerequisites

• A Share must be created.

• To successfully create a data transfer in a share, you must have at least one Data source preconfigured in Bobsled.

• If you’re sharing data to a Bobsled-managed destination, all you need do is to pick where you want your data to be shared. Sharing to an externally managed destination may need more details before you are ready to start creating a transfer. Check our supported Destinations for more information.

NOTE:
What you can see and do will differ based on your role and permissions.

Override data type guide

1. Within a share (with a source and cloud data warehouse destination), click the create transfer button

2. Choose your source entities to share (paths if your source is a file storage, or entities if the source is another cloud data warehouse), click continue

3. In the table transfer configuration step, locate and select the schema icon

NOTE:
While Bobsled is inferring the schema of your source path for File Storage Sources, the Schema modal will relay the ‘No columns available’.

TIP:
For File Storage Sources, Bobsled allow’s you to re-infer the schema. Locate and select the Re-infer schema button in this modal to do so.

4. In the schema modal, locate the data type that you wish to override and click the value drop-down

TIP:
Bobsled provides feedback on which data type was overridden. Hover over the swap icon (two-way arrow) to see the original inferred value.

5. Choose the desired data type, and select save.

6. Continue with the data transfer setup, and click start transfer.

Override a data type on an existing data transfer

In the event you need to override a data type after a data transfer has already been performed, follow the steps required for editing a data transfer and perform the desired changes. The Bobsled application will advise you that editing may result in breaking changes.

TIP:
In this scenario, if you wish to override the data type on an already existing data transfer, it is advisable to trigger a back-fill for your source data to avoid breaking changes. This may vary pending on the loading patterns in your data transfer. Reach out to your account team to assist you.

View schema

Bobsled allows you to observe the data transfer in two ways:

• In the review screen during the data transfer setup, locate and select the schema icon.

• On the share page, in the data transfer box, select the more (ellipsis) icon, click the view option, and locate and select the schema icon you wish to observe