Backfills in Bobsled allow you to reprocess historical data in your Sledhouse Table and ensure that any downstream Data Products reflect the latest schema or logic changes.
You can initiate a backfill through the Bobsled Application or API ↗, with fine-grained control over whether to include downstream Data Products—and which ones.
FEATURE IN PUBLIC PREVIEW:
• While in Public Preview, expect some limitations. Your feedback is crucial as we build the product.
• We look forward to your feedback and to helping you integrate Sledhouse into your data architecture. Feel free to reach out if you have any questions about setup, configuration, or advanced use cases.
Overview
Backfill gives your team full operational control over how and when data is reprocessed:
Predictable Resource Management: Control when compute is used, preventing unexpected costs during business hours.
Operational Control: Schedule backfills during maintenance windows or off-peak periods.
Selective Backfill: Target only specific Sledhouse Tables or Data Products without affecting the entire pipeline.
When to Use Backfill
Schema change in the source table (e.g., a new column).
Change to a replication pattern in the Sledhouse Table.
A data quality incident requiring downstream reprocessing.
Missed syncs or stale Data Products that need to be refreshed.
Switching to or from Sledhouse Table local copy mode (e.g., Snowflake).
NOTE:
Bobsled will notify you in the application when a backfill is recommended—typically after configuration changes that could impact how data is loaded or queried. However, backfills are optional: you can choose not to run one if you're confident it's not needed.
Common use-cases
Scheduled Maintenance | Trigger backfills during planned windows to avoid disrupting production jobs. |
Selective Recovery | After a quality issue or source correction, backfill only the impacted Data Products. |
Cost Optimization | Batch backfills during off-peak compute pricing to reduce spend. |
Manually backfilling in the Bobsled Application
Backfill a Sledhouse Table
Go to the Settings tab of the Sledhouse Table you wish to backfill.
Click “Backfill Sledhouse Table”.
Optionally choose to also backfill associated Data Products.
Confirm and monitor progress through logs.
NOTE:
• Local copies are automatically backfilled if the Sledhouse Table is backfilled.• Backfill of local copies starts only after the parent Sledhouse Table has finished.
• Local copy backfills apply across all selected regions (not per-region granularity).
• Backfilling creates new log events and data product versions.
• Data Products will reflect the backfill only after their refresh completes.
Backfill a Data Product
Navigate to a Data Product you wish to backfill.
Select Backfill Data Product from the settings tab and optionally backfill any fulfilled Data Products.
A new version will be generated and fulfilled.