BRIGHT’s Three-Times-a-Day Data Refresh

BRIGHT’s data is updated three times per day in an 8-hour cycle. This involves:

1. Extract & Rebuild:

• • An ETL (Extract-Transform-Load) script runs to fetch data from the FL3XX backend tables and rebuild the full dataset.
  • This process starts every 8 hours, then takes around 3 hours to complete.
  • Each cycle begins with an ETL process that pulls data from the FL3XX backend, followed by a refresh in Amazon QuickSight. Importantly, different datasets are extracted at different moments within the cycle—some near the start, others later. Typically, all data is captured within 1 hour and 45 minutes from the beginning of the process. As a result, recent changes may appear in BRIGHT dashboards even if they occur after the cycle has begun, depending on the exact timing of each dataset's extraction.

2. Datasets (Quicksight) Refresh:

• • After the dataset refresh, BRIGHT (powered by Amazon Quicksight) refreshes its datasets, dashboards, and reports.
  • This final step ensures that you have access to the newly rebuilt data in your BRIGHT dashboards.

Below is a breakdown of how it looks for each region:

EU Region (eu-central-1.quicksight)

• ETL Start Times: 03:00, 11:00, 19:00 UTC
• QuickSight Refresh: Fixed schedule at 07:00, 15:00, and 23:00 UTC
• Data Lake S3 Buckets Refresh: ~2.5 hours after start, i.e., at 07:30, 15:30, and 23:30 UTC
• ETL Completion: ~3 hours later

Example: Data entered in the EU Region at 02:59 UTC will appear in BRIGHT dashboards by 07:01 UTC that same day.

 

US Region (us-east-1.quicksight)

• ETL Start Times: 06:00, 14:00, 22:00 UTC
• QuickSight Refresh: Fixed schedule at 08:00, 16:00, and 00:00 UTC (next day)
• Data Lake S3 Buckets Refresh: ~2.5 hours after start, i.e., at 08:30, 16:30, and 00:30 UTC (next day)
• ETL Completion: ~3 hours later
   

Summer / Daylight Saving Time

Winter / Standard Time