Warehoused reports temporarily unavailable

Incident Report for aXcelerate

Resolved

We have added a series of database optimisations to lower the risk of this issue occurring again. We will continue to monitor the performance of the overnight job. We are also planning to upgrade our reporting database server to a newer generation server in the near future.
Posted Feb 20, 2020 - 17:07 AEST

Update

The Warehoused reports are now enabled. Unfortunately, unit enrolment warehoused reports are still not available. We are now investigating further to find the cause of this issue.
Posted Feb 20, 2020 - 15:48 AEST

Update

The automated job that produces the warehoused reports had an issue this morning where the last step of the job simply would not complete. This final step creates data for any warehoused report that relates to Unit Enrolments. We have now rolled back this last step, rebooted the database server and we are attempting to run this step again. If it works correctly, it should take approximately 90 minutes.
If this issue reoccurs, we will abort and turn on the warehoused reports for all reports that do not require this data.
Posted Feb 20, 2020 - 13:21 AEST

Update

We are continuing to investigate this issue and will provide you with updates as they become available.
Posted Feb 20, 2020 - 10:56 AEST

Identified

The automated job that allows our warehoused reporting system to operate has hit a snag overnight whilst aggregating the data. This will take time to run and we expect the warehoused reports will not be available until later today.
Posted Feb 20, 2020 - 08:46 AEST
This incident affected: Reporting System.