We run a job tracking service as part of the company software. Reports are generated by SQL Server (Standard) Reporting Services usually daily but often as and when needed before meetings etc.
As the jobs progress their status changes and the details of the reports change. Due to the legacy of the front end application and the database we do not store as many milestones as needed for managers to look back at historical reports.
Managers quite often ask “Can we see where XYZ was last week?” or things like “Let’s have a look how Project1 was performing last June?” but running the reports on the Project1 database backup all show 100% complete as it is now finished.
What is the best way to enable historical reporting at a daily level with SQL Server Standard?
Ideally we’d like to keep 3 months of 5 different reports stored daily if possible?
Best Answer
If you have lost the historical data (i.e. it gets overwritten rather than stored as a transaction history) then you can't reconstruct the historical state - full stop. However, there are a few approaches to dealing with this.
Build an automated process that queries the system through SSRS or some other mechanism and saves the historical reports to an archive somewhere. SSRS exports a web service API that lets you run reports (paramaterised if necessary) and then save the output to a file. This can be operated by anything capable of consuming it.
Build an ETL process that takes a snapshot of the data and compares it with the previous snapshot position. Where changes are detected they can be written out to a historical table. If you have the option of putting triggers on the source database then you can make triggers that write out the audit logging information instead.
If you need something more elaborate then you could build a data mart - do a google search on 'type 2 slowly changing dimensions' for some ideas on how to implement this. You can use a change capture mechanism like that described in (2) to source the data as necessary. Either you can store the transaction history and reconstruct the status in the reports, or you can make a periodic snapshot of the data with its status at that point.
Note that Standard Edition doesn't support changed data capture.