Tracking changes in a SQL server 2005 database

Question

I have been tasked with developing a solution that tracks changes to a database.

For updates I need to capture:

• date of update
• old value
• new value
• field affected
• person doing change
• record id
• table record is in

For deletes:

• date of delete
• person doing delete
• The title/description/id of record deleted. The tables I am tracking changes on all have a title or description field. I'd like to capture this before the record is deleted.
• table record was in

For inserts:

• date of insert
• person doing change
• record id
• table record is in

I've thought of a few ways to do this:

• I am using stored procedures for any updates/deletes/inserts. I would create a generic "tracking" table. It would have enough fields to capture all the data. I would then add another line in each stored proc to the effect of "Insert record into tracking table".
  • downside: all updates/deletes/inserts are all jumbled in the same table
  • lots of NULLed fields
  • how do I track batch updates/deletes/inserts? <---- this might not be an issue. I don't really do any thing like this in the application.
  • how do I capture the user making the update. The database just sees one account.
  • edit a lot of existing code to edit.
• Lastly, I could create a trigger that is called after updates/deletes/inserts. Many of the same downsides as the first solution except: I would have to edit as much code. I am not sure how I would track updates. It doesn't look like there's a way using triggers to see recently updated records.

I am using asp.net, C#, sql server 2005, iis6, windows 2003. I have no budget so sadly I can't buy anything to help me with this.

Thanks for your answers!

Answer 1

First off, in all your tables you should have at least these columns added to the data columns DateCreated, UserCreated, DateModified, UserModified. Possibly you might want to add a "Status" or "LastAction" column so that you don't ever actually delete a row you just set it to a deleted/inserted/updated status. Next you could create a "History table" which is an exact copy of the first table. Then on any updates or deletes have the trigger copy the Deleted table entries into the History table changing the DateModified, UserModified, and Status fields at the same time.

Answer 2

we built our own and just needed the user and pc passed into each add/update stored procedure. then it's just a matter of getting the original record adn populating the variables and comparing them to the passed in variables and logging the data to our table. for deletes we just have a copy of the originating tables + a timestamp field so the record is never really deleted and can be restored anytime we need (obviously the delete routine checks for FK relationships and such).

add/update log table looks like datetime, table_name, column_name, record_id, old_value, new_value, user_id, computer

we never insert nulls so we convert them to empty strings, new entries are marked with '{new entry}' in the old_value column. record_id is made up of as many key columns to uniquely identify that single record ( field1 + '.' + field2 + ... )

Answer 3

I've had a setup in SQL Server where we would use views to access our data, which would handle inserts, updates and deletes with INSTEAD OF triggers.

For example: an INSTEAD OF DELETE trigger on the view would mark the records in the underlying table as deleted, and the view was filtered to not show deleted records.

In all triggers, we updated a modification date and user name. The trouble is that that logs the database user name, which is not the same as the ultimate application user name.

The view needs to be schema bound for this to work.

Answer 4

I hate to side-step the issue and I know you have no budget, but the simplest solution will be to upgrade to SQL Server 2008. It has this feature built in. I thought that should at least be mentioned for anyone else who comes across this question, even if you can't use it yourself.

(Among the deployable editions of SQL 2008, this feature is only available in Enterprise.)

Answer 5

A trigger wouldn't have all the information you need for a bunch of reasons - but no user id is the clincher.

I'd say you're on the right track with a common sp to insert wherever a change is made. If you're standardizing on sp's for your interfaces then you're ahead of the game - it will be hard to sneak in a change that isn't tracked.

Look at this as the equivalent of an audit trail in an accounting application - this is the Journal - a single table with every transaction recorded. They wouldn't implement separate journals for deposits, withdrawals, adjustments, etc. and this is the same principle.

Answer 6

About logging users that changes DB: You can create as many SQL users as you need for your DB and if you use sessions and restricted/registered access to your program/script you can use that information to initiate different DB connection settings (i.e. username), before any operation with DB.

At least that should be doable for PHP-wise scripts, but I might be wrong for asp.net.