It is very likely that you come across the problem – How do I ensure safety of my azure service from data-loss? Our service development is in progress. Before we onboard any customer for trial, we need to ensure that his data can’t be lost. In that context, I started with the problem definition as how do I backup & recover azure table storage? The service has maximum data in the table storage.

Very soon, I realized that data reliability of azure table storage is very high. It makes multiple local copies and if geo-rep is enabled, additional copies in another data center. it must be order of 99.9%. I tried to find documented link in msdn for this. Documented SLA of 99.9% is only for availability. Hence, do we need to do nothing for data loss? Actually, we need to handle the data loss protection in following cases:

1. No data loss during service administration and maintenance. For example: do not execute delete table api in any administration task. This can only be done through very tight control and strict guidelines for the production environment.
2. Data loss caused by end user in the application. Take for example: email application. user can accidentally delete a mail or folder.

After striking out (1) by following strict documented guidelines – problem can be redefined as – Backup and recovery of the application data of the service. This is typically very specific for the application. For example: email application has very rich experience built into the client application . End user can help himself/herself for any loss of data by recovery the item from ‘deleted items’ folder. He needs to approach the service administrator only if whole mailbox is lost, deleted items folder is deleted, etc. rarely. In our service case, accidental deletions will be rare and can be handled by user raising a request to the service administrator. Hence, service client (app) will not have rich recovery experience built in like it is required in an email client.

How do we backup application data in table storage? It will likely very specific for each service. In our case, current thinking is – to backup each user data separately – to a set of excel (xls) files – providing daily backup. Why excel?

1. Excel not only provides excellent data access and manipulation, it is de facto standard.
2. If user wishes to do manual recovery – it can be shared as is with user.
3. The service needs to be seeded with user existing data. For that also, recovery flow can be used.

Regarding backing each user data separately, it partitions the problem of backing up table storage that will grow in size with number of service users.

Backup format is specific for the service. It is not likely relevant for other services. I need to think about building incremental daily backup yet. Design needs to be implemented. It may likely go through changes as I implement – as in regular cycle of software development.

How do you handle backup and recovery of your application data in service?