Recently we have started the service trial. VM is unused past 11PM and should be shutdown to save dollars. so, I set out to figure out how to do it.
On net search – I hit across azure automation services which let you schedule your powershell script for process automation. It was not straight forward to get going with it without spending day or two to understand it and its billing. I was simply looking for a quick solution without any learning curve. I know shutdown /s /t 0 shuts down a windows computer. Can I schedule it? I found that schedule task mmc is available in windows azure vm. Hence, I created a basic task and scheduled it for 1 am to shutdown.
In morning, I did find the VM in stopped state. but I have typically seen the VM in stopped(deallocated) state when I shutdown using azure management portal. I looked around and found that stopped VM is still billed. stopped(deallocated) VM is not billed. I have to now figure out how can I de-allocate it in midnight? azure powershell and cross platform CLI both provide commands to shutdown VM (including deallocation). Download of azure powershell is ~120mb whereas azure-cli (cross platform) is available through npm (nodejs). I have nodejs on my devbox and it is our server platform too. npm install azure-cli -g command installed the command line in less than a minute. command line is also very intuitive and simple to follow.
azure account download
-- launch the browser, login, download & save the azure account publisher settings.
azure account import <path to publisher settings file>
-- imports setting for the subscription.
azure vm shutdown -v <vm name>
-- shutdown vm including deallocation.
Now, I need to figure out the solution to trigger this command at midnight. My laptop is also sleeping at that time including me. Windows task scheduler also have option to wake computer & ensure network connection before launching a task. It requires changing advanced power settings on the computer – to enable wake up alerts. I tried to schedule it (using windows task scheduler) in the VM itself which did not require all of this and it did work to de-allocate itself!
ps/ I have set few additional settings in the task. 1) run with highest privileges 2) run whether user is logged on or not. It asks the admin password to schedule the task. Without the password – it can’t have network access which is required for the command line to issue the command.
It is very likely that you come across the problem – How do I ensure safety of my azure service from data-loss? Our service development is in progress. Before we onboard any customer for trial, we need to ensure that his data can’t be lost. In that context, I started with the problem definition as how do I backup & recover azure table storage? The service has maximum data in the table storage.
Very soon, I realized that data reliability of azure table storage is very high. It makes multiple local copies and if geo-rep is enabled, additional copies in another data center. it must be order of 99.9%. I tried to find documented link in msdn for this. Documented SLA of 99.9% is only for availability. Hence, do we need to do nothing for data loss? Actually, we need to handle the data loss protection in following cases:
- No data loss during service administration and maintenance. For example: do not execute delete table api in any administration task. This can only be done through very tight control and strict guidelines for the production environment.
- Data loss caused by end user in the application. Take for example: email application. user can accidentally delete a mail or folder.
After striking out (1) by following strict documented guidelines – problem can be redefined as – Backup and recovery of the application data of the service. This is typically very specific for the application. For example: email application has very rich experience built into the client application . End user can help himself/herself for any loss of data by recovery the item from ‘deleted items’ folder. He needs to approach the service administrator only if whole mailbox is lost, deleted items folder is deleted, etc. rarely. In our service case, accidental deletions will be rare and can be handled by user raising a request to the service administrator. Hence, service client (app) will not have rich recovery experience built in like it is required in an email client.
How do we backup application data in table storage? It will likely very specific for each service. In our case, current thinking is – to backup each user data separately – to a set of excel (xls) files – providing daily backup. Why excel?
- Excel not only provides excellent data access and manipulation, it is de facto standard.
- If user wishes to do manual recovery – it can be shared as is with user.
- The service needs to be seeded with user existing data. For that also, recovery flow can be used.
Regarding backing each user data separately, it partitions the problem of backing up table storage that will grow in size with number of service users.
Backup format is specific for the service. It is not likely relevant for other services. I need to think about building incremental daily backup yet. Design needs to be implemented. It may likely go through changes as I implement – as in regular cycle of software development.
How do you handle backup and recovery of your application data in service?