I was asked recently to provide my perspective on why and how to determine how to consolidate storage. Obviously I’m no sys admin but I thought that managing storage shouldn’t be much different to other types of housekeeping with applications in IT or for real life for that matter. So here goes:
10 Tips for Storage Consolidation
1. Clean House Before You Move – Just like in real life, if you can throw out unwanted, unused files and get your business users to consider applications that can be “retired” before consolidation, will make the process a lot more efficient and less painful. Naturally, just like in real life letting go of what might turn out to be important later is a challenge, not to mention finding the original owner of the data who can okay its demise, but “don’t ask don’t get”.
2. Be A Hero Either Way – By inventorying and sending out lists of files and objects to their apparent owners and asking if they really still need them, you might get one of the following reactions: “I thought we lost that data, thank goodness we were looking for that” or “Good grief, we are still holding on to that past the retention period? It’s a major liability if we get audited. Delete it asap!”
3. Set Best By Dates – Even if you find it hard to accomplish Tip #1 above because you can’t quite figure out what old data can be removed. You can stem the tide for all new files and applications that are introduced using tools that allow you to set mutually agreed upon retention and expiry polices. This at the very least will force the conversation sometime down the road.
4. Have A Compression Obsession – With unstructured file and block level compression, plus document and object de-dupe available as commodity offerings, you should be able to save significant space and stretch your $ per TB. But do not stop there, a new wave of technologies will allow you to de-duplicate structured data held in databases such as Oracle too.
5. Go Small Or Go Home – By reducing the physical footprint without sacrificing the accessibility of your data, your backups will run faster, migrations become easier and in some cases with structured data value de-duplication your end-users may even get faster query access!
6. Expect Your Users To Be Wrong – Once you’ve figured out how much space you need in your new consolidated storage environment, you’ll be eagerly awaiting feedback from your users as to their anticipated growth rates and requirements. Since it’s unlikely that they’ll be able to get this right, especially with the onslaught of social media data, video and a deluge of business transactions and increasing retention periods, it’s best to go with easily extensible storage offerings or try Tip #8.
7. Your Oldie Can Still Be Goldie – Consolidating doesn’t necessarily have to mean replacing older storage devices. Data that is less frequently accessed with compression capabilities that use less disk reads can still find a home on spinning rust.
8. Let Your Judgment Be Clouded – If you are consolidating anyway, consider the cloud for pay-as-you go storage for some of your data. Amazon continues to set the bar for compelling prices per GB/month for commodity S3 storage. Apart from using only as much as you need, you can also only pay for processing when your users really need to have it (which is likely to be less often than they claim). You can also use “cloud bursting” by making use of cloud storage in a pinch as your data center runs out if Tip 6 wasn’t enough.
9. Look Beyond All That Platter – While disk performance, efficiency and low failure rates are key. Today’s storage vendors are differentiating their offerings by providing major administration, management and compliant retention features that could significantly reduce the resources you need to run your data center.
10. Don’t Look Back – Your storage consolidation project is a key enabler to allow your organization to have a modernized and more efficient data center for anticipated growth ahead. While there will be hiccups along the way, your business users may thank you, your CFO may or may not thank you, but your CIO certainly will.