101010 The Answer to Life, the Universe and Big Data

On October 10, 2010 a day represented as 10/10/10 raises awareness and brings back memories of many iconic events and topics related to the magical number 10. In the spirit of this day, here are 10:

10. David Letterman’s top 10

9. Bo Derek, Dudley Moore and Julie Andrews in the Movie “10”

8. Big Ten conference in college football (which actually has 11 members expanding to 12)

7. World soccer great Pele from Brazil wore the number 10

6. The Ten Commandments, the bestselling book and the movie with Charlton Heston

5. NFL Dime defense configuration

4. 10 Downing Street, the home of the UK Prime Minister

3. 10 things you didn’t know about me

2. 101010 is the binary representation of 42 – the answer to “life the universe and everything”-  Douglas Adams’ Hitchhiker’s guide to the galaxy

1. and … also the Ten signs you need a Big Data retention solution at 10x less cost

When it comes down to it data ends up being stored as 1’s and 0’s representing a binary (either 1 or the other) “bit”. All forms of data stored therefore display patterns when committed to physical storage.

At a simple level, Zip type compression technologies squeeze out any white space in between “fragmented” blocks of data, and de-duplication at the byte-level look for similar sequences of storage areas or files that can be consolidated.

Using such technologies to manage and handle unstructured data such as documents, images and videos provides tremendous benefit since these often very large binary objects (BLOBs) account for a significant amount of disk storage and each object is self contained.

Structured data, such as records within a database or log files, can also benefit from such “brute force” compression. However, there is more to be gained beyond just squeezing their physical bytes. Examining individual data values that make up the rows and columns of structure data, we can detect a high degree of duplication at the individual value and pattern level which can result in the highest possible level of compression for structured big data retention.

104 = Message received

Leave a Reply