In a world where the explosion of data is already becoming a huge challenge, the big questions are how do you get it under control and how can you derive benefit from it? John Culkin from Crown Records Management, offers advice.

Dark data. It’s a catchy title which intentionally makes you think of dark matter, the invisible force in space that has never been directly observed, but is something astronomers tell us has a significant impact on everything around it.

Gartner – a leading information technology research and advisory company - defined dark data as “information assets that organisations collect, process and store during regular business activities, but generally fail to use for other purposes” - and that hints not only at waste but also at wasted opportunities.

There’s no doubt that holding unnecessary data ‘just in case’ is going to prove expensive for businesses in future as technology continues to progress and the Internet of Things - a proposed development of the Internet in which everyday objects have network connectivity, allowing them to send and receive data - takes hold.

In the future the amount of ‘excess’ data being generated will become an increasingly serious issue both in terms of how much it costs to store and the inherent risks it contains.

Perhaps that’s why some people believe dark data is a negative force and its rise is inexorable.

Of course there are costs and risks associated with not understanding and managing it – and yet, by finding and exposing the data and finding all the places it hides, easy gains can be realised, too.

The bottom line is organisations need to know what dark data is and where it is. If you don’t understand it how can you possibly be in control of it, let alone realise value hidden in it?

Perhaps the first place to start is to understand why so many businesses keep dark data in the first place.

Hiding dark data away seems to be not just planned but instinctive, as if it’s down to human nature. People fill their attics and garages with ‘useful’ things that could be needed in the future. The corporate equivalent is when people say ‘keep the data just in case’. How much money around the world is wasted due to those few words?

Perhaps data is being kept ‘in case it is needed in the future’ – assuming it can be found at the time of need, of course. Or because a company believes some unknown future insight can be found using future analytical technology. Perhaps it is stored away out of a mistaken belief that keeping more data means a company is less likely to lose something. Or because a business believes data is like a rare book and its monetary and nostalgic value might increase.

All of these misconceptions lead to a hoarding of dark data which is unnecessary and inherently risky. Having a proper information management system in place to prevent the storage of too much dark data is crucial. Here, then, are some pointers to get started:

10 top tips to prevent dark data accumulating in your business:

1. Don’t accept that because data storage is cheap everything can be kept.

2. Create policies for ‘on-boarding training’ (when staff join the business) and for what happens to data (including data on local drives, laptops, shared drives, removable devices and mobile devices) when staff leave.

3. Understand which regulations require what data to be kept for how long. Don’t keep it longer than necessary unless clear benefit can be derived from it.

4. Don’t believe all data in enough volume is ‘Big Data’ and therefore has value. Much of it will not be useful in big data projects, especially as it’s often unstructured and in various formats.

5. Ensure defensible deletion (a comprehensive policy to reduce both storage costs and the legal risks associated with storing too much data) is used actively.

6. As new systems or applications are adopted, consider what data they accumulate and how it is used. Don’t have debugging log levels run at too high a setting or longer than necessary to find problems.

7. Just because you can get data and store it, don’t do so without a valid reason.

8. Charge cost centres for storage. If it’s free, the data addiction will only grow and there is little incentive to reduce it.

9. Have an email management system that is centralised and does not use local ‘.pst’ files, that automatically delete messages as per the organisation’s policy.

Remember newer forms of data such as voice, video, mobile text or social media data. Much of this data will be in the cloud - often put there by creative departments without contacting IT first, and is therefore hidden.

Having taken steps to get dark data storage under control, the next step for businesses is to start to think about data in a completely different way and to understand what is already stored and what value it contains.

Here, then, are some tips on how to shine a light in those dark corners and to make data work for your business:

Five ways to shine a light on dark data and get value from it

• Use File Level Analysis software, to analyse the content of data rather than just its creation and expiry date. It can help a business understand information content, type, size and location.

• Map data sources generated by internal systems or received from external sources – this is about documenting ‘where’ and ‘why’ the data lives and how it was derived.

• Have an Information Governance programme in place supported by the most senior management levels and practiced every day.

• Plan to delete data from a database at specified times in the future. Even if you retain responsibly it’s easy to run out of time. Adjust your backup and disaster recovery plans so they no longer backup copies of unnecessary data.

• Talk and communicate with people because not everything will be officially documented or known about. Work-arounds and short-term fixes often become routine in the business place, so make people aware of the impact of not being able to manage everyone’s data effectively. It affects everyone.

Only after taking these steps can businesses begin to find, understand and derive benefit from data they previously looked on as nothing more than an expensive waste product. Shining a light on dark data is just the start – but it’s a very good one.

www.crownrms.com

 

John Culkin is Director of Information Management at Crown Records Management

Click here to see the next Editor's Pick