Munich, Germany
+49 (0) 160 - 9222 4556

Optimization with Dark Data and Big Data

Dark Data & Big Data

Optimization with Dark Data and Big Data

According to our “Duden” the definition for big data is the huge amount of data. This describes very well the challenge for companies on how to deal with big data.

Dark data is the term Wikipedia uses to describe data that is collected and stored by information systems but not used.

Dark data contains a lot of customer information that is extremely important for the company’s planning. This is exactly the reason why I am taking a closer look at this volume of data.

As an expert in the field of marketing analytics and digitalisation, my focus in this article is on customer data that is used for sales and marketing. Big Data and Dark Data cover a much broader spectrum in the company.

What does Dark Data mean?

Veritas Value of Data Studie 2019

The statistic from Veritas shows, that dark data comprises more than 50% of the data volume in Germany. On the one hand, this means that valuable customer data remains unused. On the other hand, it means that expensive data storage space is wasted. Both points put pressure on the cost and profit balance.

Another part of the data structure is ROT data, i.e. redundant, obsolete and trivial data, that is not needed for later use. These can and should be permanently deleted.

Only a small part of the data volume is so-called clean data, which is stored in a structured way and can be used for planning.


Most customer data has important potential for planning, including dark data. Unstructured customer data can be, for example

  • Customer correspondence via email
  • Telephone calls (customer service, technical, accounting …)
  • Server log files that provide information about the behaviour of website visitors
  • Customer behaviour in the web shop
  • From where (location) and how (mobile, desktop …) the website is accessed
  • Bounce rate
  • Top sites (landing pages …)
  • Customer behaviour in social media
  • Payment preferences and morale
  • Returns
  •  etc.

The author Daniel Keys Moran already said: “You can have data without information, but you cannot have information without data.”

Accordingly, to many statistics means dark data:

  • Less than half of the structured data stored in a company is used for business decisions.
  • For unstructured data, the percentage of information used is even less than 1%.
  • More than 70% of employees in companies can access data that they do not need for their tasks.

The market research company IDC has calculated that by 2025 around 80% of the data stored worldwide will be unstructured and therefore hardly usable.

This shows the potential of the data, but also the cost trap that lies behind it. That is why it is important to structure the data and make it usable. Digital tools and software are needed to structure large amounts of data. “Only” the valuable customer data already have a huge amount of data, Big Data. For the company this means, implementing software that can process Big Data, structure it and make it accessible.

What does Big Data mean?

The chart from Deloitte University Press shows an overview of the amount of data worldwide – 44 ZB (Zettabyte). The fast and huge data growth is clearly visible. 

The trend of enormous data growth continues. A survey by Statista on the topic of big data shows that the volume of data will grow to 175 zettabytes by 2025. That is why companies invest large sums in software that offers big data solutions.


Deloitte University Press
Bild Big data Statista Study 2020

What does this mean for your Customer Data?


Let me use the dark data mentioned above to explain. Salesforce, a leader for CRM software, is able to process huge amounts of customer data, generate reports and developments on the topic of customer preferences and needs.

  • Customer correspondence by e-mail
  • Telephone calls (customer service, technology, accounting …)
  • Customer visits
  • Trade fair contacts
  •  etc. 

Web analytics programmes such as Google Analytics, Matomo and Adobe track visitor behaviour on the website and online shop.

  • Server log files that provide information on the behaviour of website visitors
  • Customer behaviour in the web shop
  • From where (location) and how (mobile, desktop …) the website is accessed
  • Bounce rate
  • Top sites (landing pages …)

Content Management Software show the activities on the social media portals.

  • Customer behaviour in social media
  • Key figures in online marketing such as CTA, bounce rate, registration figures ….

Tableau, SAP and Oracle analyse data from ERP, accounting, production and shipping.

  • Payment preferences and morale
  • Delivery preferences
  • Product preferences
  • Returns
  •  etc.

Clever implementation of the respective software means for the company that

  • the data is only stored in one software (database) and
  • the data is only made available to the relevant employees via profiles and
  • the data can be used for planning, cost control and optimisation at any time.

Conversely, this means that

  • multiple storage is excluded and
  • the data are, in accordance with the Data Protection Ordinance (DSGVO)
    • is secure and
    • is only made accessible to the relevant employees and
    • is stored and used within the company.

What is the benefit for you?

For my understanding, unused data means a high cost without benefit and at the same time the loss of customer satisfaction. With the use of customer data, customer preferences and needs are made visible. Software solutions enable the analysis of even large volumes of data and ultimately the possibility to increase customer loyalty and customer satisfaction. My experience shows that turnover and profit increase disproportionately to customer satisfaction.

A good database offers to filter out relevant information and to improve processes in a targeted manner – and ultimately increase turnover and profit. Big data solutions have become indispensable for reliable planning. They enable fact-based analyses of customer data from all functional areas of the company and

  • a cross-functional view of customer 360°
  • a derivation, transparency and insights of customer preferences and needs
  • a detailed target group analysis, USP, SWOT
  • an analysis of product portfolio and benefits according to customer preferences
  • a reliable analysis of turnover development and price structure
  • a fact-based analysis of the market environment, of the competition, of trends, of the overall economic development as well as the development of the sector
  • and much more.

Contact me, I would be happy to come to your company and analyse the possibilities of your data treasure (Dark Data and Big Data), make them transparent and show you the potential that can result from them.

Mobile: +49 (0) 160 – 9222 4556


Leave a Reply

Your email address will not be published. Required fields are marked *