Cognitive technology: Unlocking new insights in dark data

Today’s businesses are faced with unprecedented volumes and variety of data—but that will come as no surprise to anyone who works in either business or data. But if you don’t work in enterprise content management, you might be surprised to learn that as much as 80 percent of all data is only minimally usable by the computers and systems that store it. Such data is called dark data—data into which systems can provide little insight.

Cognitive systems, however, can help unlock the untapped potential of dark data for use by a full range of businesses and industries. But what makes a cognitive system different from traditional data processing systems? Of particular importance, cognitive systems are designed to reason about data while learning by experience, much as their human creators do. These defining capabilities—reasoning and learning—can bring intelligence to digital business.

Where is your dark data?

Any discussion of dark data must start by looking at how such data is captured and stored. Traditional document capture systems excel at capturing and extracting information from known and predictable document types—forms, for example—but fall woefully short when asked to capture complex transactions such as loan applications, cross-border shipments, insurance enrollments and claims: indeed, any transaction during which information is communicated in complex and highly variable documents.

What’s more, traditional systems usually require human intervention to review and classify unknown or divergent document formats, but such intervention is time-consuming and incurs undue financial expense. Just imagine a scenario in which a bank must look at each document of a particular type to verify signatures. Such a process would be painful in the extreme done manually, yet in the absence of cognitive computing, only manual processing could account for a variety of layouts and formats while doing the work. A human can easily glance at a document to confirm its signature, but a computer that lacked cognitive computing capabilities might be next to worthless at the task.

Capture your data cognitively

Manual intervention cannot become the order of the day. Rather, we must adopt computer systems that are flexible enough to quickly and accurately address customers and regulators with equal facility. Instead of relying on manual intervention, we must use cognitive document capture systems, which combine advanced imaging, natural language processing and machine learning technologies—such as those used in IBM Watson—to exceed the limits of traditional capture, offering the following capabilities:

Automatic classification and extraction of information—including text, photos and handwriting—from complex and variable documents in real time

Analysis of content to understand information in context and then choose an appropriate response

Application of rules to pass documents to appropriate persons who can initiate cases and drive workflow for additional actions

Learning through training, if required, when a user classifies and defines a sample set of documents

Creation of a knowledge base over time to shorten reaction times and boost classification accuracy, allowing ever faster and more precise execution of business processes

Find out how you can harness the power of IBM Datacap Insight Edition, IBM’s cognitive capture solution.

Follow @IBMBigData