Mass-market software able to do this is still nascent. Learning what that software is, what it will be able to do, is still being developed and understood. But by using the human learning analogy, we can start recognizing how intuitive pattern-based insight can come about using information technology. To start that, let's see where patterns come from in human perception.The Basics
Even as early as the kindergarten level, children are taught to recognize patterns. Rhyme schemes in poetry, stripes of color on a piece of cloth, those are patterns a 5-year old can identify with the following steps:
The child takes contextual information and extracts relevant sections of data, or features. For example, hearing or reading the following poems, there is a vast amount of processing going on in the brain to convert dots on a page or sounds in the air into words.
Roses are red,
Violets are blue,
Sugar is sweet;
And so are you.
(from Solitude, by Alexander Pope)
Happy the man whose wish and care
A few paternal acres bound,
Content to breathe his native air
In his own ground.
The child has been introduced to the concepts before and remembers them. The pattern types have names or identifiers, which means they are special enough to identify with their own terminology. Once the words have been extracted by eyes and/or ears, training, instruction, and having seen the concepts before makes reducing the rhyme scheme to ABAB a “simple task”.The poems are drastically different (different words and themes), but the ABAB pattern is fundamental to poetry. If you are searching for the ABAB pattern in a poetry search engine, both of these poems would come back.
Patterns are fundamental components of visualization. A Google search for pattern images (http://www.google.com/images?q=patterns ) yields images like these:
And here are some non-patterns (image search for static or white noise):
People instinctively know the pictures that are patterns ARE patterns - human evolution has suited vision systems to identifying patterns and detecting visual cues based on changes in imagery: graphs with trends, detecting outlines, detecting subtle shifts in color. Going back to the kindergarten education example, the picture below is a simple pattern for children to understand (ABC ABD ABC ABD) once they’ve been taught to use it, but it is still a difficult task for someone to get a computer to take the image below and make sense of it.
In some sense, many advanced patterns, especially those encoded in graphs and charts, are built on the fundamentals described in the previous steps. Seasonal sales trends, stock market/financial market data, radiology images, with this kind of data there are shapes or trends that the human in the loop can understand. Any software that can capitalize on the innate human ability to understand a pattern within data once it’s been recognized, and still use that exact same pattern to make decisions in an automated fashion as computers are very good at doing is key to making pattern based analytics a “must-have” tool in the analyst’s toolbox, regardless of what their data might actually contain.