« You know the great thing about databases? You ask them a question – or, to be more technical, you generate a query – and the answer comes back. That’s highly simplified, of course, because for your query to be successful, it has to at least match the data therein.

But what if you have so much data that you’re not even sure what question to ask? Typically, you know what you don’t know, and that’s why you ask a question. But what if you don’t know what you don’t know? It takes real insight to admit that you have no insight; I’ve worked with people in my career who didn’t have this kind of insight, and they’re really no fun to interact with.

That’s the challenge that Crossmark faces everyday – not dim-bulb people but rather so much data that they’re not always sure what they’re looking at. Crossmark labels itself as a sales and marketing services company for the consumer goods industry, but that really doesn’t begin to tell the story of its big data challenge.

Crossmark has 30,000 employees in North America, Australia, and New Zealand that visit retail locations on a weekly basis on behalf of its clients, which could be the retailers themselves or the manufacturers. These employees collect lots and lots of data: point-of-sale data; where are the products in the store; how much shelf space do they have; how much are the competitive products selling for; what does the display look like. Part of the work falls into the category of competitive intelligence; part of it falls into the category of compliance – if the manufacturer paid for a special endcap display, is the retailer complying?

That’s just the in-store data. There’s also the external PESTLE data, which stands for political, economic, social, technological, legal, and environmental data. For example, in the environmental category, it’s demand for snow shovels in Massachusetts and demand for suntan lotion in California. Lest you think this is a new term, it was actually originally coined in 1967. »
Read article
By Howard Baldwin
Source: forbes.com