Tuesday, April 22, 2014
Have you ever heard of "Dark Data" ?
I am sure that many of us are hearing this word "Dark Data" for the first time !!!
So what is this term is all about, let's find out..
It sounds like an ominous plot by some evil mastermind intent on world domination. But don’t worry, "dark data" is more benign than the name suggests.
Although is collects in unlit corners and neglected back rooms, dark data is not a serious threat to your business. In fact, it might be more properly termed “dusty data.”
It’s that neglected data that accumulates in log files and archives that nobody knows what to do with. Although it never sees the light of day, no one feels comfortable destroying it because it might prove useful someday.
Will It Be The “Someday” You have Been Waiting For?
With all the recent press about the value of big data, you may be thinking that now is the time to dive into the secrets of the dark data hiding in your organization.
But before you invest in expanded storage capacity or sophisticated data analytic tools, take time to ask the big questions first – the ones that seek out the real value of the data for your business.
The authors of the CIO.com ebook, Big Data Analysis: What Every CIO Should Know, suggest that you start with such blue-sky questions as:
- If only we knew . . . .
- If we could predict . . . .
- If we could measure . . . .
Determine what information you need in order to answer those high-value questions and use that as the standard by which you evaluate all the available data, including the dark data that has never been a part of your regular business operations.
Is Your Dark Data a Business Intelligence Gold Mine?
By itself, some of that dark data may not have much value, but combine it with data you already collect or purchase and you may have a digital gold mine. Those web log files that were once just digital clutter could be the key to unlocking changing patterns in customer behavior that can put you ahead of your competition.
By taking the time to assess the value to your business and investing in the tools you need to shine a light on dark data, you may be able to turn those digital “black holes” into real business intelligence that you can put in the hands of your decision-makers.
Even if you determine that it has negligible value for business intelligence, you have accomplished something of merit. Now that you have established the business case for freeing up IT resources wasted on maintaining low-value data, you’re free – at last - to hit the delete key.
Thursday, April 17, 2014
Points to remember for building a Big Data supply chain
Big data can have a large impact on the supply chain and that is exactly how the majority of supply chain executives think about it. Using the different sales data, product sensor data, market information, events and news happening in the world, competitor data and weather conditions can give insights in the expected demand of products used or required in the supply chain. Using predictive algorithms the inventory can be optimized for Just-in-Time delivery and inventory based on real-time demand forecasts. Collaboration with different players within the supply chain can help to shape demand for all organizations within the supply chain to deliver a better B2B and B2C experience.
Following are the few points to remember:
1. Identify business goals
No one should deploy big data without an overall vision for what will be gained. The foundation for developing these goals is your data science and analytics team working closely with subject matter experts. Data scientists, analysts, and developers must collaborate to prioritize business goals, generate insights, and validate hypotheses and analytic models.
2. Make big data insights operational
It's imperative that the data science team works in conjunction with the devops team. Both groups should ensure that insights and goals are operational, with repeatable processes and methods, and they communicate actionable information to stakeholders, customers, and partners.
3. Build a big data pipeline
The data management and analytics systems architecture must facilitate collaboration and eliminate manual steps. The big data supply chain consists of four key operations necessary for turning raw data into actionable information.
These include:
- Acquire and store: Access all types of data from any platform at any latency through adapters to operational and legacy systems, social media, and machine data, with the ability to collect and store data in batch, real-time and near-real-time modes.
- Refine and enrich: Integrate, cleanse, and prepare data for analysis, while collecting both technical and operational metadata to tag and enrich data sets, making them easier to find and reuse.
- Explore and curate: Browse data and visualize and discover patterns, trends, and insights with potential business impact; curate and govern those data sets that hold the most business value.
- Distribute and manage: Transform and distribute actionable information to end-users through mobile devices, enterprise applications, and other means. Manage and support service-level agreements with a flexible deployment architecture.
