data analytics

Data-driven learning systems – extracting operational value from data warehouses

bigstock-Web-Analytics-Doodle-Elements-52160116

Since the early 2000s, virtually all modern companies have invested heavily on different kinds of data collection and warehouse solutions. These massive investments have now been in use for a decade or so and the outcome is that most of the firms have significant amounts of data from their business. The question that now has been raised, what practical impact these systems actually had?

Of course much has change in companies – both for good and for bad. Much of the development has been on managerial level, where e.g. periodical reporting could be largely automated. But when looking these systems from operational level, the picture starts to look different. There seems to be a growing feeling that these large databases would have great value for firm’s daily operations, but so far the usage of these systems has been limited or fully neglected.  I have even heard these databases referred as ‘graveyards of information’, as most of the data recorded will never be analysed or used to improve firm’s operations. It seems that something is missing from the link between firm’s information systems and daily operations.

“It seems that something is missing from the link between firm’s information systems and daily operations.”

In a traditional organization, the link between large data systems and firm’s operational processes is organized so that analysts are in a central role. They implement tailored data analyses to understand selected problems. The outcome from analysis is a report or presentation which is then disseminated for personnel in firm’s operational processes.

traditional

Figure 1. Traditional data analysis structure

The benefit of analyst driven system is that there is virtually no limitation on the potential analyses, other than the skill level and persistency of the analyst. The downside is that level of automation in the cycle is limited, especially delivery of results relies on traditional reporting or education services.

Development of computational data analysis methods has made it possible to create automated decision support systems to facilitate continuous connection between firm’s data storages and daily operations. Technically these systems can be based on e.g. machine learning, where the massive data storages are used to teach the system to understand firm’s operational processes. In machine learning world, the more data is available – the better the suggestions made by the system.

Integration of these automated decision support systems offers an alternative connection between firm’s data warehouse and daily processes that should be seen as parallel method to the traditional analyst. The analyst based learning still offers the most flexible way to find new insights from data, while the automated system offers continuous and scalable support for a specific problem area. The key in designing these systems, is to focus on ease of use and effective delivery of key findings from analytics. If these can be achieved, these systems can have significant effect to the efficiency of firm’s daily operations.

“Integration of these automated decision support systems, offers an alternative connection between firm’s data warehouse and daily processes that should be seen as parallel method to the traditional analyst.”

Integrated

Figure 2. Data analysis structure with automated analysis

The expected benefit from computational support system is that the quality of operational decisions can increase significantly leading to either increased production capability or process efficiency. The further benefit of automated system is that it can be integrated to be a part of normal daily operations. As such the system can always be present to support daily operations. The downside of such a system is that they can be used only for the intended use purpose(s), creating a need for outside analyst in a support role. Additionally, development of these systems is also a relatively large software development project, but the costs are likely small compared to high costs of building and maintaining data collection systems and the potential benefits from the system.

These kind of systems are currently still rare, but due to current hype around analytics systems they are likely to become more common. Most public cases of such systems are from service industry such as banking, where machine learning based systems are used to apprise loan applications[1].

“…the quality of operational decisions can increase significantly leading to either increased production capability or process efficiency.”

From more philosophical level this new link creates a new form of organizational learning. In this learning process the firm’s data warehouses are treated as an ‘extremely large memory’, where the amount of data is so high that computational analysis methods are the only way to rapidly make sense of the data – Data-driven learning systems. We are only now starting to reach the maturity level in analytics that allows for creating these data-driven learning systems. The future is full of great opportunities and it is going to be exciting to see how far the boundaries of these systems can be pushed in the near future.

[1] See case description e.g. from Siegel (2013), book contains also additional case stories. Siegel, Eric (2013), ’Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die’, Wiley publishing.

Knowledge is broken

OLYMPUS DIGITAL CAMERA

Data is the new oil” is well hyped slogan. As with oil, data is meaningless unless you use and capitalize it. Amount of data is growing but is our sense-making capacity matching the speed?

The way to make money with oil is to pump it out, refine it and only then we have modern infrastructure. With data money-making is similar as it needs to be refined to valuable information and then it can truly drive the development.

Intriguingly, combining different sources and reusing data just increases its value. We can save atoms, nature, human lives and nature with smart traffic, cities and healthcare.

“In addition, tools perfect for linear world are still mainstream, even though our world and business environment are growing in complexity.”

But do we have capacity for data refinement in firms and society at large?

Our experience from different industries and public sector tells quite grim story; most organizations still work in process silos supported by process specific data silos. Horizontal view or enterprise architectures are either completely missing or only theoretical concept level designs. In addition, tools perfect for linear world are still mainstream, even though our world and business environment are growing in complexity.

However, there is a growing minority of leaders and professionals accepting the fact that in complex world you cannot survive with command and control type of management systems and tools. These businesses are coming up with tools to refine and make new value with their data.

In the future they are the ones controlling the business as the likes of Google and Facebook are currently dominating information platforms.