data-driven learning

Data-driven learning systems – extracting operational value from data warehouses

bigstock-Web-Analytics-Doodle-Elements-52160116

Since the early 2000s, virtually all modern companies have invested heavily on different kinds of data collection and warehouse solutions. These massive investments have now been in use for a decade or so and the outcome is that most of the firms have significant amounts of data from their business. The question that now has been raised, what practical impact these systems actually had?

Of course much has change in companies – both for good and for bad. Much of the development has been on managerial level, where e.g. periodical reporting could be largely automated. But when looking these systems from operational level, the picture starts to look different. There seems to be a growing feeling that these large databases would have great value for firm’s daily operations, but so far the usage of these systems has been limited or fully neglected.  I have even heard these databases referred as ‘graveyards of information’, as most of the data recorded will never be analysed or used to improve firm’s operations. It seems that something is missing from the link between firm’s information systems and daily operations.

“It seems that something is missing from the link between firm’s information systems and daily operations.”

In a traditional organization, the link between large data systems and firm’s operational processes is organized so that analysts are in a central role. They implement tailored data analyses to understand selected problems. The outcome from analysis is a report or presentation which is then disseminated for personnel in firm’s operational processes.

traditional

Figure 1. Traditional data analysis structure

The benefit of analyst driven system is that there is virtually no limitation on the potential analyses, other than the skill level and persistency of the analyst. The downside is that level of automation in the cycle is limited, especially delivery of results relies on traditional reporting or education services.

Development of computational data analysis methods has made it possible to create automated decision support systems to facilitate continuous connection between firm’s data storages and daily operations. Technically these systems can be based on e.g. machine learning, where the massive data storages are used to teach the system to understand firm’s operational processes. In machine learning world, the more data is available – the better the suggestions made by the system.

Integration of these automated decision support systems offers an alternative connection between firm’s data warehouse and daily processes that should be seen as parallel method to the traditional analyst. The analyst based learning still offers the most flexible way to find new insights from data, while the automated system offers continuous and scalable support for a specific problem area. The key in designing these systems, is to focus on ease of use and effective delivery of key findings from analytics. If these can be achieved, these systems can have significant effect to the efficiency of firm’s daily operations.

“Integration of these automated decision support systems, offers an alternative connection between firm’s data warehouse and daily processes that should be seen as parallel method to the traditional analyst.”

Integrated

Figure 2. Data analysis structure with automated analysis

The expected benefit from computational support system is that the quality of operational decisions can increase significantly leading to either increased production capability or process efficiency. The further benefit of automated system is that it can be integrated to be a part of normal daily operations. As such the system can always be present to support daily operations. The downside of such a system is that they can be used only for the intended use purpose(s), creating a need for outside analyst in a support role. Additionally, development of these systems is also a relatively large software development project, but the costs are likely small compared to high costs of building and maintaining data collection systems and the potential benefits from the system.

These kind of systems are currently still rare, but due to current hype around analytics systems they are likely to become more common. Most public cases of such systems are from service industry such as banking, where machine learning based systems are used to apprise loan applications[1].

“…the quality of operational decisions can increase significantly leading to either increased production capability or process efficiency.”

From more philosophical level this new link creates a new form of organizational learning. In this learning process the firm’s data warehouses are treated as an ‘extremely large memory’, where the amount of data is so high that computational analysis methods are the only way to rapidly make sense of the data – Data-driven learning systems. We are only now starting to reach the maturity level in analytics that allows for creating these data-driven learning systems. The future is full of great opportunities and it is going to be exciting to see how far the boundaries of these systems can be pushed in the near future.

[1] See case description e.g. from Siegel (2013), book contains also additional case stories. Siegel, Eric (2013), ’Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die’, Wiley publishing.