In the iconic Hollywood classic Sunset Boulevard, Norma Desmond famously utters the last line “All right Mr. DeMille, I’m ready for my close-up.” Unlike Ms. Desmond, who was a fading and delusional actress past her prime, “Digital Data” is a rising star that just needs some smart direction to realize its full glory!
Smart: We’re hearing that word a lot. In fact, in our new digital world, the most common adjective we keep hearing is “smart”: smart devices, smart cities, smart systems… to use our Hollywood metaphor, we even hear smart writing, smart dialogue, etc. Through digital transformation, every application, system, device, etc. is being made smarter. Data Analytics has been a major driver in making things smart, but how can we make analytics SMART? Just like in Hollywood, we like to really work a storyline, so looking at an analytics dashboard is like watching a movie trailer; the success of the movie (or the analytics dashboard) largely depends on the trailer conveying SMART messages – and it helps to have a big star.
In the previous posts of this series, we have covered the readiness of the Enterprise Data Platform for digital transformation along with mechanisms and tools for ingesting various kinds of enterprise data, the data processing options and capabilities including data lakes, how enterprise platforms have different shapes and flavors to store data and importance of data quality. In this blog post, Sunil Agrawal and I discuss the aspects of analytics that make it SMART in the digital enterprise.
Traditionally, enterprises have deployed data warehouse solutions, usinganalytics layers to determine historical trends, and predictive layers to forecast using sample data. Today’s mobile and digital world poses many challenges to enterprises. If enterprises embrace a digital data-driven platform for their data warehouse option, it can open a whole new world of opportunity, as they can engage with a wide spectrum of existing customers very easily. In a digital data platform, it is important to build the right kind of analytics layer based on business use cases, whether it is to improve customer satisfaction, or to increase sales revenue, or to optimize operational efficiency, or any combination of these. The diagram below shows where analytics fits into the enterprise digital data driven platform.
Enterprises need to make best use of the analytics layer by ensuring they identify their use case priorities. For consumer-centric enterprises, most of the digital business use cases can be categorized into:
- Understanding individual customer patterns and behaviors at a fine grain by including data from multiple sources, including public sources such as the web and social media – rather than classifying them in only 2-3 broad categories based on their profile (like gender, age, and address).
- Using the insight to do smart cross-selling and up-selling to existing customers.
- Identifying inefficiencies in systems based on usage pattern of infrastructure and customers to optimize operational cost, in turn helping reduce price of products.
In this blog, we will focus on the following ways of making the analytics layer SMART.
S: Search-Based Analytics
Traditionally, enterprise search has been mainly used to search through file servers and content management servers for knowledge management applications. In the digital world, however, it has become important to extend search for deploying applications with analytical facets based on search. Enterprises need to build connectors to traverse through internal enterprise database systems as well as external sources – social, web and (possibly) vendors’ procurement systems. This helps enterprise business users perform self-service data discovery. One other common use of search is going through various system logs to detect anomalies by applying machine learning algorithms.
M: Machine Learning Analytics
While enterprises have deployed predictive analysis in their existing data warehouse solution, they have shied away from retaining many years’ worth of data due to the cost of data storage and processing. With new enterprise digital data platforms supporting big data technologies, it now becomes feasible to run statistical models across multiple years of data instead of against sample data of last 3-6 months. This additional capability will improve the accuracy and self-learning of these models to make them truly “machine learning analytics”. A good example comes from the telecom industry, where customer churn has been the most significant problem to solve. Improving prediction accuracy and reducing customer churn by even 0.1 percent could save a telecom service provider millions of dollars, depending on the size of its subscriber base. So it is important that these machine learnings can turn into effective actionable analytics.
A: Actionable Analytics
Traditionally, enterprises use data warehouses for everything from reporting to monitoring. With the digital data driven platform, enterprises have the opportunity to go beyond this limitation into increasing the business value of their data platform into actionable analytics. That is, not only should they be able to forecast what is going to happen, but they should also be able to control how that is going to happen and make it happen (or prevent it from happening) based on the business requirement.
For actionable analytics to be successful, it is important that analytics is done as real-time as possible as data flows (torrents) through the enterprise data platform.
R: Real-Time Analytics
In building an analytics layer for an enterprise, it is important to understand the time horizon required for the business use cases and to use that information to figure out priorities. Everything has associated cost: if more real-time data is required by a business use case, be ready to shell out more money for it.
Real-time analytics helps enterprises track the popularity of their products and services as they are launched. Customer representatives look for real-time predictive analytics in order to make suggestions to customer while in discussion with them. Hence real-time solution has become the desire of the analytics layer; however, as this capability is not free, the ROI needs to be used as a guide to determining the amount of real-time analytics that can be supported for a given budget.
T: Torrent (Stream) Analytics
While customers would love for enterprises to provide them with unlimited real-time analytics, the reality is that performing real-time analytics on the entire data could be technically challenging due to the volume and velocity attributes. In addition, the flow of this data could be complex, unstructured, and unpredictable, which can be overwhelming for a real-time solution to handle. In practice, therefore, an enterprise would need to implement stream analytics solutions to analyze the information from a bucket of data as it flows for a minute or so. This helps to ensure that analytics algorithm runs on data sets rather than individual data items (to provide context) while keeping these sets reasonably sized to keep real-time analytics feasible.
Conclusion
While data volume and velocity have increased multi-fold in the digital world, this data will rapidly become stale and lose its value unless a SMART Analytics layer is built to convert it into useful insights. Data scientists and business leaders within enterprises must drive proper use of this layer and raise the maturity level of the organization from reporting to actionable analytics. SMART analytics will be the key factor in enterprise digital transformation that will differentiate the “smart enterprises” from others. To paraphrase that famous opening line, “All right Enterprise, Digital Data is ready for its close-up!”
Image Credits: Paramount Pictures
Dr. Siddhartha Chatterjee is Chief Technology Officer at Persistent Systems.
Sunil Agrawal is Chief Architect at Persistent Systems.
