Although many of us would like to believe that we have a knack for instinctively knowing what someone else is thinking, in reality, what people really want or need is not always what we think it is.
A few years back there was a popular set of movies – “What Women Want” and “What Men Want” – with story lines based on the premise that the hero and heroine somehow acquires magical powers (Mel Gibson gets an electrical shock from a hairdryer) that lets them know someone’s innermost thoughts. The plot of the comedies revolved around how far off the protagonists were with regard to awareness versus reality.
What I have found during the many years that I have been working with customers and their data and analytics strategy is that, like in the movies, there are some common disconnects. In my experience, what customers say they want in a data analytics solution and what they really need to address their business challenges are frequently not the same. The following thoughts are not from magical powers depicted in the movies but from my own experience.
Problem of plenty
It is no secret that there are a huge number of data tools and technologies for data analytics in the marketplace. The number of start-ups coming up in this space is rising. There is also ongoing consolidation in the data analytics space through M&A activities, which requires due diligence to make sure any investments are not short-lived.
Most customers today are either not sure which is the right technology for them, or if they even know enough about what‘s available in the market to make an informed choice. The fear of missing out on something cool that could be a ticket to the next promotion is very high!
Yet, while most leading and competing technologies are at feature parity, in terms of what an enterprise would need, what is more critical is which technologies work well together. Another aspect to consider: does a specific technology or a tool have something unique that you can significantly exploit and make a differentiating factor for your solution?
A lot depends on making the best choice for data analytics. What if there was a way for customers to get their hands dirty and actually try out options so they could be sure they have made the right choice? In my experience, this is what most customers really want when they are choosing new technologies.
Modernization with cloud-native and DevOps
Modernization is not a problem for the New Age born-in-the-cloud enterprises, but as we all know, there a not many of these around. Many enterprises today are built on legacy data technologies and, more importantly, still follow a legacy process to go from data to insights. “Speed to Insights,” which is the most important metric in today’s world of data and analytics, is lost.
Companies have many choices and de-risking the critical path on those choices is essential. A trusted neutral advisor to help select the right platform and tools is important.
More challenging than the choice of the technology itself, however, are the critical steps in the cloud migration process. What should go first? What happens while the data migration is in progress? How do you get everyone on the new platform? A misstep could cause the entire cloud migration process to stumble.
One more important aspect often overlooked is that the process of creating and consuming analytics must also modernize, along with the technology, or all of the advantages of moving to a new technology are lost.
For example, your organization may be accustomed to creating reports and dashboards based on business users. With New Age technologies, you can ask your business users to type their questions in a Google-like search to obtain insights. In this situation, you focus more on ensuring a fast and efficient process for ingesting new, clean data into your data store. After all, with a power like this, end users will be asking questions from your system at speeds and quantities that were previously unimaginable. So while customers will focus a lot on the migration aspects related to technology, the modernization of the process and the consumption ways and means associated with it may not be discussed enough.
While DevOps did not exist ten years ago, as your data processes progress will you be prepared to devote the necessary focus and resources?
Outcome and solutions, not tools
What I have seen time and again during my career is that one of the most common reasons for a failed analytics initiative is a data strategy without an aligned business strategy. This will fail.
What I mean is that well-defined business outcomes are crucial for any data strategy to succeed. What does your business want to accomplish with data analytics? Most data architectures are tech-centric, rather than information-centric. In many cases, companies focused solely on the outcome are not interested or inclined to understand the technology stack that was used for analytics. They may believe that they simply need an easy and quick way to validate outcomes from the data assets that they have and then push them to production as quickly as possible. Do you think this is a viable approach?
The non-functional world
Customers sometimes tell us they are not too concerned about the business logic or the functional aspects of their analytics application since they know they will get it done. What they struggle with is the adoption of that analytics application, which essentially boils down to the application’s non-functional aspects.
This phenomenon is not limited to traditional nonfunctional requirements (NFRs) such as security, performance, authentication and authorization, but goes well beyond. What will be the OPEX cost of the solution (which is important in the cloud pay-as-you-go model), what-if calculations around that, should we go serverless, the common security model across the entire tech stack, job frequency/monitoring/status, data dictionary, and so on. Even with artificial intelligence (AI) and machine learning (ML), it is not about the algorithm or the accuracy and recalls as much as it is about the explainability or the auditability of the ML module in terms of the model version and the data sets that it was trained on, the consumption of the ML insights, and so on.
What’s your take on the importance of non-functional requirements and adoption over the functional requirements
AI/ML onslaught
What I’ve found is that customers are tired of hearing how AI/ML can transform their business. While there are still a few who want to know the AI/ML use cases that could impact their business, most already have a good idea of the relevant ones. What’s missing is the feasibility and the “how” part of the whole AI/ML equation. Partners who can address those questions have a higher chance of success.
Enterprise adoption of ML is still at an early stage since the whole model development life cycle has not completely evolved. A successful ML enterprise adoption requires considering these aspects of a model: governance (including traceability, verifiability and reproducibility); collaboration-reviews-workflows; smart provisioning (infra allocation and usage monitoring); monitoring and tracking (deployment and validations); and operationalization (deployment and validation).
Of the four key stages of an ML project — Data Management, Model Development, Production Deployment and Model Lifecycle Management — enterprises are highly focused and spend significant effort on the first two phases, often at the cost of the last two phases. The later phases have not matured as much, but are increasingly a differentiating factor between a successful and a failed ML program.
Can we talk?
It doesn’t take a goofy box office comedy to realize that what people really want or need is not what always seems to be obvious or visible. And no weird drink or bizarre injury is going to give any of us magical powers for sudden insight. Today, we must challenge assumptions about what customers truly need, from a data analytics perspective.
Data and analytics are much more than tools and technologies. Practitioners who understand this, without becoming enamored by technology itself, and address these needs are the ones who have a high chance of success. What is essential is not just a product or services, but something that sits in between. Services which are accelerated by IP or solutions and which address these needs and enforce best practices as a part of the process will see significant success in the highly complex and competitive data and analytics space. Something that will deskill the game, does not require reinventing the wheel each time, and de-risk the critical path is what is needed.
Will you be attending AWS re:Invent in Las Vegas December 2-6? If you’re there, please feel free to stop by our Booth #312 and let’s discuss if you agree or disagree with my views. If we won’t see you in Las Vegas, would you like to share your thoughts on this blog?
