What Makes Dataiku DSS the Ideal Solution for Enterprise AI?

An assignment I was working on a few months ago involved predicting the renewal rates of different products that had different parameters. This time-bound activity needed to produce qualitative results and thus it had me analyzing various data sets, making histograms, imputing missing values, normalizing data and more. Despite applying various pre-processing techniques, fitting various models and attempting innumerable dimensionality reduction methods due to the large data size, I was unable to bring down the RMSE (Root Mean Square Error) score. With the clock ticking, I began to get more than a little worried!

Fortunately, I had some training in the deep learning RNN model I could lean on, which gave me some satisfactory results that we presentable to the client. But satisfactory is not something I was happy with! While performing this analysis on Python, I realized that a tool, into which I could drop this dataset and use different drag and drop features to perform different types of analysis, would be highly convenient and time-efficient.

Although several cloud-based sources like Google, Microsoft and Amazon can provide various tools to accomplish tasks like these, however when dealing with sensitive client data, one needs to be more diligent when choosing the right tool. It was while exploring these options that I came across the DataIKU Data Studio (DSS).

What is Dataiku DSS?

An AI-powered data product, Dataiku DSS provides machine learning capabilities with a user-friendly graphical interface. This collaborative data science software platform is perfect for teams of data scientists, data analysts, and engineers to explore, prototype, build, and deliver data products in a quick and efficient way.

It is available in different categories, at varying price-points and for a range of Operating Systems, but the best part is that the trial version is free. For my specific requirements, I went ahead and downloaded the ‘.ova’ file and installed it in the virtual machine. After a quick installation, I was excited and ready to explore the tool.

Here is what I found.

Key Features

Following are some key highlights of Dataiku DSS that can give you a quick introduction to the tool.

Allows input from the various sources
Contains various data pre-processing techniques
Holds auto-ML capabilities
Includes inbuilt ML algorithms and options to provide custom algorithms
Provides various evaluation metrics and export options
Enables easy dashboard creation
Supports ML model deployment with Rest API

Another great feature of DSS is that it provides a step-by-step tutorial for quickly setting up a project. The ML data required is preloaded. Users can follow this tutorial page for a quick overview of tools. For demo purposes, I am unable to share details about my previous work since it involves sensitive client data, therefore I will show you the tool’s capabilities with the very popular ‘titanic’ data set. You can download the set from here.

Once you have the tool and the data set handy, we can start the model building process. Please note, during this model building, I will be skipping the data pre-processing part and feature engineering part.

Demonstration

Please follow the tutorial given in the reference section and create a sample project. Then, import the data and Titanic data set into it. Create a sample lab environment and configure the different algorithms as shown in the images below.

1) Model building

Image 1 features the different available algorithms and configuration options. To see all the algorithms, go to Design -> Modeling -> Algorithms. You can change the different hyperparameter of any algorithm in this window. A variety of feature engineering and feature reduction options are also available.

2) Result

Without modifying any parameters, I start the training with the default 80:20 train/test ratio. Execution is very quick and results get generated in less than 2 minutes. Image 2 has more details. Surprisingly, DSS gives 90% accuracy on test data for “Gradian Boosting Tree” with Auto ML capability, all in less than 2 minutes. Apart from GBM, other algorithms also perform astonishingly well. For every algorithm, internal details like confusion metric, accuracy, ROC curves, key features are available in their individual algorithm tab.

Architecture Of DSS

DSS Architecture

The DSS Architecture, as shown in the above image from the perspective of Data Science, has 3 major components – Insight, Model and Services/Dashboard. DSS also supports many advanced features using which one can enhance the model’s capability, namely –

1) Input: DSS can take inputs from different sources like the SQL dataset, Hadoop & Spark, No-SQL database, cloud-based dataset, and file system as shown in Architecture.

2) Explorative Data Analysis: DSS provides a wide variety of options for data processing techniques. This part forms the ‘insights’ that the architecture is able to provide. These include,

Building univariate statistics automatically & deriving data quality checks
Filtering data based on different criteria
Offering data cleansing and string pre-processing methods
Providing different numeric formulae to process numeric fields
Presenting custom formulae in Python or R

3) Data Visualization: DSS provides different visualization to plots. This is also a feature that provides insights. This involves,

Creating standard charts (histogram, bar chart, etc.) and scale charts’ computation by leveraging underlying systems
Creating custom charts using Custom Python-based or R-based Charts

4) Model Generation: This provides a wide variety of Machine Learning models with the ability to handle a variety of features. This forms the ‘model’ part of the architecture and includes the following characteristics,

Feature handling for Machine Learning
- Support for numerical, categorical, text and vector features
- Automatic pre-processing of categorical features (Dummy encoding, impact coding, hashing, custom pre-processing, etc.)
- Automatic pre-processing of numerical features (Standard scaling, quantile-based binning, custom pre-processing, etc.)
- Automatic pre-processing of text features (TF/ IDF, Hashing trick, Truncated SVD, Custom pre-processing)
- Various strategies for imputation of missing values
  - Features generation
    - Feature-per-feature derived variables (square, square root, etc.)
    - Linear and polynomial combinations
  - Features selection
    - Filter and embedded methods
- Outliers detection in clustering methods
In-build support for hyperparameter optimization with grid search
Analysis of graphical model results with different parameters for supervised and unsupervised techniques
Deep Learning support

5) Dashboard: (Services/Dashboard part of Architecture)

Dashboard mechanism that showcases insights
Graphs from Data Visualization methods can be exported to the dashboard with one click
Different model artifacts like an optimal set of parameters and scoring metrics (AUC ROC, confusion metrics, accuracy, etc.) can be exported to the dashboard
The dashboard can be exported in PPT and PDF format

6) Deployment:

Inbuilt capability for production deployment with version control
Bundles of the developed model, train data and other folders created. Bundles can be deployed on any node with the help of Dataiku.

The above features only cover the Machine Learning aspects, there are many more features to be explored and utilized.

Conclusion

We have explored only a small portion of what the DataIKU DSS is capable of. DSS supports various functionalities related to diverse ML tasks and provides a one-click option to build dashboards quickly. In addition to ML tasks, DSS also provides parallel computing, GPU support and Rest API services. Clearly, we have a very powerful tool on our hands that can be useful for developing POCs very quickly.

I encourage you to explore this tool with the links I have provided throughout the article and also via the reference links listed below.