Data mining is a set of processing and analysis techniques used to identify connections and correlations in large volumes of data. It is used to convert raw data into meaningful and useful information. Data Mining uses techniques (usually algorithms) from the fields of Machine Learning and Statistics which include:

• Data preprocessing gathers, normalizes and cleans data in order to make it accessible and usable by algorithms.

• Outlier detection aims to identify entries that significantly deviate from the rest of a dataset (input error, exceptional atypical value, etc.) in order to delete, keep or correct them.

• Data visualization is the representation of information and data through visual elements (e.g., graphs, charts, etc.). It can be an end in itself or help Data Scientists see and understand features, trends and patterns in data.

• Pattern mining is a set of techniques that help to find and identify relationships (causality, sequential, etc.) between variables in a dataset.

• Clustering divides data into groups with similar properties. This technique is useful to find similarities between unlabeled data or to check the quality of a data labeling process.

• Predictive analysis aims to predict future values and trends based on known preprocessed and labeled data. To do this, it looks for patterns that can lead to predictions and plausible models.

Machine Learning is a branch of Artificial Intelligence. It relies on methods, or algorithms, to automatically create models from data. Unlike explicit rule-based systems (which always perform a task in the same way), Machine Learning models learn from experience, adapt and improve their performance when they are exposed to more data.

Machine Learning algorithms are “trained” to recognize features or patterns in data and derive “lessons” from them. They then apply these “lessons” to new data to make predictions and/or decisions.

There are two main types of Machine Learning:

• Supervised learning exposes algorithms to labeled training datasets in which inputs and outputs are predefined by data scientists. This method is used to perform classification tasks (assigning a class to a data item) or regression tasks (assigning a mathematical value, number or percentage, to a data item).

• Unsupervised learning exposes algorithms to unlabeled datasets in which they seek to identify recurring features or connections. This method is used for clustering tasks (grouping data into the most homogeneous sets possible) or collaborative filtering (e.g. personalized recommendation system).

Reinforcement learning, a behavioral approach to learning, complements the two main types. It involves setting up a system of “rewards” and “punishments” in a closed environment. This system allows the algorithm to develop, by trial and error, the optimal sequence of actions or decisions to solve a problem.

Machine Learning models have various structures and learning processes. “Classic” ML methods are often integrated with, or used alongside, more recent approaches based on Artificial Neural Networks (ANN).

Deep Learning is a subfield of Machine Learning. Deep Learning models are based on Deep Neural Networks which are ANNs with several hidden layers. In these networks, each layer refines the findings of the previous layer. The term “deep” refers to the depth of the Neural Network which is defined by the number of hidden layers it contains.

At the crossroads of linguistics and computer science, NLP is a branch of artificial intelligence that helps computers decipher and understand human language (written or spoken). NLP uses algorithmic models to identify and extract elements and rules from unstructured natural language. It then uses this information to turn natural language into data that can be processed by a computer system.

Computer Vision is another branch of artificial intelligence. This technology reproduces some complex aspects of human vision and allows computers to recognize and process objects, people or movements in images and videos. Computer vision has greatly benefited from the advances in the field of Deep Learning. It is now able to match or surpass human performance in certain tasks of object detection and identification.

Although ML code is the essence of a Machine Learning solution and drives all the decisions, it is only a small part of the overall system that needs to be developed for a solution to be efficient and reliable.

MLOps (or Machine Learning Operations) is an emerging engineering discipline that aims to unify the development and deployment of Machine Learning models. Inspired by DevOps, MLOps involves creating an automated production environment (ML pipeline), from data collection and preparation to model deployment and monitoring. Its goal is to sustain the performance of an ML model over time and thus ensure the success of an automation solution.