Insights for the
intelligence era.

Low-Rank Factorization

The key idea behind low-rank factorization is to replace high-dimensional tensors with lower-dimensional tensors. One type of low-rank factorization is compact convolutional filters, where the over-parameterized (having too many parameters) convolution filters are replaced with compactblocks to both reduce the number of parameters and increase speed.

Perturbation Test for a Regression Model

A perturbation test is a method used to evaluate a model’s robustness and stability. In machine learning, this test helps determine how sensitive the model’s predictions are to small changes (perturbations) in the input data. If a model is stable, small changes in the input should lead to minimal changes in the output. This method…

Calibration Curve for Classification Models

Calibration curves are specifically used for classification models. The primary goal of a calibration curve is to evaluate the reliability of the predicted probabilities in a classification task. A calibration curve checks how well predicted probabilities align with the actual observed frequencies (e.g., when a model predicts 70% probability of being positive, we expect about…

Correlation Coefficient

Correlation measures the strength and direction of the linear relationship between two variables. The formula for the correlation coefficient (Pearson’s r) is: Where:

Delete a folder in Google Colab

To delete a folder in Google Colab, you need to first remove all the files and subfolders within it. Here is a step-by-step guide on how to do this using Python and shell commands:

Building a CNN model for Fashion MNIST dataset

Keras library wrapper classes 

KerasClassifier is a wrapper class provided by the Keras library that allows you to use a Keras neural network model as an estimator in scikit-learn workflows. This wrapper enables you to leverage the extensive functionality of scikit-learn, such as cross-validation, grid search, and pipelines, with Keras models seamlessly. Here’s how KerasClassifier works: Here’s a simple…

LabelEncoder of scikit-learn library

LabelEncoder is a utility class provided by the scikit-learn library in Python, specifically in the sklearn.preprocessing module. It is commonly used for encoding categorical labels into numerical labels. Here’s what LabelEncoder does: Here’s an example of how to use LabelEncoder: Keep in mind that LabelEncoder is suitable for encoding target labels (dependent variables) in supervised…

What is Deep Learning

Deep learning is a subset of machine learning that utilizes artificial neural networks with multiple layers (hence “deep”) to learn and extract features from data. It has gained significant attention and popularity due to its ability to automatically learn hierarchical representations of data, which allows for more effective feature extraction and modeling of complex relationships…

Neural Network model building

In neural networks, there are so many hyper-parameters that you can play around with and tune the network to get the best results. Some of them are: Here’s an example of building a neural network model with two hidden layers using the Sequential API in TensorFlow/Keras: Once we are done with the model architecture, we…

Gradient Descent Optimization

Gradient Descent is an optimization algorithm commonly used in machine learning and deep learning to minimize the loss function and find the optimal parameters (weights and biases) of a model. It’s based on the principle of iteratively moving in the direction of the steepest descent of the loss function with respect to the model parameters.…

TensorFlow

TensorFlow is an open-source machine learning library developed by Google Brain team. It is one of the most popular frameworks for building and training machine learning and deep learning models. TensorFlow provides a comprehensive ecosystem of tools, libraries, and community resources to facilitate the development and deployment of various types of machine learning models. Key…

MNIST dataset in artificial neural network

In the context of artificial neural networks (ANNs), MNIST refers to the MNIST dataset, which is often used as a benchmark for training and testing ANN models, particularly for image classification tasks. The MNIST dataset consists of a large collection of grayscale images of handwritten digits from 0 to 9. Each image is a 28×28…

Multi-Layer Perceptron (MLP) in artificial neural network

A Multi-Layer Perceptron (MLP) is a type of artificial neural network that consists of multiple layers of nodes (perceptrons). Unlike a single-layer perceptron, an MLP has one or more hidden layers between the input and output layers. Each node in a layer is connected to every node in the subsequent layer. Here’s a basic overview…

Perceptron in artificial neural network

A perceptron is one of the simplest forms of artificial neural networks. It’s a binary classifier that takes multiple binary inputs and produces a single binary output. Here’s how it works:

Quantile-based discretization of continuous variables

n Pandas library in Python pd.qcut is a function for performing quantile-based discretization of continuous variables. Quantile-based discretization involves dividing a continuous variable into discrete intervals or bins based on the distribution of its values. This process ensures that each bin contains approximately the same number of observations, making it useful for creating categories or…

RandomizedSearchCV vs GridSearchCV

RandomizedSearchCV is a method provided by scikit-learn for hyperparameter tuning and model selection through cross-validation. It’s similar to GridSearchCV, but instead of exhaustively searching through all possible combinations of hyperparameters, it randomly samples a fixed number of hyperparameter settings from specified distributions. Here’s a basic overview of how RandomizedSearchCV works: Here’s a basic example of…

Get available Hyperparameters

get_params() is a method provided by scikit-learn estimators (such as classifiers, regressors, transformers, etc.) that returns a dictionary of the estimator’s parameters. These parameters are the hyperparameters that define the behavior of the estimator and can be tuned during the model selection or hyperparameter optimization process. Here’s a simple example of how you might use…

Handling missing values with SimpleImputer

SimpleImputer is a class in scikit-learn, a popular machine learning library in Python, used for handling missing values in datasets. It provides a simple strategy for imputing missing values, such as filling missing entries with the mean, median, most frequent value, or a constant. Here’s a basic example of how you might use SimpleImputer: This…

Undersampling Technique – Tomek Links

Tomek Link Undersampling is a technique used to address class imbalance in machine learning datasets. It involves identifying Tomek links, which are pairs of instances from different classes that are nearest neighbors of each other, and removing instances from the majority class that form these links. The main idea behind Tomek Link Undersampling is to…

Oversampling Technique – SMOTE

SMOTE (Synthetic Minority Over-sampling Technique) is an upsampling technique used in machine learning to address the class imbalance problem, which occurs when the number of instances of one class (minority class) is significantly lower than the number of instances of the other class (majority class) in a dataset. This class imbalance can lead to biased…

Differences between Bagging and Boosting

Bagging (Bootstrap Aggregating) and Boosting are both ensemble learning techniques that aim to improve the predictive performance of machine learning models by combining multiple base learners. However, they differ in their approach to training and how they leverage the base learners’ predictions to improve model performance. Bagging focuses on reducing variance, whereas Boosting focuses on…

XGBoost (eXtreme Gradient Boosting)

XGBoost stands for eXtreme Gradient Boosting, and it’s an optimized and highly scalable implementation of the Gradient Boosting framework. Developed by Tianqi Chen and now maintained by the Distributed (Deep) Machine Learning Community, XGBoost has gained widespread popularity in machine learning competitions and real-world applications due to its efficiency, flexibility, and outstanding performance. XGBoost Parameters…

Gradient Boosting

Gradient Boosting is another ensemble learning technique used for classification and regression tasks and has its own specific way of building the ensemble of weak learners. Here’s a brief overview of Gradient Boosting: Gradient Boosting typically produces more accurate models compared to AdaBoost but can be more computationally expensive and prone to overfitting, especially with…

AdaBoost (Adaptive Boosting)

AdaBoost (Adaptive Boosting) is a popular ensemble learning algorithm used for classification and regression tasks. It works by combining multiple weak learners (typically decision trees, often referred to as “stumps”) to create a strong learner. Here’s how it generally works: AdaBoost is effective because it focuses on improving the classification of difficult examples by giving…

BaggingClassifier from Scikit-Learn

The BaggingClassifier is an ensemble meta-estimator in machine learning, belonging to the bagging family of methods. Bagging stands for Bootstrap Aggregating. The main idea behind bagging is to reduce variance by averaging the predictions of multiple base estimators trained on different subsets of the training data. Here’s how the BaggingClassifier works: The BaggingClassifier in scikit-learn…

Parameter stratify from method train_test_split in scikit Learn

In the context of the train_test_split function in machine learning, the stratify parameter is used to ensure that the splitting process preserves the proportion of classes in the target variable. When you set stratify=y, where y is your target variable, the data is split in a way that maintains the distribution of classes in both…

t-distributed Stochastic Neighbor Embedding (t-SNE)

t-SNE, which stands for t-distributed Stochastic Neighbor Embedding, is a popular dimensionality reduction technique (of type Feature Extraction) used in machine learning and data visualization. It is particularly useful for visualizing high-dimensional data in a lower-dimensional space, typically two or three dimensions, while preserving the local structure of the data as much as possible. The…

Principal Component Analysis (PCA)

Principal Component Analysis (PCA) is a widely used linear dimensionality reduction technique (of type Feature Extraction) used for reducing the dimensionality of datasets containing many correlated variables while preserving most of the variability in the data. Here’s how PCA works: Each of the “new” variables after PCA are all independent of one another. PCA has…

Unsupervised Learning Dimensionality Reduction – Feature Elimination vs Extraction

Feature Elimination and Feature Extraction are two common techniques used in dimensionality reduction, a process aimed at reducing the number of features (or dimensions) in a dataset while preserving the most important information. Both techniques are used to address the curse of dimensionality, improve computational efficiency, and potentially enhance model performance. However, they differ in…

Cophenetic coefficient

he cophenetic coefficient is a measure used to evaluate the quality of a hierarchical clustering solution. It quantifies how faithfully the hierarchical structure (dendrogram) preserves the original pairwise distances or dissimilarities between data points. Here’s how it works: A high cophenetic coefficient suggests that the hierarchical clustering solution accurately captures the underlying structure of the…

Complete linkage hierarchical clustering

omplete linkage hierarchical clustering is another method used in cluster analysis, like single linkage clustering, but with a different approach to determining the distance between clusters. In complete linkage clustering, the distance between two clusters is defined as the maximum distance between any two points in the two clusters. So, the distance between two clusters…

Single linkage hierarchical clustering

ingle linkage hierarchical clustering is a method used in cluster analysis to group similar data points into clusters based on their proximity or similarity. It is a bottom-up approach, starting with each data point as its own cluster and then iteratively merging the closest pairs of clusters until only one cluster remains. In single linkage…

CDF plot of Numerical columns

The provided code below generates a grid of subplots (dynamic rows and 2 columns) and plots cumulative distribution function (CDF) plots for numerical variables in a DataFrame (df).

Finding the optimal number of clusters (k) using Elbow Method

he elbow method is a technique used to find the optimal number of clusters (k) in a dataset for a clustering algorithm, such as k-means. The idea is to run the clustering algorithm for different values of k and plot the sum of squared distances (inertia) between data points and their assigned cluster centroids. The…

Standardizing features by StandardScaler

n scikit-learn (sklearn), the StandardScaler is a preprocessing technique used to standardize features by removing the mean and scaling them to have a unit variance. Standardization is a common step in many machine learning algorithms, especially those that involve distance-based calculations or optimization processes, as it helps ensure that all features contribute equally to the…

What is Silhouette Coefficient

he silhouette coefficient is a measure of how well-separated clusters are in a clustering analysis. It provides a way to assess the quality of clustering by evaluating both the cohesion within clusters and the separation between clusters. The silhouette coefficient ranges from -1 to 1, with higher values indicating better-defined clusters. Here’s how the silhouette…

What is Mahalanobis Distance

he Mahalanobis distance is a measure of the distance between a point and a distribution, taking into account the correlation between variables. It is often used in statistics and machine learning to identify outliers and to assess the dissimilarity between a data point and a distribution. The Mahalanobis distance is defined for a point (x)…

What is Jaccard Distance

accard distance is a measure of dissimilarity between two sets. It is calculated as the complement of the Jaccard similarity coefficient and is particularly useful when dealing with binary data or sets. The Jaccard similarity coefficient measures the proportion of shared elements between two sets, and the Jaccard distance is essentially the complement of this…

What are the common Distance Measures in Clustering

istance measures (or similarity measures, depending on the context) play a crucial role in clustering algorithms, as they determine the similarity or dissimilarity between data points. Here are some common distance measures used in clustering: The choice of distance measure depends on the nature of your data and the specific requirements of your clustering task.…

Choosing the right estimator

Often the hardest part of solving a machine learning problem can be finding the right estimator for the job. Different estimators are better suited for different types of data and different problems. The flowchart below is designed to give users a bit of a rough guide on how to approach problems with regard to which…

What is Logistic Regression?

ogistic Regression is a statistical method used for binary classification tasks, where the outcome variable is categorical and has two classes. Despite its name, it is used for classification rather than regression. The logistic regression algorithm models the probability that a given input belongs to a particular class. The logistic regression model applies the logistic…

NumPy function argmax

np.argmax is a NumPy function that returns the indices of the maximum values along a specified axis in an array. If the input array is multi-dimensional, you can specify the axis along which the maximum values are computed. Here’s a simple example: Output: In this example, np.argmax(arr) returns the index (position) of the maximum value…

Parameter cv in GridSearchCV

In scikit-learn’s GridSearchCV (Grid Search Cross Validation), the parameter cv stands for “cross-validation.” It determines the cross-validation splitting strategy to be used when evaluating the performance of a machine learning model. When cv is set to an integer (e.g., cv=5), it represents the number of folds in a (Stratified) K-Fold cross-validation. For example, cv=5 means…

NumPy function argsort

np.argsort is a NumPy function that returns the indices that would sort an array along a specified axis. It performs an indirect sort on the input array and returns an array of indices that represent the sorted order of the elements. The returned indices can be used to construct a sorted version of the input…

Post-pruning Decision Tree with Cost Complexity Parameter ccp_alpha

Now put the printed best_model here below and run:

Pre-pruning Decision Tree – GridSearch for Hyperparameter tuning

Grid search is a tuning technique that attempts to compute the optimum values of hyperparameters. It is an exhaustive search that is performed on the specific parameter values of a model. The parameters of the estimator/model used to apply these methods are optimized by cross-validated grid-search over a parameter grid.

Pre-pruning Decision Tree – depth restricted

In general, the deeper you allow your tree to grow, the more complex your model will become because you will have more splits and it captures more information about the data and this is one of the root causes of overfitting. We can limit the tree with max_depth of tree:

Feature Importance in Decision Tree

In scikit-learn, the feature_importances_ attribute is associated with tree-based models, such as Decision Trees, Random Forests, and Gradient Boosted Trees. This attribute provides a way to assess the importance of each feature (or variable) in making predictions with the trained model. When you train a tree-based model, the algorithm makes decisions at each node based…

Visualizing the Decision Tree

To visualize a decision tree in scikit-learn, you can use the plot_tree function from the sklearn.tree module. This function allows you to generate a visual representation of the decision tree. Here’s a simple example: To show the decision tree as text in scikit-learn, you can use the export_text function from the sklearn.tree module. This function…

Python scikit-learn library for Decision Tree model

For building a decision tree model in scikit-learn (sklearn), you need to import the relevant modules and classes. Here are the main components you’ll typically use:

Import your functions library to a Google Colab notebook

To import a function from a separate Jupyter Notebook (.ipynb file) into another notebook, you can use the %run magic command:

Get a random sample from your dataset

To grab random sample from a dataset in Python, you can use the pandas library. Assuming your dataset is stored in a pandas DataFrame, you can use the sample method to randomly select rows. Here’s an example: In this example, n=5 specifies the number of rows to sample, and random_state is set to ensure reproducibility.

Receiver Operating Characteristic (ROC) and Area Under Curve (AUC)

he term “Receiver Operating Characteristic” (ROC) originated in the field of signal detection theory during World War II. Initially, it was used to analyze and measure the performance of radar receivers. The ROC curve, in Machine Learning, is a graphical representation that illustrates the trade-off between true positive rate (sensitivity) and false positive rate (1…

Classification metrics: Accuracy, Precision, Recall, and F1-score

uppose we have a binary classification problem in which we have to predict two classes: 1 and 0. A machine learning model tends to make some mistakes by incorrectly classifying data points, resulting in a difference between the actual and predicted class of the data point. Four possible scenarios that can happen are: Clearly, we want…

Python warnings module

In Python, the warnings module provides a way to handle warnings emitted by the Python interpreter or third-party libraries. When you use import warnings, you can control how warnings are displayed or handle them programmatically. Here are some common use cases:

Difference between R-square and Adjusted R-square

very time you add an independent variable to a model, the R-squared increases, even if the independent variable is insignificant. It never declines. Whereas Adjusted R-squared increases only when independent variable is significant and affects dependent variable. where: Example 1: Calculate Adjusted R-Squared with sklearn Example 2: Calculate Adjusted R-Squared with statsmodels A sample function to…

Forward Feature Selection using SequentialFeatureSelector

SequentialFeatureSelector is a feature selection technique. It is part of the feature_selection module and is used for selecting a subset of features from the original feature set. This technique follows a forward or backward sequential selection strategy. Here’s a brief overview: SequentialFeatureSelector is often used in conjunction with machine learning models to identify the most…

One-Hot Encoding

One-hot encoding is a technique used in machine learning and data preprocessing to represent categorical variables as binary vectors. In one-hot encoding, each category or label in a categorical variable is represented as a binary vector, where each element corresponds to a unique category. The process involves the following steps: For example, consider a dataset…

Linear regression model coefficients

Model coefficients, also known as regression coefficients or weights, are the values assigned to the features (independent variables) in a regression model. In a linear regression model, the relationship between the input features (X) and the predicted output (y) is represented as: Here: The model coefficients are estimated during the training of the regression model.…

What is PolynomialFeatures preprocessing technique?

PolynomialFeatures is a preprocessing technique used in machine learning, particularly in polynomial regression. It transforms an input feature matrix by adding new features that are polynomial combinations of the original features. For example, if you have a feature (x), PolynomialFeatures can generate additional features like , etc., up to a specified degree. This allows the…

What is Uniform Distribution?

uniform distribution is a probability distribution in which all outcomes or events are equally likely to occur. In other words, every possible outcome has the same probability of occurring. In Python, you can use the numpy library to generate random numbers following a uniform distribution. For example:

What is Binomial Distribution?

he binomial distribution is a discrete probability distribution that describes the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success. In other words, it models the number of successes (e.g., heads in a series of coin flips) in a fixed number of independent experiments, where each…

How-to: save a Google Colab notebook as HTML

To save a Google Colab notebook as an HTML file, you can follow these steps: Replace the path and your_notebook_name.ipynb with the actual path and name of your Colab notebook in your Google Drive. Now you have an HTML version of your Google Colab notebook that you can save, share, or submit.

How-to: stack up two plots using the subplot function

You can use the subplot function in Matplotlib to create multiple plots arranged in a grid. To put two plots on top of each other, you can use the following approach: In this example, plt.subplot(2, 1, 1) creates the first subplot in a 2-row, 1-column grid, and plt.subplot(2, 1, 2) creates the second subplot beneath…

What is Gaussian Distribution?

Gaussian distribution, also known as a normal distribution, is a continuous probability distribution that is symmetric around its mean, forming a bell-shaped curve. It is a fundamental concept in statistics and probability theory. The shape of the distribution is characterized by its mean (average) and standard deviation. The probability density function (PDF) of a Gaussian…

How-to: give a specific sorting order to categorical values

In pandas, you can give a specific sorting order to categorical values by creating a categorical variable with an ordered category. Here’s an example: In this example: This can be useful when you want to ensure that certain operations, such as sorting or plotting, take into account the natural order of the days of the…

How-to: cap/clip outliers in a column

To cap or clip outliers in a column, you can use the clip method in pandas. The clip method allows you to set a minimum and maximum threshold for the values in a DataFrame or a specific column. Here’s an example: Clipping is a simple method, and it’s important to consider the impact on your…

How-to: When missing data is of type categorical

hen dealing with missing data of type categorical, several methods can be used to impute the missing values. Here are some common approaches: The choice of imputation method depends on the nature of the data, the underlying patterns, and the goals of the analysis. Always consider the context of the data and the potential impact…

How-to: clean a dataset

leaning a dataset involves handling missing values, correcting errors, and preparing the data for analysis. Here are common steps to clean a dataset using Python and pandas: Always document the steps taken during the cleaning process for transparency and reproducibility. Additionally, it’s crucial to thoroughly understand the context of the data and the goals of…

How-to: formatting options for floating-point numbers in Pandas

n Pandas, pd.set_option(‘display.float_format’, …) is used to set the formatting options for floating-point numbers when they are displayed in the console or output. It allows you to customize how floating-point numbers are presented, including the number of decimal places, scientific notation, and other formatting details. In this example, the pd.set_option(‘display.float_format’, ‘{:,.2f}’.format) line sets the floating-point…

Univariate Analysis in EDA

nivariate exploration refers to the analysis of a single variable in isolation. In data analysis, univariate exploration involves examining the distribution, central tendency, and variability of a single variable without considering its relationship with other variables. Common techniques used in univariate exploration include: Univariate exploration is often the first step in data analysis, providing insights…

What is Plotly Library

lotly is a data visualization library that allows users to create interactive and visually appealing plots and dashboards. It supports a wide range of chart types, including scatter plots, line charts, bar charts, pie charts, 3D plots, geographic maps, and more. Plotly is known for its interactive features, allowing users to explore and interact with…

What is Seaborn Library

eaborn is a data visualization library for Python that is built on top of Matplotlib. It provides a high-level interface for creating attractive and informative statistical graphics. Seaborn is particularly well-suited for visualizing complex datasets with multiple variables. Key features of Seaborn include: To use a library in your Python code, you typically need to…

What is Pandas?

andas is a powerful open-source data manipulation and analysis library for Python. It provides data structures for efficiently storing, manipulating, and analyzing structured data, such as tabular data and time series. Key features of Pandas include: To use Pandas, you typically start by importing it into your Python script or Jupyter Notebook: After importing, you…

How to Save Your Python Objects in Google Colab

In Google Colab, you can use np.save to save NumPy arrays to your Google Drive. Here are the steps: Mount Google Drive Start by mounting your Google Drive. Run the following code and follow the instructions to authorize and mount your Google Drive:

NumPy View array vs. Copy array

hen you create a subset of a NumPy array and modify its values, it can affect the original array if the subset is actually a view of the original array rather than a copy. NumPy provides views to enhance performance and memory efficiency by avoiding unnecessary data copying. Understanding whether you’re working with a view…

What is NumPy?

umPy is a powerful numerical library in Python that provides support for large, multi-dimensional arrays and matrices, along with a collection of mathematical functions to operate on these elements. It is a fundamental package for scientific computing in Python and is widely used in various domains such as data science, machine learning, signal processing, and…

How to create a smaller dataset for a specific month

Suppose we have a file that contains historical weather temperature between years 1948 to 2024 in which we want to look at how January temperatures have changed over time.

Process of Fitting the models in machine learning

The steps to follow to use machine learning models are: In “fit” and “predict” steps, you can use several models, and evaluate them, to keep the most performing one. Python libraries: Here, we train a model to guess a comfortable boot size for a dog, based on the size of the harness that fits them:…

Feature Engineering: Scaling, Normalization, and Standardization

Feature scaling is considered a part of the data processing cycle that cannot be skipped, so that we can achieve stable and fast training of our ML algorithm. eature Scaling is a technique to standardize the independent features present in the data in a fixed range. It is performed during the data pre-processing to handle…

Handling missing data in a dataset

There are many ways to address missing data, each with pros and cons. Let’s take a look at the less complex options: Option 1: Delete data with missing rows. When we have a model that cannot handle missing data, the most prudent thing to do is to remove rows that have information missing. Let’s remove…

Finding missing data in a dataset

Do we have a complete dataset in a real-world scenario? No. We know from history that there is missing information in our data! How can we tell if the data we have available is complete? We could print the entire dataset, but this could involve human error, and it would become impractical with this many…

Improve model with hyperparameters

Simple models with small datasets can often be fit in a single step, while larger datasets and more complex models must be fit by repeatedly using the model with training data and comparing the output with the expected label. If the prediction is accurate enough, we consider the model trained. If not, we adjust the…

Train-and-test isn’t the only approach

It’s worth keeping in mind that train-and-test is common, but not the only widely used approach in machine learning. Two of the more coming alternatives are the hold-out approach and statistical approach methods. hese statistical methods are powerful, well established, and form the foundation of modern science. The advantage is that the training set doesn’t…

What is Overfitting?

A model is overfit if it works better on training data than it does on other data. verfitting can be avoided in several ways. The simplest way is to have a dataset that’s a better representation of what is seen in the real world. A complimentary way we can avoid overfitting is to stop training…

AI & ML Solution Workflow

The workflow for implementing Artificial Intelligence and Machine Learning solutions typically involves several stages. Collaborative efforts among data scientists, domain experts, and stakeholders are crucial throughout the process. The specific details of the workflow can vary based on the complexity of the problem, the type of algorithm used, and the specific requirements of the project.…