ML Regression in Python
Visualize regression in scikitlearn with Plotly.
Write, deploy, & scale Dash apps and Python data visualizations on a Kubernetes Dash Enterprise cluster.
Get Pricing

Demo Dash Enterprise

Dash Enterprise Overview
New to Plotly?
Plotly is a free and opensource graphing library for Python. We recommend you read our Getting Started guide for the latest installation or upgrade instructions, then move on to our Plotly Fundamentals tutorials or dive straight in to some Basic Charts tutorials.
This page shows how to use Plotly charts for displaying various types of regression models, starting from simple models like Linear Regression, and progressively move towards models like Decision Tree and Polynomial Features. We highlight various capabilities of plotly, such as comparative analysis of the same model with different parameters, displaying Latex, surface plots for 3D data, and enhanced prediction error analysis with Plotly Express.
We will use Scikitlearn to split and preprocess our data and train various regression models. Scikitlearn is a popular Machine Learning (ML) library that offers various tools for creating and training ML algorithms, feature engineering, data cleaning, and evaluating and testing models. It was designed to be accessible, and to work seamlessly with popular libraries like NumPy and Pandas.
Basic linear regression plots¶
In this section, we show you how to apply a simple regression model for predicting tips a server will receive based on various client attributes (such as sex, time of the week, and whether they are a smoker).
We will be using the Linear Regression, which is a simple model that fit an intercept (the mean tip received by a server), and add a slope for each feature we use, such as the value of the total bill. We show you how to do that with both Plotly Express and Scikitlearn.
Ordinary Least Square (OLS) with plotly.express
¶
This example shows how to use plotly.express
's trendline
parameter to train a simply Ordinary Least Square (OLS) for predicting the tips waiters will receive based on the value of the total bill.
import plotly.express as px
df = px.data.tips()
fig = px.scatter(
df, x='total_bill', y='tip', opacity=0.65,
trendline='ols', trendline_color_override='darkblue'
)
fig.show()
Linear Regression with scikitlearn¶
You can also perform the same prediction using scikitlearn's LinearRegression
.
import numpy as np
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LinearRegression
df = px.data.tips()
X = df.total_bill.values.reshape(1, 1)
model = LinearRegression()
model.fit(X, df.tip)
x_range = np.linspace(X.min(), X.max(), 100)
y_range = model.predict(x_range.reshape(1, 1))
fig = px.scatter(df, x='total_bill', y='tip', opacity=0.65)
fig.add_traces(go.Scatter(x=x_range, y=y_range, name='Regression Fit'))
fig.show()
ML Regression in Dash¶
Dash is the best way to build analytical apps in Python using Plotly figures. To run the app below, run pip install dash
, click "Download" to get the code and run python app.py
.
Get started with the official Dash docs and learn how to effortlessly style & deploy apps like this with Dash Enterprise.
Model generalization on unseen data¶
With go.Scatter
, you can easily color your plot based on a predefined data split. By coloring the training and the testing data points with different colors, you can easily see if whether the model generalizes well to the test data or not.
import numpy as np
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
df = px.data.tips()
X = df.total_bill[:, None]
X_train, X_test, y_train, y_test = train_test_split(X, df.tip, random_state=0)
model = LinearRegression()
model.fit(X_train, y_train)
x_range = np.linspace(X.min(), X.max(), 100)
y_range = model.predict(x_range.reshape(1, 1))
fig = go.Figure([
go.Scatter(x=X_train.squeeze(), y=y_train, name='train', mode='markers'),
go.Scatter(x=X_test.squeeze(), y=y_test, name='test', mode='markers'),
go.Scatter(x=x_range, y=y_range, name='prediction')
])
fig.show()
Comparing different kNN models parameters¶
In addition to linear regression, it's possible to fit the same data using kNearest Neighbors. When you perform a prediction on a new sample, this model either takes the weighted or unweighted average of the neighbors. In order to see the difference between those two averaging options, we train a kNN model with both of those parameters, and we plot them in the same way as the previous graph.
Notice how we can combine scatter points with lines using Plotly.py. You can learn more about multiple chart types.
import numpy as np
import plotly.express as px
import plotly.graph_objects as go
from sklearn.neighbors import KNeighborsRegressor
df = px.data.tips()
X = df.total_bill.values.reshape(1, 1)
x_range = np.linspace(X.min(), X.max(), 100)
# Model #1
knn_dist = KNeighborsRegressor(10, weights='distance')
knn_dist.fit(X, df.tip)
y_dist = knn_dist.predict(x_range.reshape(1, 1))
# Model #2
knn_uni = KNeighborsRegressor(10, weights='uniform')
knn_uni.fit(X, df.tip)
y_uni = knn_uni.predict(x_range.reshape(1, 1))
fig = px.scatter(df, x='total_bill', y='tip', color='sex', opacity=0.65)
fig.add_traces(go.Scatter(x=x_range, y=y_uni, name='Weights: Uniform'))
fig.add_traces(go.Scatter(x=x_range, y=y_dist, name='Weights: Distance'))
fig.show()
Displaying PolynomialFeatures
using $\LaTeX$¶
Notice how linear regression fits a straight line, but kNN can take nonlinear shapes. Moreover, it is possible to extend linear regression to polynomial regression by using scikitlearn's PolynomialFeatures
, which lets you fit a slope for your features raised to the power of n
, where n=1,2,3,4
in our example.
With Plotly, it's easy to display latex equations in legend and titles by simply adding $
before and after your equation. This way, you can see the coefficients that our polynomial regression fitted.
import numpy as np
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
def format_coefs(coefs):
equation_list = [f"{coef}x^{i}" for i, coef in enumerate(coefs)]
equation = "$" + " + ".join(equation_list) + "$"
replace_map = {"x^0": "", "x^1": "x", '+ ': ' '}
for old, new in replace_map.items():
equation = equation.replace(old, new)
return equation
df = px.data.tips()
X = df.total_bill.values.reshape(1, 1)
x_range = np.linspace(X.min(), X.max(), 100).reshape(1, 1)
fig = px.scatter(df, x='total_bill', y='tip', opacity=0.65)
for degree in [1, 2, 3, 4]:
poly = PolynomialFeatures(degree)
poly.fit(X)
X_poly = poly.transform(X)
x_range_poly = poly.transform(x_range)
model = LinearRegression(fit_intercept=False)
model.fit(X_poly, df.tip)
y_poly = model.predict(x_range_poly)
equation = format_coefs(model.coef_.round(2))
fig.add_traces(go.Scatter(x=x_range.squeeze(), y=y_poly, name=equation))
fig.show()
3D regression surface with px.scatter_3d
and go.Surface
¶
Visualize the decision plane of your model whenever you have more than one variable in your input data. Here, we will use sklearn.svm.SVR
, which is a Support Vector Machine (SVM) model specifically designed for regression.
import numpy as np
import plotly.express as px
import plotly.graph_objects as go
from sklearn.svm import SVR
mesh_size = .02
margin = 0
df = px.data.iris()
X = df[['sepal_width', 'sepal_length']]
y = df['petal_width']
# Condition the model on sepal width and length, predict the petal width
model = SVR(C=1.)
model.fit(X, y)
# Create a mesh grid on which we will run our model
x_min, x_max = X.sepal_width.min()  margin, X.sepal_width.max() + margin
y_min, y_max = X.sepal_length.min()  margin, X.sepal_length.max() + margin
xrange = np.arange(x_min, x_max, mesh_size)
yrange = np.arange(y_min, y_max, mesh_size)
xx, yy = np.meshgrid(xrange, yrange)
# Run model
pred = model.predict(np.c_[xx.ravel(), yy.ravel()])
pred = pred.reshape(xx.shape)
# Generate the plot
fig = px.scatter_3d(df, x='sepal_width', y='sepal_length', z='petal_width')
fig.update_traces(marker=dict(size=5))
fig.add_traces(go.Surface(x=xrange, y=yrange, z=pred, name='pred_surface'))
fig.show()
Visualizing coefficients for multiple linear regression (MLR)¶
Visualizing regression with one or two variables is straightforward, since we can respectively plot them with scatter plots and 3D scatter plots. Moreover, if you have more than 2 features, you will need to find alternative ways to visualize your data.
One way is to use bar charts. In our example, each bar indicates the coefficients of our linear regression model for each input feature. Our model was trained on the Iris dataset.
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LinearRegression
df = px.data.iris()
X = df.drop(columns=['petal_width', 'species_id'])
X = pd.get_dummies(X, columns=['species'], prefix_sep='=')
y = df['petal_width']
model = LinearRegression()
model.fit(X, y)
colors = ['Positive' if c > 0 else 'Negative' for c in model.coef_]
fig = px.bar(
x=X.columns, y=model.coef_, color=colors,
color_discrete_sequence=['red', 'blue'],
labels=dict(x='Feature', y='Linear coefficient'),
title='Weight of each feature for predicting petal width'
)
fig.show()
Prediction Error Plots¶
When you are working with very highdimensional data, it is inconvenient to plot every dimension with your output y
. Instead, you can use methods such as prediction error plots, which let you visualize how well your model does compared to the ground truth.
Simple actual vs predicted plot¶
This example shows you the simplest way to compare the predicted output vs. the actual output. A good model will have most of the scatter dots near the diagonal black line.
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LinearRegression
df = px.data.iris()
X = df[['sepal_width', 'sepal_length']]
y = df['petal_width']
# Condition the model on sepal width and length, predict the petal width
model = LinearRegression()
model.fit(X, y)
y_pred = model.predict(X)
fig = px.scatter(x=y, y=y_pred, labels={'x': 'ground truth', 'y': 'prediction'})
fig.add_shape(
type="line", line=dict(dash='dash'),
x0=y.min(), y0=y.min(),
x1=y.max(), y1=y.max()
)
fig.show()
Enhanced prediction error analysis using plotly.express
¶
Add marginal histograms to quickly diagnoses any prediction bias your model might have. The builtin OLS
functionality let you visualize how well your model generalizes by comparing it with the theoretical optimal fit (black dotted line).
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
df = px.data.iris()
# Split data into training and test splits
train_idx, test_idx = train_test_split(df.index, test_size=.25, random_state=0)
df['split'] = 'train'
df.loc[test_idx, 'split'] = 'test'
X = df[['sepal_width', 'sepal_length']]
y = df['petal_width']
X_train = df.loc[train_idx, ['sepal_width', 'sepal_length']]
y_train = df.loc[train_idx, 'petal_width']
# Condition the model on sepal width and length, predict the petal width
model = LinearRegression()
model.fit(X_train, y_train)
df['prediction'] = model.predict(X)
fig = px.scatter(
df, x='petal_width', y='prediction',
marginal_x='histogram', marginal_y='histogram',
color='split', trendline='ols'
)
fig.update_traces(histnorm='probability', selector={'type':'histogram'})
fig.add_shape(
type="line", line=dict(dash='dash'),
x0=y.min(), y0=y.min(),
x1=y.max(), y1=y.max()
)
fig.show()
Residual plots¶
Just like prediction error plots, it's easy to visualize your prediction residuals in just a few lines of codes using plotly.express
builtin capabilities.
import numpy as np
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
df = px.data.iris()
# Split data into training and test splits
train_idx, test_idx = train_test_split(df.index, test_size=.25, random_state=0)
df['split'] = 'train'
df.loc[test_idx, 'split'] = 'test'
X = df[['sepal_width', 'sepal_length']]
X_train = df.loc[train_idx, ['sepal_width', 'sepal_length']]
y_train = df.loc[train_idx, 'petal_width']
# Condition the model on sepal width and length, predict the petal width
model = LinearRegression()
model.fit(X_train, y_train)
df['prediction'] = model.predict(X)
df['residual'] = df['prediction']  df['petal_width']
fig = px.scatter(
df, x='prediction', y='residual',
marginal_y='violin',
color='split', trendline='ols'
)
fig.show()
Visualize regularization across crossvalidation folds¶
In this example, we show how to plot the results of various $\alpha$ penalization values from the results of crossvalidation using scikitlearn's LassoCV
. This is useful to see how much the error of the optimal alpha actually varies across CV folds.
import numpy as np
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
from sklearn.linear_model import LassoCV
N_FOLD = 6
# Load and preprocess the data
df = px.data.gapminder()
X = df.drop(columns=['lifeExp', 'iso_num'])
X = pd.get_dummies(X, columns=['country', 'continent', 'iso_alpha'])
y = df['lifeExp']
# Train model to predict life expectancy
model = LassoCV(cv=N_FOLD, normalize=True)
model.fit(X, y)
mean_alphas = model.mse_path_.mean(axis=1)
fig = go.Figure([
go.Scatter(
x=model.alphas_, y=model.mse_path_[:, i],
name=f"Fold: {i+1}", opacity=.5, line=dict(dash='dash'),
hovertemplate="alpha: %{x} <br>MSE: %{y}"
)
for i in range(N_FOLD)
])
fig.add_traces(go.Scatter(
x=model.alphas_, y=mean_alphas,
name='Mean', line=dict(color='black', width=3),
hovertemplate="alpha: %{x} <br>MSE: %{y}",
))
fig.add_shape(
type="line", line=dict(dash='dash'),
x0=model.alpha_, y0=0,
x1=model.alpha_, y1=1,
yref='paper'
)
fig.update_layout(
xaxis_title='alpha',
xaxis_type="log",
yaxis_title="Mean Square Error (MSE)"
)
fig.show()
Grid search visualization using px.density_heatmap
and px.box
¶
In this example, we show how to visualize the results of a grid search on a DecisionTreeRegressor
. The first plot shows how to visualize the score of each model parameter on individual splits (grouped using facets). The second plot aggregates the results of all splits such that each box represents a single model.
import numpy as np
import pandas as pd
import plotly.express as px
import plotly.graph_objects as go
from sklearn.model_selection import GridSearchCV
from sklearn.tree import DecisionTreeRegressor
N_FOLD = 6
# Load and shuffle dataframe
df = px.data.iris()
df = df.sample(frac=1, random_state=0)
X = df[['sepal_width', 'sepal_length']]
y = df['petal_width']
# Define and fit the grid
model = DecisionTreeRegressor()
param_grid = {
'criterion': ['mse', 'friedman_mse', 'mae'],
'max_depth': range(2, 5)
}
grid = GridSearchCV(model, param_grid, cv=N_FOLD)
grid.fit(X, y)
grid_df = pd.DataFrame(grid.cv_results_)
# Convert the wide format of the grid into the long format
# accepted by plotly.express
melted = (
grid_df
.rename(columns=lambda col: col.replace('param_', ''))
.melt(
value_vars=[f'split{i}_test_score' for i in range(N_FOLD)],
id_vars=['mean_test_score', 'mean_fit_time', 'criterion', 'max_depth'],
var_name="cv_split",
value_name="r_squared"
)
)
# Format the variable names for simplicity
melted['cv_split'] = (
melted['cv_split']
.str.replace('_test_score', '')
.str.replace('split', '')
)
# Single function call to plot each figure
fig_hmap = px.density_heatmap(
melted, x="max_depth", y='criterion',
histfunc="sum", z="r_squared",
title='Grid search results on individual fold',
hover_data=['mean_fit_time'],
facet_col="cv_split", facet_col_wrap=3,
labels={'mean_test_score': "mean_r_squared"}
)
fig_box = px.box(
melted, x='max_depth', y='r_squared',
title='Grid search results ',
hover_data=['mean_fit_time'],
points='all',
color="criterion",
hover_name='cv_split',
labels={'mean_test_score': "mean_r_squared"}
)
# Display
fig_hmap.show()
fig_box.show()
Reference¶
Learn more about the px
figures used in this tutorial:
 Plotly Express: https://plot.ly/python/plotlyexpress/
 Vertical Lines: https://plot.ly/python/shapes/
 Heatmaps: https://plot.ly/python/heatmaps/
 Box Plots: https://plot.ly/python/boxplots/
 3D Scatter: https://plot.ly/python/3dscatterplots/
 Surface Plots: https://plot.ly/python/3dsurfaceplots/
Learn more about the Machine Learning models used in this tutorial:
 https://scikitlearn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html
 https://scikitlearn.org/stable/modules/generated/sklearn.linear_model.LassoCV.html
 https://scikitlearn.org/stable/modules/generated/sklearn.neighbors.KNeighborsRegressor.html
 https://scikitlearn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html
 https://scikitlearn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html
Other tutorials that inspired this notebook:
What About Dash?¶
Dash is an opensource framework for building analytical applications, with no Javascript required, and it is tightly integrated with the Plotly graphing library.
Learn about how to install Dash at https://dash.plot.ly/installation.
Everywhere in this page that you see fig.show()
, you can display the same figure in a Dash application by passing it to the figure
argument of the Graph
component from the builtin dash_core_components
package like this:
import plotly.graph_objects as go # or plotly.express as px
fig = go.Figure() # or any Plotly Express function e.g. px.bar(...)
# fig.add_trace( ... )
# fig.update_layout( ... )
import dash
import dash_core_components as dcc
import dash_html_components as html
app = dash.Dash()
app.layout = html.Div([
dcc.Graph(figure=fig)
])
app.run_server(debug=True, use_reloader=False) # Turn off reloader if inside Jupyter