In this blog post, we will explore the concept of linear regression and how it can be implemented using PyTorch. Regression is a fundamental statistical modeling technique used to establish a relationship between a dependent variable and one or more independent variables. We will use PyTorch, a popular deep learning framework, to develop and train a linear regression model.you can find more details about linear concept here .
Dataset:
For this analysis, we will use a synthetic dataset generated using the make regression() function from the scikit-learn library. The dataset consists of input features and a target variable. The input features represent independent variables, while the target variable represents the dependent variable we aim to predict.
import seaborn as sns
import numpy as sns
import torch
import torch.nn as nn
import torch.optim as optim
import sklearn
from sklearn import datasets
import pandas as pd
# from sklearn we are going to select one dataset
data=datasets.make_regression()
df = pd.DataFrame(data[0], columns=[f"feature_{i+1}" for i in range(data[0].shape[1])])
df["target"] = data[1]PyTorch Basics:
PyTorch is a powerful open-source deep learning framework that provides a flexible way to build and train neural networks. It offers a range of functionality for tensor operations, automatic differentiation, and optimization algorithms. PyTorch’s dynamic computational graph makes it easy to define and modify complex neural network architectures.Learn Torch basics here
Make data ready for developing a model using sklearn Train-Test-split
x=df.iloc[: , :-1]
y=df.iloc[: , -1]
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(x,y,test_size=0.2,random_state=42)
print(type(X_train))
# X_train=torch.tensor(X_train,dtype=torch.float32)
X_train = torch.tensor(X_train.values, dtype=torch.float32)
X_test = torch.tensor(X_test.values, dtype=torch.float32)
y_train = torch.tensor(y_train.values, dtype=torch.float32)
y_test = torch.tensor(y_test.values, dtype=torch.float32)Model Architecture:
Our linear regression model is implemented as a subclass of thenn.Module class in PyTorch. The model consists of multiple fully connected (linear) layers connected in a sequential manner. These layers are responsible for transforming the input features into predicted output values. The model architecture includes hidden layers with various sizes and an output layer with a single neuron.
# since data is ready we can develop the model:
class linearRegression(nn.Module): # all the dependencies from torch will be given to this class [parent class] # nn.Module contains all the building block of neural networks:
def __init__(self,input_dim):
super(linearRegression,self).__init__() # building connection with parent and child classes
self.fc1=nn.Linear(input_dim,10) # hidden layer 1
self.fc2=nn.Linear(10,5) # hidden layer 2
self.fc3=nn.Linear(5,3) # hidden layer 3
self.fc4=nn.Linear(3,1) # last layer
def forward(self,d):
out=torch.relu(self.fc1(d)) # input * weights + bias for layer 1
out=torch.relu(self.fc2(out)) # input * weights + bias for layer 2
out=torch.relu(self.fc3(out)) # input * weights + bias for layer 3
out=self.fc4(out) # input * weights + bias for last layer
return out # final outcome
input_dim=X_train.shape[1]
torch.manual_seed(42) # to make initilized weights stable:
model=linearRegression(input_dim)Training Process:
To train the model, we use the Mean Squared Error (MSE) loss function, which measures the average squared difference between the predicted values and the actual target values. The optimization is performed using the Adam optimizer, which adjusts the model’s parameters based on the computed gradients. The model is trained over a specified number of epochs, where each epoch involves forward propagation, loss calculation, back propagation, and weight updates.
# select loss and optimizers
loss=nn.MSELoss() # loss function
optimizers=optim.Adam(params=model.parameters(),lr=0.01)
# training the model:
num_of_epochs=1000
for i in range(num_of_epochs):
# give the input data to the architecure
y_train_prediction=model(X_train) # model initilizing
loss_value=loss(y_train_prediction.squeeze(),y_train) # find the loss function:
optimizers.zero_grad() # make gradients zero for every iteration so next iteration it will be clear
loss_value.backward() # back propagation
optimizers.step() # update weights in NN
# print the loss in training part:
if i % 10 == 0:
print(f'[epoch:{i}]: The loss value for training part={loss_value}')Evaluation and Performance:
During training, we monitor the loss value to assess the model’s performance. We split the dataset into training and testing sets to evaluate the model’s generalization capability. The trained model is evaluated using the test dataset, and the test loss is calculated. Lower test loss indicates better performance.
# we can do check it with test data:
with torch.no_grad():
model.eval() # make model in evaluation stage
y_test_prediction=model(X_test)
test_loss=loss(y_test_prediction.squeeze(),y_test)
print(f'Test loss value : {test_loss.item():.4f}')# Inference with own data:
pr = torch.tensor(torch.arange(1, 101).unsqueeze(dim=0), dtype=torch.float32).clone().detach()
print(pr)
Saving and Loading the Model:
After training, we save the trained model’s parameters using PyTorch’s save() function. This allows us to reuse the model in the future without retraining. We can load the saved model using the load_state_dict() method and make predictions on new data.
# save the torch model:
from pathlib import Path
filename=Path('models')
filename.mkdir(parents=True,exist_ok=True)
model_name='linear_regression.pth' # model name
# saving path
saving_path=filename/model_name
print(saving_path)
torch.save(obj=model.state_dict(),f=saving_path)
# we can load the saved model and do the inference again:
load_model=linearRegression(input_dim) # creating an instance again for loaded model
load_model.load_state_dict(torch.load('/content/models/linear_regression.pth'))
load_model.eval() # make model in evaluation stage
with torch.no_grad():
pred = load_model(torch.tensor([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12.,
13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.,
25., 26., 27., 28., 29., 30., 31., 32., 33., 34., 35., 36.,
37., 38., 39., 40., 41., 42., 43., 44., 45., 46., 47., 48.,
49., 50., 51., 52., 53., 54., 55., 56., 57., 58., 59., 60.,
61., 62., 63., 64., 65., 66., 67., 68., 69., 70., 71., 72.,
73., 74., 75., 76., 77., 78., 79., 80., 81., 82., 83., 84.,
85., 86., 87., 88., 89., 90., 91., 92., 93., 94., 95., 96.,
97., 98., 99., 100.]]))
print(f'prediction value : {pred.item()}')Conclusion:
In this blog post, we explored linear regression analysis using PyTorch. We learned how to prepare a dataset, construct a linear regression model, train the model using gradient descent optimization, and evaluate its performance. PyTorch’s flexibility and extensive functionality make it a powerful tool for developing and training various machine learning models.
Stay up to date with the latest advancements in Machine Learning and Artificial Intelligence by following my GitHub and Medium pages.
