Linear Regression That Just Clicks: A Coded Tutorial

Getting Our Hands Dirty: The Setup

Enough theory! Let's write some code. We'll be using Python with a few essential data science libraries. If you don't have them installed, open your terminal or command prompt and run:

pip install numpy pandas scikit-learn matplotlib

• NumPy and Pandas are for data manipulation.
• Scikit-learn is the powerhouse for machine learning models.
• Matplotlib is for visualizing our data and results.

Step-by-Step Code Walkthrough

We'll follow a standard machine learning workflow from start to finish.

1. Importing Libraries

First, let's import everything we'll need for this tutorial.

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn import metrics
import matplotlib.pyplot as plt

2. Creating and Exploring the Data

Instead of a complex dataset, we'll create a simple, intuitive one representing years of experience versus salary. This makes it easy to see the relationship.

# Create a dictionary of data
data = {
    'YearsExperience': [1.1, 1.3, 1.5, 2.0, 2.2, 2.9, 3.0, 3.2, 3.2, 3.7, 3.9, 4.0, 4.0, 4.1, 4.5, 4.9, 5.1, 5.3, 5.9, 6.0],
    'Salary': [39343, 46205, 37731, 43525, 39891, 56642, 60150, 54445, 64445, 57189, 63218, 55794, 56957, 57081, 61111, 67938, 66029, 83088, 81363, 93940]
}

# Create a pandas DataFrame
df = pd.DataFrame(data)

# Define our features (X) and target (y)
X = df[['YearsExperience']] # Independent variable
y = df['Salary']          # Dependent variable

# Let's visualize our data points
plt.figure(figsize=(8, 5))
plt.scatter(X, y)
plt.title('Salary vs. Years of Experience')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.grid(True)
plt.show()

Running this code will show you a scatter plot. You can clearly see the positive linear relationship we talked about. Now, let's draw the line!

3. Splitting Data for Training and Testing

A crucial step in machine learning is to evaluate your model on data it has never seen before. To do this, we split our dataset into two parts:

• Training Set: The majority of the data, used to teach (or "fit") our model.
• Testing Set: A smaller portion of the data, held back to test how well our trained model performs.

Think of it like studying for an exam. You use the textbook chapters (training data) to learn, and then you take a practice test (testing data) to see how well you've learned the material.

# Split data into 80% training and 20% testing
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

4. Building and Training the Model

This is where the magic happens, and with Scikit-learn, it's incredibly straightforward.

# Create an instance of the Linear Regression model
model = LinearRegression()

# Train the model on our training data
model.fit(X_train, y_train)

print("Model training complete!")

That's it! The .fit() method is what performs the Ordinary Least Squares calculation to find the best-fit line. It has now calculated the optimal intercept (b) and slope (m) for our data.

Let's see what they are:

# The intercept (b or β₀)
print(f"Intercept: {model.intercept_}")

# The slope (m or β₁)
print(f"Coefficient: {model.coef_[0]}")

This will output something like:
Intercept: 25202.8
Coefficient: 9731.2

This means our model's equation is: Salary = 9731.2 * (Years of Experience) + 25202.8. The model predicts a starting salary of about $25k and an increase of ~$9.7k for each additional year of experience.

5. Making Predictions

Now that our model is trained, we can use it to make predictions on our test data (the data it hasn't seen yet).

# Make predictions on the test data
y_pred = model.predict(X_test)

# Compare actual vs. predicted values
comparison_df = pd.DataFrame({'Actual': y_test, 'Predicted': y_pred})
print(comparison_df)

You'll see a small table comparing the actual salaries from our test set with the salaries our model predicted. They won't be perfect, but they should be pretty close!

How Do We Know if Our Model is Any Good?

Just looking at the predictions isn't enough. We need metrics to quantify the model's performance. Here are three common ones:

Let's calculate them in Python:

print('Mean Absolute Error (MAE):', metrics.mean_absolute_error(y_test, y_pred))
print('Mean Squared Error (MSE):', metrics.mean_squared_error(y_test, y_pred))
print('R-squared (R²):', metrics.r2_score(y_test, y_pred))

An R-squared value of, say, 0.90 would mean that 90% of the variation in salaries can be explained by years of experience, according to our model. That's pretty good!

The best way to see the result is to visualize it. Let's plot our regression line on top of our test data points.

# Plot the test data
plt.figure(figsize=(8, 5))
plt.scatter(X_test, y_test, color='blue', label='Actual Data')
# Plot the regression line
plt.plot(X_test, y_pred, color='red', linewidth=2, label='Regression Line')
plt.title('Model Performance on Test Data')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.legend()
plt.grid(True)
plt.show()

Seeing that red line slice through the blue dots is the moment it all clicks. You've successfully modeled the relationship!

Putting It All Together: The Complete Code

Here is the full script from start to finish for easy reference.

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn import metrics
import matplotlib.pyplot as plt

# 1. Prepare Data
data = {
    'YearsExperience': [1.1, 1.3, 1.5, 2.0, 2.2, 2.9, 3.0, 3.2, 3.2, 3.7, 3.9, 4.0, 4.0, 4.1, 4.5, 4.9, 5.1, 5.3, 5.9, 6.0],
    'Salary': [39343, 46205, 37731, 43525, 39891, 56642, 60150, 54445, 64445, 57189, 63218, 55794, 56957, 57081, 61111, 67938, 66029, 83088, 81363, 93940]
}
df = pd.DataFrame(data)
X = df[['YearsExperience']]
y = df['Salary']

# 2. Split Data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# 3. Train Model
model = LinearRegression()
model.fit(X_train, y_train)

# 4. Make Predictions
y_pred = model.predict(X_test)

# 5. Evaluate Model
print(f"Intercept: {model.intercept_}")
print(f"Coefficient: {model.coef_[0]}")
print('R-squared (R²):', metrics.r2_score(y_test, y_pred))

# 6. Visualize Results
plt.figure(figsize=(8, 5))
plt.scatter(X_test, y_test, color='blue', label='Actual Data')
plt.plot(X_test, y_pred, color='red', linewidth=2, label='Regression Line')
plt.title('Salary vs. Experience (Test Set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.legend()
plt.grid(True)
plt.show()

Conclusion: Your Next Steps in Prediction

Congratulations! You've successfully navigated the world of linear regression, from the core intuition of the "best-fit line" to implementing and evaluating a model in Python. You've seen that it's not black magic, but a logical process of finding a mathematical relationship in your data.

Linear regression is a fundamental building block. From here, you can explore more advanced topics like Multiple Linear Regression (using several independent variables to predict a single dependent one) or Polynomial Regression (for modeling non-linear relationships). The principles you learned today—training, testing, and evaluation—will apply to almost every machine learning model you encounter.

So, find a new dataset, fire up your code editor, and start making predictions. The journey has just begun!