← All posts

From School Math to Neural Networks: A Simple Connection

I've been revisiting some basic machine learning concepts lately, and one thing I found fascinating is how a simple equation from school mathematics connects to modern AI.

Remember this equation?

y = mx + c

It's the equation of a straight line. Surprisingly, a very similar mathematical idea appears throughout neural networks, including the large language models (LLMs) we use today.

Let's understand this through two simple concepts: Linear Regression and Logistic Regression.

1. Linear Regression: Predicting a Number

Imagine we're trying to predict the price of a house based on its size.

Generally, a bigger house costs more. If we collect enough examples of house sizes and their prices, we can try to draw a straight line that represents the relationship.

That's essentially Linear Regression.

In machine learning, we often write the equation as:

ŷ = wx + b

Where:

  • x = Input (house size)
  • w = Weight (how much the size influences the price)
  • b = Bias (a starting value)
  • ŷ = Predicted house price

For example, let's assume:

  • House size = 1,000 sq. ft.
  • Weight = $200 per sq. ft.
  • Bias = $50,000

Our prediction becomes:

ŷ = (200 × 1000) + 50,000 = $250,000

Of course, real housing prices depend on many more factors. But the interesting part is that the machine doesn't initially know the correct weight and bias.

It learns them from training examples by repeatedly making predictions, measuring errors, and adjusting its parameters.

That's a very basic form of machine learning!

2. Logistic Regression: Predicting a Probability

Now imagine a different problem.

Instead of predicting a house price, we want our machine to recognize whether an image contains a dog.

Here, we aren't looking for a price or another continuous number. We want a Yes or No answer.

That's where Logistic Regression comes in.

It starts with the same familiar calculation:

z = wx + b

But adds something interesting called the Sigmoid function.

p = 1 / (1 + e−z)

Don't worry about the equation. The sigmoid function simply converts any real number into a value between 0 and 1.

For example:

  • 0.05 → Very unlikely to be a dog
  • 0.45 → Not quite sure
  • 0.90 → Very likely to be a dog

We can then use a threshold, say 0.5, to classify the image as Dog or Not Dog.

So the main difference is simple:

Linear Regression predicts a number. Logistic Regression predicts a probability for a binary classification.

Linear Regression Logistic Regression House size↓wx + b↓$250,000Predict a number Image features↓wx + b → Sigmoid↓90% DogPredict a probability ↓Neural NetworksMany learned weighted transformations, activation functions, and layers
Linear and logistic regression both use a weighted input plus a bias. Neural networks stack many versions of that basic transformation.

3. How Does This Connect to Neural Networks?

This is where things get interesting.

An artificial neuron uses a very similar mathematical operation.

It takes inputs, multiplies them by weights, adds a bias, and usually passes the result through an activation function.

In simplified form:

Output = Activation(wx + b)

Sounds familiar, doesn't it?

A neuron with a sigmoid activation can behave like a logistic regression classifier.

Now imagine having hundreds or thousands of neurons connected across multiple layers. Each layer learns different patterns from the data.

That's the basic idea behind a neural network.

Instead of working with a single input, these models work with many inputs simultaneously, using vectors, matrices, and tensors.

4. And What About LLMs?

Modern LLMs like GPT and Qwen are far more sophisticated than basic regression models. They use Transformer architectures, attention mechanisms, and enormous numbers of learned parameters.

But they still rely heavily on the mathematical idea of multiplying inputs by learned weights.

When an LLM predicts the next token, its final representation is converted into scores called logits. These scores can then be transformed into probabilities using a function called Softmax.

If you'd like a more detailed walkthrough of logits and Softmax, I've explained them step by step in my GitHub repository: LLM Basics for Beginners.

The underlying mathematics is much more complex, but the connection to basic machine learning is still there.

My Takeaway

The more I learn about AI, the more I realize how valuable it is to revisit the fundamentals.

A simple equation like y = mx + c leads us to linear regression, logistic regression, weights, biases, activation functions, and eventually neural networks.

Of course, understanding that equation alone doesn't explain an LLM. But it gives us a useful starting point.

Sometimes, understanding modern AI begins with revisiting the mathematics we learned in school.