Neural network fundamentals

We like simple rules. Neural networks are one of those big ideas in artificial intelligence (AI) that look complicated, but you can get far with just a few rules of thumb. Think of them as stacked building blocks, each block nudging numbers around until the output makes sense.

Neurons

Start with the neuron. It’s just a function that takes numbers in, multiplies them by weights, adds them up, and pushes the result through a gate. The gate is there so the neuron doesn’t just spit out another number linearly. Without that twist, the whole thing would be a dull calculator.

Activation functions

The gate is the activation function. Common ones: sigmoid, ReLU (rectified linear unit), tanh. Sigmoid squashes everything between 0 and 1. ReLU keeps positives, dumps negatives as zero. Tanh sits in the middle, stretching from -1 to 1. Which one you choose changes how fast the network learns and how much subtlety it can capture.

Layers

Neurons don’t work alone. We put them in layers. An input layer takes raw data. Hidden layers pass signals forward, each one reshaping what came before. Finally, the output layer gives us a decision, like “cat” or “dog.” More layers let her capture more complex patterns, but too many can make her forget the basics—overfitting, in coder-speak.

Training

Training is teaching by trial and error. We show her examples with answers. She makes guesses, we measure how wrong she was, and then we push the weights slightly in the right direction. That’s backpropagation: adjust, test, repeat. Over time, the wrongness shrinks. Eventually, she gets good enough to surprise us.

A coder’s thought

We like to think we’re building clever machines. Really, we’re just giving numbers more chances to talk to each other.