Lesson 4 Cheat Sheet

🧠 Neural Network Basics

Architecture Components

INPUT → [WEIGHTS] → HIDDEN → [WEIGHTS] → OUTPUT
  (4)      (W1)       (10)       (W2)        (3)

Input Layer: Receives raw data (size = # features)

Hidden Layer: Extracts patterns (size = flexible)

Output Layer: Makes predictions (size = # classes)

🔢 Key Formulas

Weighted Sum

z = (x₁ × w₁) + (x₂ × w₂) + ... + (xₙ × wₙ) + bias

Matrix Form

z = X · W + b

Activation Functions

ReLU (Hidden Layers):

f(x) = max(0, x)

Sigmoid:

f(x) = 1 / (1 + e^(-x))

Softmax (Output Layer):

f(xᵢ) = e^(xᵢ) / Σ(e^(xⱼ))

Loss Function

Loss = -Σ(y_true × log(y_pred))

Gradient Descent

weight_new = weight_old - learning_rate × gradient

💻 Essential Code Snippets

Load Data

from sklearn.datasets import load_iris
iris = load_iris()
X, y = iris.data, iris.target

Split Data

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

Scale Features

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

One-Hot Encode Labels

from sklearn.preprocessing import OneHotEncoder
encoder = OneHotEncoder(sparse_output=False)
y_train_enc = encoder.fit_transform(y_train.reshape(-1, 1))

Initialize Weights

import numpy as np
W1 = np.random.randn(input_size, hidden_size) * 0.01
b1 = np.zeros((1, hidden_size))

Forward Pass

z1 = np.dot(X, W1) + b1
a1 = relu(z1)
z2 = np.dot(a1, W2) + b2
a2 = softmax(z2)

Calculate Loss

loss = -np.sum(y_true * np.log(a2 + 1e-8)) / m

Update Weights

W1 -= learning_rate * dW1
b1 -= learning_rate * db1

📊 Training Process

The Training Loop

1. Forward Propagation: Input → Hidden → Output
2. Calculate Loss: Compare prediction to actual
3. Backward Propagation: Compute gradients
4. Update Weights: Apply gradient descent
5. Repeat for all epochs

Checking Progress

# During training, monitor:
print(f"Epoch {epoch}: Loss={loss:.4f}, Acc={acc:.4f}")

# Plot after training:
plt.plot(losses)
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.show()

🎛️ Hyperparameters Guide

Common Values

Parameter	Typical Range	Good Starting Point
Learning Rate	0.001 - 0.1	0.01 or 0.1
Hidden Neurons	10 - 100	10-20 for Iris
Epochs	100 - 2000	500-1000
Batch Size	16 - 128	32

Tuning Tips

Learning Rate:

• Too high → Unstable, doesn't converge
• Too low → Very slow learning
• Just right → Steady decrease in loss

Hidden Neurons:

• Too few → Underfitting (can't learn patterns)
• Too many → Overfitting (memorizes data)
• Just right → Generalizes well

Epochs:

• Too few → Not fully trained
• Too many → Overfitting, wasted time
• Just right → Loss plateaus, good test accuracy

🐛 Common Errors & Fixes

"Shape mismatch"

# Check dimensions
print(X.shape, W.shape)
# Ensure: (samples, features) × (features, neurons)

"Loss is NaN"

# Clip values to prevent overflow
np.clip(x, -500, 500)
# Or reduce learning rate

"All predictions same class"

# Check weights initialized (not zeros)
# Verify learning rate not too small
# Ensure sufficient training epochs

"Accuracy not improving"

# Try different learning rate
# Add more hidden neurons
# Train for more epochs
# Check data is normalized

📈 Evaluation Metrics

Accuracy

from sklearn.metrics import accuracy_score
acc = accuracy_score(y_true, y_pred)

Confusion Matrix

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_true, y_pred)

Visual Check

# Plot predictions vs actual
plt.scatter(range(len(y_test)), y_test, label='Actual')
plt.scatter(range(len(y_pred)), y_pred, label='Predicted')
plt.legend()

🔍 Debugging Checklist

Before asking for help, check:

• [ ] All libraries imported
• [ ] Data loaded correctly
• [ ] Train/test split performed
• [ ] Features normalized
• [ ] Labels encoded
• [ ] Network architecture correct
• [ ] Forward pass implemented
• [ ] Loss function correct
• [ ] Backpropagation or optimizer used
• [ ] Weights being updated
• [ ] Training for enough epochs
• [ ] No syntax errors

💡 Quick Tips

Best Practices

✓ DO:

• Normalize your data
• Start with simple architecture
• Plot training curves
• Test different hyperparameters
• Document your experiments
• Use proper variable names
• Add comments to code

✗ DON'T:

• Train on unnormalized data
• Start with huge networks
• Ignore training curves
• Use only one set of parameters
• Skip documentation
• Use confusing variable names
• Leave code uncommented

🎯 Architecture Patterns

Classification (Iris)

Input(4) → Hidden(10) → Output(3)
         ReLU         Softmax

Digits Recognition

Input(64) → Hidden(50) → Output(10)
          ReLU          Softmax

Binary Classification

Input(n) → Hidden(20) → Output(1)
         ReLU          Sigmoid

Deep Network

Input → Hidden1 → Hidden2 → Hidden3 → Output
        ReLU      ReLU      ReLU      Softmax

📚 Key Terminology

Epoch: One complete pass through training data

Batch: Subset of data processed at once

Learning Rate (η): Step size for weight updates

Loss: Measure of prediction error

Gradient: Direction to adjust weights

Activation: Non-linear transformation function

Overfitting: Memorizing training data

Underfitting: Failing to learn patterns

Backpropagation: Algorithm to compute gradients

Forward Pass: Computing predictions

Backward Pass: Computing gradients

🚀 Next Steps

After mastering basics:

1. Convolutional Neural Networks (CNNs)
2. For image processing
3. Learn filters and convolution

1. Recurrent Neural Networks (RNNs)
2. For sequential data
3. Time series, text

1. Deep Learning Frameworks
2. TensorFlow/Keras
3. PyTorch

1. Advanced Topics
2. Transfer learning
3. GANs
4. Transformers

🆘 Need Help?

During Class:

• Ask your teacher
• Check with lab partner
• Review lesson materials

Outside Class:

• Review solutions notebook
• Check documentation
• Search error messages
• Email: info@evolveaiinstitute.com

🔗 Useful Links

Visualizations:

• TensorFlow Playground: playground.tensorflow.org
• Neural Network Simulator: nn-simulator.com

Tutorials:

• 3Blue1Brown Neural Networks: youtube.com/@3blue1brown
• Fast.ai Course: course.fast.ai

Documentation:

• NumPy: numpy.org/doc
• scikit-learn: scikit-learn.org
• Matplotlib: matplotlib.org

Print this card and keep it handy during labs!

Evolve AI Institute

Free AI Education for All

info@evolveaiinstitute.com | www.evolveaiinstitute.com