5. Machine Learning Fundamentals

5.1 Features and Target

Example:

hours_studied | attendance | passed
-----------------------------------
5             | 80         | 1
2             | 50         | 0

hours_studied + attendance = features (X)
passed                     = target (y)

5.2 Classification vs Regression

Classification

Predict a category.

Examples: - scam / not scam - churn / stay - fraud / legitimate - cat / dog

Regression

Predict a continuous quantity.

Examples: - house price - revenue - temperature - delivery time

Important: A target being numeric does not automatically mean regression.

0 = legitimate
1 = scam

is still classification.

6. Training a scikit-learn Model

Example:

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X_train, y_train)

predictions = model.predict(X_test)

Meaning

fit()
→ learn patterns/parameters from training data

predict()
→ use learned patterns on new data

7. Train / Validation / Test

Typical idea:

Training set
→ model learns

Validation set
→ tune/select models and hyperparameters

Test set
→ final unbiased evaluation

Example:

70% train
15% validation
15% test

Not a universal rule---just a common split.

Generalization

A good model should perform well on unseen data, not merely memorize training examples.

8. Data Leakage

Leakage occurs when information the model should not have during training improperly influences training/preprocessing.

Bad idea:

Entire dataset
     ↓
fit scaler
     ↓
split train/test

Better:

split
 ↓
fit preprocessing on training data
 ↓
apply learned preprocessing to validation/test

Pipelines help prevent this.

9. Overfitting vs Underfitting

Underfitting

Training performance: poor
Validation performance: poor

Model hasn't learned enough useful structure.

Overfitting

Training performance: excellent
Validation performance: much worse

Model learned training-specific patterns/noise and generalizes poorly.

Good fit

Training and validation performance are both reasonably strong and close.

Ways to reduce overfitting can include: - more useful training data - simpler model - regularization - feature selection - cross-validation

10. Classification Metrics

Confusion Matrix

              Predicted
             +         -
Actual +     TP        FN
Actual -     FP        TN

Precision

Question:

When the model predicts positive, how often is it correct?

Useful when false positives are costly.

Example: automatically blocking legitimate transactions.

Recall

Question:

Of all actual positives, how many did we detect?

Useful when missing positives is costly.

Example: detecting fraud or serious safety cases.

F1 Score

Balances precision and recall.

Accuracy

correct predictions / all predictions

Accuracy can be misleading for highly imbalanced datasets.

Example:

99 legitimate
1 scam

A model predicting everything as legitimate gets 99% accuracy while detecting zero scams.

11. Preprocessing

Real datasets often need preprocessing before training.

Missing values

SimpleImputer(strategy="median")

Categorical data

OneHotEncoder(handle_unknown="ignore")

Example:

Basic
Standard
Premium

becomes numerical encoded features.

Scaling

StandardScaler()

Useful for algorithms sensitive to feature scale, including: - Logistic Regression - KNN - SVM - neural networks

Tree models usually need scaling less.

12. Pipelines

A pipeline chains preprocessing and modeling.

Pipeline([
    ("preprocessor", preprocessor),
    ("classifier", LogisticRegression())
])

Conceptually:

Raw data
   ↓
Missing-value handling
   ↓
Encoding
   ↓
Scaling
   ↓
Model

Benefits: - consistent preprocessing - easier deployment - less leakage risk - easier cross-validation

13. ColumnTransformer

Different columns often require different preprocessing.

preprocessor = ColumnTransformer([
    ("numeric", numeric_pipeline, numeric_features),
    ("categorical", categorical_pipeline, categorical_features)
])

Example:

age              → numeric processing
monthly_spend    → numeric processing
plan             → categorical processing

14. Cross-Validation

Instead of evaluating against only one validation split, cross-validation repeatedly trains/evaluates across different folds.

For 5-fold CV:

Fold 1 → validation
Folds 2–5 → training

Fold 2 → validation
others → training

...

Useful for more robust model comparison, especially with limited data.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X, y, cv=5)

15. Parameters vs Hyperparameters

Parameters

Learned by the model.

Examples: - regression coefficients - neural-network weights - neural-network biases

Hyperparameters

Chosen/configured by the developer.

Examples:

learning rate
batch size
number of epochs
tree max_depth
regularization strength

Search tools:

GridSearchCV(...)
RandomizedSearchCV(...)

For pipeline parameters:

"model__C"

16. Churn Predictor Project

Workflow we built:

customers.csv
      ↓
Pandas / EDA
      ↓
Features X + Target y
      ↓
Train/Test Split
      ↓
Preprocessing Pipeline
      ↓
Logistic Regression
      ↓
Evaluation
      ↓
Save Model
      ↓
FastAPI
      ↓
Prediction API

Saving a model

import joblib

joblib.dump(model, "models/churn_model.pkl")

Loading:

model = joblib.load("models/churn_model.pkl")

Training vs inference

train.py
→ train model
→ save model

api.py / predict.py
→ load saved model
→ make predictions

Important: Do not retrain the model for every API request.