ImageFirm intelligence / field guide 01

Know the method.
Question the claim.

20 machine learning methods, taken beyond the cheat sheet. Understand what each learns, where it breaks, and how to test whether it deserves a place in your system.

Explore the 20 methods ↓
20entries audited2interactive experiments26 OCT 2026research review
Read this first

Three ideas the poster leaves out.

Similarity is a design choice.

Neighbors and clusters depend on features, scaling and distance. A clean-looking group can be a preprocessing artifact.

Probability is not certainty.

A model score needs calibration and a decision threshold. Business costs determine which errors you can afford.

Validation is part of the model.

A random split can leak time, customers or projects. Test the future situation you actually intend to deploy into.

01 / Audited reference

From intuition to implementation.

Every card audits the supplied blueprint. Open the detail panel for mechanisms, tuning, limitations and validation. Business examples are proposed experiments, not documented deployments. Select up to three entries to compare.

01 / Regression

Linear regression

Estimate a number using an additive model.

ŷ = β₀ + Σ βⱼxⱼ
Blueprint audit

Accurate, simplified: linear means linear in coefficients; transformed features can represent curves. The intercept is implicit in Xθ only if X includes a constant column.

Mechanism, trade-offs & application

How it works

Ordinary least squares minimizes squared residuals. Ridge penalizes large coefficients; lasso can set coefficients to zero.

What to tune

Regularization strength; transformations; interactions.

Where it fails

Outliers and correlated features can destabilize estimates. Normal residuals are not required to compute predictions.

Illustrative application

Ruz: estimate construction cost from floor area, specification and location; coefficients describe association, not causal effects.

How to validate

Compare MAE and RMSE with a median baseline; hold out newer projects and inspect residuals by project type.

02 / Classification

Logistic regression

Estimate class probabilities from a linear score.

P(y=1|x) = 1 / (1 + exp(−(β₀ + βᵀx)))
Blueprint audit

Accurate for binary classification: multinomial logistic regression uses softmax. A linear boundary is in the chosen feature space.

Mechanism, trade-offs & application

How it works

Fits log loss, usually with regularization. A decision threshold converts probability into an action.

What to tune

Penalty strength; class weights; decision threshold.

Where it fails

Probabilities are not automatically well calibrated; class weighting changes their interpretation.

Illustrative application

ImageFirm: rank inbound leads by conversion probability using only information available at arrival.

How to validate

Check log loss, precision–recall and calibration; choose a threshold using the cost of missed and wasted follow-ups.

03 / Classification

K-nearest neighbors classification

Let nearby labeled examples vote.

ŷ = argmax_c Σᵢ∈Nₖ(x) wᵢ · 1[yᵢ=c]
Blueprint audit

Incomplete: similarity depends on representation, scaling and distance. Raw-pixel image classification is only a limited example.

Mechanism, trade-offs & application

How it works

Stores training examples; queries retrieve k neighbors and aggregate labels. Weights may be uniform or distance based.

What to tune

k; metric; weighting; feature scaling.

Where it fails

Distances lose usefulness in high dimensions; brute-force prediction costs roughly O(nd) per query.

Illustrative application

ImageFirm: classify support requests using normalized text embeddings and reviewed neighboring examples.

How to validate

Tune k inside cross-validation; check minority-class recall and prediction latency.

04 / Classification

Naive Bayes

Combine class priors with feature evidence.

P(y|x) ∝ P(y) ∏ⱼ P(xⱼ|y)
Blueprint audit

Correction: independence is conditional on the class, not unconditional feature independence.

Mechanism, trade-offs & application

How it works

Gaussian, multinomial and Bernoulli variants use different likelihoods. Smoothing avoids zero likelihoods for unseen tokens.

What to tune

Likelihood family; smoothing; class priors.

Where it fails

Correlated evidence can produce overconfident probabilities even when labels are useful.

Illustrative application

ImageFirm: create a fast text-routing baseline before paying for a larger model.

How to validate

Track macro-F1 and per-class errors; assess calibration if probabilities drive decisions.

05 / Classification · Regression

Decision tree

Split the feature space into simple rules.

Choose splits that reduce impurity or prediction loss.
Blueprint audit

Accurate, qualified: small trees are readable; deep trees are not automatically interpretable.

Mechanism, trade-offs & application

How it works

Greedy splits create leaves that predict classes or numeric values. A locally best split need not yield a globally best tree.

What to tune

Maximum depth; minimum leaf size; pruning.

Where it fails

Small data changes can alter the structure. Regression leaves do not extrapolate trends beyond observed target values.

Illustrative application

Ruz: build a compact feasibility triage rule set for human review.

How to validate

Compare shallow and pruned trees on held-out projects; inspect rare leaves and unstable rules.

06 / Classification · Regression

Random forest

Average many deliberately different trees.

Regression: ŷ = (1/B) Σᵦ Tᵦ(x)
Blueprint audit

Incomplete: random feature subsets at splits are central, alongside bootstrap sampling in the usual formulation.

Mechanism, trade-offs & application

How it works

Averaging decorrelated trees reduces variance. Out-of-bag samples provide an internal evaluation when bootstrapping is used.

What to tune

Tree count; feature subset size; leaf size; depth.

Where it fails

High accuracy is not guaranteed. Impurity importance can favor high-cardinality variables.

Illustrative application

Ruz: estimate project overruns from historical project attributes.

How to validate

Use held-out permutation importance; test on later projects and measure memory and serving cost.

07 / Classification · Regression

Gradient boosting

Add weak models that reduce the current loss.

Fₘ(x) = Fₘ₋₁(x) + ηhₘ(x)
Blueprint audit

Overstated: strong tabular performance is common, not universal state-of-the-art. XGBoost and LightGBM are implementations with distinct designs.

Mechanism, trade-offs & application

How it works

New learners follow negative loss gradients; residual fitting is the squared-error special case.

What to tune

Learning rate; rounds; leaf complexity; subsampling; early stopping.

Where it fails

Careless validation and tuning can overfit. More rounds are not always better.

Illustrative application

ImageFirm: benchmark lead scoring against logistic regression under identical splits.

How to validate

Compare profit at a fixed review capacity, calibration and latency—not accuracy alone.

08 / Classification · Regression

Support vector machine

Prefer a boundary with a wide margin.

min ½‖w‖² + CΣᵢ max(0, 1 − yᵢ(wᵀxᵢ+b))
Blueprint audit

Incomplete: kernels create nonlinear boundaries in input space; support vector regression is also available.

Mechanism, trade-offs & application

How it works

Support vectors determine the decision surface. Kernel methods compare observations implicitly in another feature space.

What to tune

C; kernel; gamma for RBF; scaling.

Where it fails

Kernel training can become costly with many samples; scores are not probabilities without additional calibration.

Illustrative application

ImageFirm: benchmark a linear SVM for sparse text classification.

How to validate

Use consistent scaling in a pipeline; compare PR metrics and kernel serving latency.

09 / Clustering

K-means

Represent observations with k centroids.

min Σᵢ ‖xᵢ − μcᵢ‖²
Blueprint audit

Incomplete: standard k-means minimizes squared Euclidean distance, not an arbitrary distance function.

Mechanism, trade-offs & application

How it works

Alternate assigning points to nearest centers and recomputing means. Initialization affects the local solution.

What to tune

k; scaling; initialization; multiple restarts.

Where it fails

Poor fit for curved groups, unequal density and outliers. Lower inertia alone cannot choose k because it decreases as k grows.

Illustrative application

ImageFirm: explore customer behavior segments using comparable, standardized features.

How to validate

Compare silhouette, stability across seeds and whether segments support a distinct useful action.

10 / Clustering

Hierarchical clustering

Build a nested hierarchy of groups.

Merge groups according to a linkage rule.
Blueprint audit

Misleading: a hierarchy can be built without k, but obtaining final groups requires a cut rule such as k or a distance threshold.

Mechanism, trade-offs & application

How it works

Agglomerative clustering merges groups; divisive methods split them. Linkage defines how group distances are measured.

What to tune

Linkage; metric; cut threshold; connectivity.

Where it fails

Single linkage can chain groups. Ward linkage uses Euclidean variance; dense distance storage often scales quadratically.

Illustrative application

Ruz: explore a hierarchy of comparable developments by specification and location.

How to validate

Check whether groupings survive reasonable metric and linkage changes before naming segments.

11 / Clustering

DBSCAN

Connect dense neighborhoods and leave sparse points unassigned.

Core point: |Nε(x)| ≥ min_samples
Blueprint audit

Qualified: handles non-convex shapes, but one density threshold can fail when cluster densities vary. Noise does not automatically mean a business anomaly.

Mechanism, trade-offs & application

How it works

Core points connect dense regions; border points attach to them; remaining points receive a noise label.

What to tune

Neighborhood radius ε; minimum samples; distance metric.

Where it fails

Scaling and dimensionality strongly affect neighborhoods. Some implementations can require quadratic memory.

Illustrative application

Ruz: explore geographic concentration of comparable projects using an appropriate geographic distance.

How to validate

Inspect noise rate and sensitivity to ε; validate flagged records manually.

12 / Representation

Principal component analysis

Rotate data toward directions of greatest variance.

Xcentered = UΣVᵀ; Z = Xcentered Vᵣ
Blueprint audit

Incomplete: high variance is not the same as predictive information. PCA is unsupervised and components are uncorrelated, not necessarily independent.

Mechanism, trade-offs & application

How it works

Projects centered features onto orthogonal directions; a truncated representation discards remaining variation.

What to tune

Component count; standardization; optional whitening.

Where it fails

A low-variance feature may contain the signal you need. Components can be hard to interpret.

Illustrative application

ImageFirm: compress correlated telemetry before a downstream model.

How to validate

Fit PCA only on training folds; compare downstream validation performance as well as retained variance.

13 / Representation

t-SNE

Make a local-neighborhood map of high-dimensional observations.

Minimize KL(P ‖ Q) between pairwise similarity distributions.
Blueprint audit

Incomplete: local structure is emphasized, not perfectly preserved; island sizes and inter-island distances are not reliable measurements.

Mechanism, trade-offs & application

How it works

Uses heavy-tailed similarities in the embedding and a non-convex optimization objective.

What to tune

Perplexity; learning rate; initialization; seed.

Where it fails

Different runs may tell different visual stories. Standard scikit-learn t-SNE has no transform for new observations.

Illustrative application

ImageFirm: inspect document embeddings for possible labeling issues, then verify in the original space.

How to validate

Compare several seeds and perplexities; never treat a separated island as proof of a real class.

14 / Classification · Representation

Linear discriminant analysis

Model classes with a shared covariance.

δc(x) = xᵀΣ⁻¹μc − ½μcᵀΣ⁻¹μc + log πc
Blueprint audit

Incomplete: Gaussian assumptions are class-conditional, with shared covariance. This is not latent Dirichlet allocation.

Mechanism, trade-offs & application

How it works

Produces linear class boundaries; supervised dimensionality reduction has at most C−1 components for C classes.

What to tune

Shrinkage; class priors; solver.

Where it fails

Covariance estimates can be unstable with many features and few examples.

Illustrative application

ImageFirm: try a compact supervised projection for labeled quality categories.

How to validate

Compare shrinkage LDA with logistic regression using a stratified holdout.

15 / Classification

Quadratic discriminant analysis

Allow each class its own covariance.

δc(x) = −½ log|Σc| − ½(x−μc)ᵀΣc⁻¹(x−μc) + log πc
Blueprint audit

Accurate but incomplete: Gaussian class-conditional densities remain assumed; class-specific covariance creates quadratic boundaries.

Mechanism, trade-offs & application

How it works

Estimates separate covariance matrices; flexibility increases the number of fitted parameters.

What to tune

Covariance regularization; class priors.

Where it fails

Each class needs enough information to estimate covariance; singular matrices and overfitting are risks.

Illustrative application

ImageFirm: test whether quality classes differ in feature variability, not only their means.

How to validate

Compare regularized QDA with LDA; examine class-specific validation errors.

16 / Regression

K-nearest neighbors regression

Average the outcomes of similar observations.

ŷ = Σᵢ∈Nₖ wᵢyᵢ / Σᵢ∈Nₖ wᵢ
Blueprint audit

Qualified: time-series forecasting needs lag features and chronological evaluation; neighbor averaging is not inherently a temporal model.

Mechanism, trade-offs & application

How it works

Local averaging produces a piecewise prediction surface. More neighbors smooth predictions but can erase local differences.

What to tune

k; weights; metric; scaling.

Where it fails

Poor extrapolation; irrelevant features distort similarity. Training data must remain available at inference.

Illustrative application

Ruz: estimate apartment value from comparable sales after accounting for location and date.

How to validate

Use geographic or temporal holdouts; compare MAE against a local median baseline.

17 / Neural models

Multilayer perceptron

Compose learned transformations with nonlinear activations.

h = φ(W₁x+b₁); ŷ = g(W₂h+b₂)
Blueprint audit

Qualified: MLP is a neural architecture family. Large data requirements depend on capacity, regularization and the task.

Mechanism, trade-offs & application

How it works

Backpropagation computes gradients; an optimizer updates weights. Output activation and loss must match the target.

What to tune

Width; depth; learning rate; batch size; weight decay.

Where it fails

Training is non-convex; scaling, random seeds and early stopping matter.

Illustrative application

ImageFirm: benchmark nonlinear prediction on structured creative-performance features.

How to validate

Compare with boosted trees; repeat seeds and inspect learning curves before increasing capacity.

18 / Neural models

Convolutional neural network

Reuse local filters across structured inputs.

hᵢⱼ = φ(Σᵤᵥ Kᵤᵥ xᵢ₊ᵤ,ⱼ₊ᵥ + b)
Blueprint audit

Incomplete: convolutions also serve 1D signals and 3D volumes. Cost depends on architecture, input size and hardware.

Mechanism, trade-offs & application

How it works

Local connectivity and shared weights exploit spatial structure. Convolution supports translation equivariance; full invariance is not automatic.

What to tune

Kernel size; stride; channels; augmentation; pretrained backbone.

Where it fails

Background shortcuts and camera changes can break performance despite good random-split scores.

Illustrative application

Ruz: assist defect screening from site photos, with expert review of uncertain cases.

How to validate

Hold out entire sites or cameras; test per-defect recall and deployment latency.

19 / Neural models

Recurrent neural network / LSTM

Carry a learned state through a sequence.

hₜ = φ(Wₓxₜ + Wₕhₜ₋₁ + b)
Blueprint audit

Incomplete: LSTM is a gated RNN variant. Its state is a learned summary, not reliable long-term recall of every input.

Mechanism, trade-offs & application

How it works

Recurrent weights are shared across time. LSTM gates control information flow and help address vanishing gradients.

What to tune

Sequence length; hidden size; layers; gradient clipping.

Where it fails

Sequential computation limits parallelism. Future information in features or normalization causes leakage.

Illustrative application

Ruz: forecast occupancy from historical observations and genuinely known future covariates.

How to validate

Use rolling-origin evaluation against seasonal-naive and lagged tabular baselines.

20 / Reinforcement learning

Q-learning

Learn the long-term value of actions from transitions.

Q(s,a) ← Q(s,a) + α[r + γ maxₐ′ Q(s′,a′) − Q(s,a)]
Blueprint audit

Misleading: reinforcement learning is a paradigm; Q-learning is one algorithm. Optimality is conditional, not a universal promise.

Mechanism, trade-offs & application

How it works

Off-policy temporal-difference learning updates action values. At terminal states, the bootstrap term is zero.

What to tune

Learning rate α; discount γ; exploration schedule.

Where it fails

Tabular convergence needs suitable finite-MDP conditions, sufficient visits and learning-rate conditions. Neural approximation does not inherit the guarantee.

Illustrative application

Ruz: explore simulated energy-control decisions before any real-world trial; reward must include comfort constraints.

How to validate

Evaluate unseen scenarios, constraint violations and return variability—not training reward alone.

02 / Learn by changing one thing

Small experiments. Visible consequences.

Computed in your browser using fixed synthetic examples. These demonstrations teach mechanisms; they are not benchmark results.

Who counts as a neighbor?

The outlined square is a query. Its true class is unknown. Change k and see which training points vote. Coordinates use the same scale and Euclidean distance.

A score becomes a decision.

Twenty synthetic outcomes with fixed probability scores. Predict positive when score ≥ threshold. Lowering the threshold finds more positives but can create more false alarms.

03 / Practical model selection

Start with the task, not the trend.

ImageFirm’s suggested starting points below are engineering judgments. Compare candidates on the same data split, tuning budget and success metric.

NeedFirst experimentUseful challengerDecision test
Predict a numeric business outcomeMedian baseline + ridge regressionRandom forest / gradient boostingMAE by segment, temporal stability
Predict conversion or churnLogistic regressionGradient boostingCalibration, PR curve, value at capacity
Classify text cheaplyTF-IDF + naive Bayes or linear SVMEmbedding-based classifierMacro-F1, rare-class recall, cost
Explore unlabeled groupsScaled k-meansHierarchical clustering / DBSCANStability, geometry and actionability
Compress / inspect featuresPCA for compressiont-SNE for visual explorationDownstream utility / neighborhood checks
Inspect imagesPretrained CNN baselineAlternative pretrained vision backboneSite-level holdout, recall and latency
Forecast a sequenceSeasonal naive + lagged regressionBoosting / RNN or LSTMRolling forecast error by horizon
Choose sequential actionsRules + simulatorQ-learning in a suitable discrete settingReturn, constraint violations, robustness

This is not an exhaustive map of ML. Transformers, Gaussian processes, probabilistic mixtures, causal models and other methods sit beyond the original 20. KNN appears twice because classification and regression aggregate different targets.

04 / Deployment discipline

The experiment must survive reality.

Define

Specify the outcome, prediction time and action. Set a simple baseline and the cost of each kind of error.

Separate

Reserve a final test set. Split by time, entity or project when needed. Fit preprocessing only on training folds.

Compare

Tune on validation data. Report variability, subgroup errors and serving cost. Keep the final test untouched until selection.

Monitor

Track input drift and delayed outcomes. Assign a human owner, rollback trigger and retraining criteria.

Methodology reference: scikit-learn: common pitfalls and leakage prevention ↗

05 / Evidence & scope

Trace the claim to its source.

Each algorithm links directly to technical documentation or an original paper. Sources were checked on 26 October 2026. Documentation can change; pin software versions when reproducing an experiment. This page uses original paraphrases and illustrations.

Blueprint: user-supplied “20 Machine Learning Algorithms” poster carrying the Neuron name. Audits address its visible text and illustrative implications; the poster is not reproduced. No numeric ranking is assigned because a useful simplification is not equivalent to a factual error.