You’ve got 25 cities. You need to visit every single one exactly once and come back home. Sound easy? There are 7.8 × 10²⁴ possible routes—more than stars in the observable universe. Your laptop will die before it tries them all. Gradient descent? Useless here. There’s no slope to descend. This is where most optimization tutorials hit a wall and shrug.
But evolution solved this billions of years ago. No calculus degree required.
Genetic algorithms (GAs) are a legitimate class of optimizer that work on problems where traditional math breaks down. And unlike some of the machine learning hype you’ve heard over the past decade—which often turns out to be glorified pattern-matching with a PR budget—GAs actually deliver something useful. They’re not flashy. They won’t make you rich. But they work.
Why Gradients Fail on Some Problems
Listen, if you’ve spent any time in data science, you know the gospel: gradient descent. Feed in data, compute a slope, follow it downhill. Beautiful. Provably convergent under the right conditions. It works for neural networks, regression, half of machine learning.
But here’s the catch—it only works if your search space is continuous and has a gradient. A smooth surface you can slide down.
The Travelling Salesman Problem is discrete and rugged—a landscape with a million local canyons and no clear downhill direction.
A route through 25 cities isn’t a number on a spectrum. It’s a permutation. A sequence. You can’t average city 3 and city 7 and get something meaningful. The search space is jagged, non-convex, full of local traps. A completely different animal than curve fitting.
That’s when evolution steps in. Nature doesn’t optimize. It experiments. Variation, selection, repetition. No gradients needed.
How Genetic Algorithms Actually Work (Without the Jargon)
The core loop is dead simple. Three steps, repeated.
First: you generate candidates. Random guesses at first. Wild swings across the solution space. Then you test each one against your problem—fitting a line to noisy data, finding a short route, whatever.
Second: you keep the winners and throw out the rest. Better solutions breed. Worse ones disappear. Ruthless. Darwinian.
Third: you mutate. Take the survivors, combine them (crossover), add random noise (mutation), and you’ve got a new generation. Repeat.
In 20 generations, or 500, the population converges to something good. Not perfect—GAs don’t guarantee optimality. But good, and fast enough to actually use.
Take line fitting. You want to find the coefficients (a, b) for y = ax + b that best match 50 noisy data points. A normal person uses calculus. A GA person starts with 100 random guesses—pairs of (a, b) flung across [-10, 10] like darts at a wall.
Each generation, the GA measures which pairs fit best (lowest RMSE), keeps the top 20 survivors, randomly samples the rest weighted by fitness, then crosses them over (averaging the coefficients) and mutates them (adding Gaussian noise).
Line Fitting: Where We Can Actually Verify Success
The beauty of line fitting is you know the right answer. The code knows it’s chasing y = 3x + 8. After 50 generations, the evolved algorithm lands at y = 2.92x + 8.03. Not perfect. But honestly? Close enough for most real problems, especially when you’re dealing with noise.
No closed-form solution. No matrix inversion. Just evolution. The population started at random (a ≈ 0, b ≈ 0) and climbed toward the true parameters like water finding a valley.
The Travelling Salesman Problem: Where It Gets Real
But line fitting is a toy. Let’s talk about something that actually matters: the Travelling Salesman Problem (TSP).
Now the representation changes everything. You can’t just average two routes and expect a third route. City 3 and City 7 don’t split the difference into City 5. A route is a permutation, and that requires completely different genetic operators.
Instead of averaging coefficients, you use ordered crossover—take a slice of cities from parent route 1, fill in the gaps from parent route 2 in the order they appear. It’s clever. It preserves structure while mixing genes.
And instead of adding Gaussian noise, you swap cities randomly. Swap city 5 and city 12. Reorder. Introduce chaos controlled.
This is where the algorithm earns its name. These aren’t just heuristics. They’re genetic operators informed by how actual DNA works—recombination and mutation—adapted to a discrete problem space.
Why This Matters (and Why It Doesn’t)
Let me be straight with you. Genetic algorithms are not going to disrupt your industry. They won’t replace your job. They’re not being VC-funded because they’re not sexy.
But they’ll solve your NP-hard problem when nothing else will. TSP, job scheduling, circuit design, portfolio optimization—problems where brute force is suicide and gradients don’t exist.
Companies like FedEx and UPS have been using variants of GAs for route optimization for decades. Not because they’re perfect. Because they’re good enough, fast enough, and they actually work on real constraints.
The catch? GAs are non-deterministic. Run it twice, you get slightly different answers. They’re not guaranteed optimal—just reasonably good. For some domains (proving a theorem, finding the absolute shortest route) this is unacceptable. For others (planning delivery routes, designing antennas) it’s perfect.
This is why I respect them. They’re honest. They don’t promise certainty. They deliver utility.
The Honest Limitations
And here’s where I push back on the tutorial tone. GAs require careful tuning: population size, mutation rate, selection pressure, number of generations. Get it wrong and you converge to garbage in 50 generations. Get it right and you converge to decent solutions in 200.
There’s also the representation problem. If you pick the wrong way to encode your solution, the genetic operators won’t work. Choose wisely or waste weeks debugging.
And they’re slow compared to modern ML. A well-tuned neural network will beat a GA on structured continuous problems. Full stop.
But on discrete, non-differentiable problems with hard constraints? GAs are one of the few tools that doesn’t give up.
Why You Should Know This (Even If You Never Use It)
The reason to understand GAs isn’t to become a GA expert. It’s to understand a different mode of thinking about optimization. Gradient-based methods assume smoothness. Evolutionary methods assume only that you can measure success and generate variants.
That’s philosophically important. It’s the difference between calculus and statistics. Between derivatives and distributions. Between following a slope and exploring a landscape.
In 20 years of covering tech, I’ve watched optimization thinking evolve from pure math to pure heuristics. GAs sit in the middle—not provably optimal, not purely empirical. They work because they’re inspired by something that optimizes better than any human algorithm: life itself.
Is that reason enough to learn them? Depends on your problem. But if you ever hit a wall where traditional optimization fails, you’ll be glad you know they exist.
🧬 Related Insights
- Read more: Veridian Guard: One Decorator to Save Your AI Agents from LLM Chaos
- Read more: Open Source Adoption Is Booming—But It’s Eating Teams Alive
Frequently Asked Questions
What is a genetic algorithm used for?
GAs solve optimization problems where you can measure success but can’t use gradients—like routing problems, job scheduling, and design optimization. They work by simulating evolution: vary solutions, keep the good ones, combine and mutate them, repeat.
Why use genetic algorithms instead of gradient descent?
Gradient descent only works on smooth, continuous problems. GAs work on discrete, non-differentiable, highly constrained problems like the Travelling Salesman Problem. Different tools for different landscapes.
Can genetic algorithms find the optimal solution?
Not guaranteed. They find good solutions fast. They’re non-deterministic and can get stuck in local optima. But for NP-hard problems, “good enough quickly” beats “optimal never.”
