Lecture 11: Solving Hard Problems
In the previous lecture we have learned why some problems are hard (e.g., NP-hard) and (probably) impossible to solve in general.However, there are many situations when such problems can be solved. In this lecture, we learn some approaches used in practice.
Running example. We will try to apply every approach we talk about in this lecture to the Minesweeper puzzle:
This is based on a computer puzzle game that is installed by default in MS Windows, and also available for most other operating systems. There are mines hidden in the square; you need to find them. Every number says how many cells around it contain mines.
This is an NP-complete problem (more precisely, its decision version -- does the solution exist? -- is NP-complete). But could we write a program to solve it?
Small Instances
Some people think "this is a NP-hard problem, it cannot be solved" without even looking at the size of the instance they need to solve. In fact, when the instances are very small, we might still solve them by brute force. For example, in a Minesweeper puzzle with about 25 potential mines, there are only 32 millions of possibilities -- on a modern computer, we could simply check all of them!The case above has about 90 potential mines, so there are too many possibilities -- but maybe with a bit of smarter checking we could also check all of them.
Approximation algorithms
This is a rich area of research, but we do not go much into detail. Some optimization NP-hard problems can be solved efficiently if we decide to be happy with a solution which is 99% as good as the best one. For example, the Knapsack Problem is hard when the numbers are large, but it is easy when the capacity and the item sizes are small. When the numbers are large, we can "approximate" them by ignoring the later digits. This way, we can obtain a solution which is close to the best solution, in polynomial time. However, some problems cannot be optimized in this way.Parameterized complexity
The instance we are attacking may have some 'parameter' that is low; then, in some cases we can devise an algorithm that, while does not work efficiently in general, works efficiently when this parameter is low.For example, consider the Minesweeper problem on a $n \times k$ board, where $n$ is big, but $k$ is small (for example, $k=5$).
This could be solved using dynamic programming, as follows:
The subproblem shown on the picture is "for the given arrangement of mines in the red stripe, is it possible to solve the left (green) part of the board in such a way the solution is correct for every number inside this area". We can solve such subproblems based on solutions to earlier subproblems (where the red stripe is moved to the left). There are $O(4^k n)$ possible subproblems, which means that Minesweeper can be solved efficiently on narrow boards. (On square boards it still requires exponential time.) This is a big area of research; there are many parameters which may help to solve the problem efficiently, such as width (as above), tree-width (similar to the board width used above, but for general graphs), etc.
Random Search
Some people, when attempting to solve a difficult optimization problem, will notice that there are too many potential solutions to check all of them, so they will just try to generate random solutions and return the best one.While Minesweeper is not an optimization problem, we could easily make it into one. We compute the penalty as follows: for every number $k$, compute the number of mines $m$ around it, and add $(k-m)^2$ to the penalty. So a correctly solved board has penalty 0, while for a board solved incorrectly, the penalty is larger.
The picture of the left shows a randomly generated solution. Some of the numbers happen to be correct, but most are wrong (the penalty is 60). The picture of the right has been obtained by generating 1000000 random solutions and taking the best of them. There are more completely or roughly correct numbers, but it is still far from being correct (the penalty is 15).
As we can see, this is by no means an efficient method, but is a good stepping stone for the later methods.
Hill Climbing
The main reason why the previous approach does not yield very good results is that, when we manage to find a solution which is correct in some part of the board, we do not try to 'improve' it by making it also correct in other parts of the board -- we just throw the whole board away and try a new, completely random solution. Although we will find correct solutions of every part of the board during our random search, it is unlikely to find a solution that is correct everywhere!The simplest approach which tackles this is called Hill Climbing. Again, we start with a random solution. Then, instead of generating a completely new random solution, we try to introduce a minor change; if this minor change yields a better solution (smaller penalty) we keep it, otherwise we go back to the previous solution.
We can imagine a person who is trying to climb the highest mountain in the fog. We do not see where the highest mountain is, but we can tell which direction should we go to climb the nearest hill (just go in the 'steepest' direction).
Here is the result of the Hill Climbing approach applied to Minesweeper. In every iteration, we try to add or remove a mine in a random location, and keep the change if it reduces the penalty. After 1414 iterations, changing a mine never improves our solution. We did not achieve the correct solution, but a quite good one (penalty = 5) -- much better and quicker than the random search.
Simulated Annealing
The problem with the Hill Climbing approach is that it finds a local optimum: if no local move improves the solution, it will stay in place, not trying anything else. Once it has climbed to the top of a hill, it will be unable to climb a higher mountain. For example, in the picture below, it could stop on the first hill on the left (A), without attempting to find larger hills on the right (C and D).
Simulated Annealing is an attempt to fix this issue. In the Simulated Annealing approach, we also start with a random solution, and then run a number of iterations. In every iteration we have a temperature parameter ($T$), which decreases during the process (typically it initially has a high value and proceeds to lower and lower values).
Contrary to the Hill Climbing approach, when the newly obtained solution is worse than the old one, we sometimes also keep it -- with probability $\exp(-c/T)$, where $c$ is the difference between the old and the new penalty.
With very high temperature we accept all changes -- in effect, this is similar to the Random Search.
With medium temperatures, after the algorithm climbs a hill, it will still try to explore. It will be unlikely to descend a mountain that is high (relative to the current temperature), but if it is in a local maximum, it will try to explore around, and try to find a higher mountain nearby. So, one the picture above, if the temperature is high enough, the algorithm will find a way to leave the first hill (A) and find the higher hills to the right. With lower temperature, the algorithm will explore the landscape on the right (C-D), but it will not try to cross the valley (B) to go back to the first hill (A).
With very low temperature this algorithm is basically equivalent to Hill Climbing -- we will only make changes which improve our solution.
Here is the result of the Simulated Annealing approach applied to Minesweeper. We did 100000 iterations, starting from temperature 1000, and decreasing it to 0.1 (logarithm of the temperature is a linear function of the iteration number). We did not find the correct solution, but the penalty is just 1.
Simulated Annealing turns out to work great in practice for optimization problems.
Genetic algorithm
Genetic algoritms are another attempt to improve the random search / hill climbing approach. Genetic algorithms are inspired by the process of natural selection in biology.Instead of generating a single random solution and improving it, we instead keep a "population" of solutions. We compute the quality ("fitness") of every solution. In the next iteration, we obtain a new "population" by simulating the process of natural selection: solutions with better fitness have more offspring. As in the biological evolution, every solution in the new population is based on combining two "parents" from the old population in some way ("crossover"). As in RS/HC/SA, we also apply mutations -- random changes to the offspring.
To solve the Minesweeper problem using a genetic algorithm, we consider solutions with a smaller penalty to have better fitness. To construct a new solution based on two old solutions (A and B), we create a random line passing through the board, and one side of that line is taken from A, and one side is taken from B. (We also apply mutations, of course.)
Here is a solution to our Minesweeper puzzle found by a very simple genetic algorithm. The first row is the best solution in the population (after 1000, 2000, 5000 and 10000 steps) while the second row is the 10th among the population of 100 (same number of steps). We have not found the correct solution this time (but with a different seed, GA does manage to find it). Genetic algorithms may yield better results than SA for problems where crossing-over works well (i.e., solutions that are 'good' in different areas can be successfully combined into a solution that is good everywhere), but in general, they are hard to do right, and the Simulated Annealing approach is often likely to work better.
SAT solvers
As said in the previous lecture, NP-complete problems (such as Minesweeper) are hard, because we can reduce any problem in NP to them. So if we knew how to quickly solve one of the NP-complete problem in general (e.g., Minesweeper), we can quickly solve all of them!In the previous lecture, we have learned that NP-completeness is a common way of showing that it is very unlikely that we will be ever able to solve a problem in general -- there are thousands of known NP-complete problems and lots of research, and we still have not been able to solve any of them.
However, these reductions can be also used for good. It is enough to write a good solver for one of the NP-complete problems; since all other NP problems reduce to it, such a solver can be then used to solve many other problems in practice!
And this approach actually works. The NP-complete problem chosen is usually some variation of the Boolean satisfiability (SAT) problem, mentioned in the previous lecture. Many SAT solvers have been written, and it turns out that, while they are unable to solve all the instances of the NP-hard problems, many problems that people run into in practice can be solved very fast, even if they have tens of thousands of variables and millions of constraints!
(Of course, not all of them -- for example, cryptography is based on NP problems which appear to be close to the worst case; SAT solvers are not likely to help you to break Internet security.)
Here is an attempt to solve the Minesweeper board above using a SAT solver z3 by Microsoft Research. This is one of the most popular and successful SAT solvers. We represent every potential mine as a boolean variable, and add a constraint for every number. (Of course we could write a Python program to generate variables and constraints automatically from the board -- the file above lists the generated variables and constraints explicitly, for simplicity.) Z3 has no problem solving our example board:
A human solving our Minesweeper puzzle will use logical reasoning. They will immediately notice that, since the number in bottom right corner is zero, neither of cells adjacent to it may contain a mine, and thus each one of the two cells adjacent to '2' above to it has to contain a mine, and so on. As we have seen, Simulated Annealing did not use such logical reasoning, and thus failed to solve even the obvious '0' in the corner. Similar to a human, a good SAT solver will be able to easily detect such obvious consequences and thus solve our Minesweeper board.