I am trying to use the example from here to detect a stop sign.
But when I change the image to search for stop signs in there are a huge number of false positives.
Am I doing something wrong?
You are not doing anything wrong. This particular example is there for a quick illustration, and it does not produce a very good detector. It trains a 5-stage detector from a very small training set. A decent detector may have 20 stages or more, and you would need thousands of positive samples and negative images to train it.
Take a look at this tutorial for more information.