Both your images contain many lines that have nothing to do with the sign you're looking for. And some of those lines are longer/have higher contrast than the lines you actually want, so I think detecting the edge lines (e.g. using a hough transform or by summing up contrasts horizontally/vertically) will not work.
But: The sign you're looking for has other characteristics that should be easier to detect:
- There sign background has (almost) constant brightness
- It takes up a relatively large area of the image
- It's near the center of the image
So you're looking for a large connected area with low contrast. I've hacked a proof-of-concept algorithm in Mathematica. (I'm not an OpenCV expert, but I'll mention the respective OpenCV function when I know them.)
First, I use gaussian derivative filters to detect the gradient magnitude at each pixel. The gaussian derivative filter has a wide aperture (11x11 pixels in this case), so it is very noise-insensitive. I then normalize the gradient image to mean=1, so I can use the same thresholds for both samples.
src = Import["http://www.freeimagehosting.net/uploads/720da20080.jpg"];
pixels = ImageData[ColorConvert[src, "Grayscale"]];
gradient = Sqrt[GaussianFilter[pixels, 5, {1, 0}]^2 + GaussianFilter[pixels, 5, {0, 1}]^2];
gradient = gradient/Mean[Flatten[gradient]];
OpenCV implementation: You can use sepFilter2D
for the actual filtering, but apparently, you'll have to calculate the filter kernel values yourself.
The result looks like this:

In this image, the sign background is dark and the sign borders are bright. So I can binarize this image and look for dark connected components.
binaryBorders = Binarize[Image[gradient], 0.2];
sign = DeleteBorderComponents@ColorNegate[binaryBorders];
largestComponent = SortBy[ComponentMeasurements[sign, {"Area", "ConvexVertices"}][[All, 2]], First][[-1, 2]];
OpenCV implementation: Thresholding should be straightforward, but I think OpenCV doesn't contain connected component analysis - you can either use flood fill or cvBlobsLib for that.
Now, just find the largest blob near the center of the image and find the convex hull (I've simply used the largest blob that's not connected to the background, but that might not be enough for every image).
Results:
