Why do we get less amount of lines then desired in line detection using HoughLines tranformation from OpenCV? How could we improve this method and get all the lines appearing in the image? I would still use HoughLines if probable.
The performance of line detectors are image and application dependent in generally. Typically, people tune the parameters of the detectors to get the best possible result. When such a result cannot be obtained, it is often because of the insufficient performance of the detector for the current problem. One approach would be to change the algorithm. For instance, OpenCV also has an LSD line segment detector. Have you tried that? Maybe it can also be used in combination with Hough lines.
Having said all those, I would also look at the hardware/system related issues, if they are under your control: Smartly selecting system parameters such as lighting, cameras and lenses generally solve half of the problem. If you would like to operate in the wild, then I would suggest more sophisticated software solutions.
Edit: This question looks very similar to the one recently posted: MatLab - Can not detect all lines in a "simple" image using houghlines
Maybe this could help you out.