Suppose I have a data set of a large number of related images and I extract HOG features and train it using SVM Classifier or KNN Classifier. How can I use this model to generate a new image which is related to the images in the dataset (rather than testing on a new image which we do traditionally)?
There are multiple aspects which render your approach infeasible:
SVM is a learning technique where high-dimensional data is mapped to lower dimension (a couple of hyperplanes, decision boundaries, which pass through support vectors). It doesn't have an inverse mapping: It is not super trivial to go from a point within a decision space back to the space of actual data. In other words, it's not generative. If your learning method supports going back from low-dimensional embedding to the higher dimensions space (such as GPLVM), your task could be easier.
HOG is not exactly reversible. Please check Hoggles, where the authors reveal what HOG features tend to understand, via formulating the visualization as in inverse problem. Therefore, it might not be able to synthesize consistent colors, or even detailed features. The synthesized image might end up looking just like a Hoggle image.
I would suggest the Generative Adverserial Networks for this task, which have been quite successful in image synthesis:
Here are some papers about it:
Here are some open source software: