1
$\begingroup$

I know that these days' style-transfer GANs should be able to style-transfer day images into night images. I however prefer to do it using a set of elementary image processing manipulations, so I can control their random parameter ranges.

I have a small dataset of images taken at night, which are very hard to train on with transfer-learning from an ImageNet pre-trained network. This is kind of making sense, as ImageNet images are usually taken in daylight conditions.

I therefore want to try to create a "night version" of ImageNet, by applying elementary image-processing manipulations on it (e.g., adding noise, shrinking the color histogram and reducing its mean, etc.). That way, I hope I will be able to train a network that will be useful for transfer-learning on night images.

Are there known algorithms/methods for doing that daylight->night conversion using image processing?

Improve this question
$\endgroup$
1
  • $\begingroup$ A few things: could you post your images or a link to your images? What camera did you use? Is this a school project, a personal project, or something for your job? What training or background do you have in computer vision and image processing? And what have you tried so far? Have you looked into traditional “day for night” techniques used in making movies? There are a lot of possible answers to your question, but I’d like to know a bit more from you. $\endgroup$ – Rethunk Feb 23 '20 at 14:27
1
$\begingroup$

That really doesn't make sense: you'd be training a neural network to mimic a conversion algorith, which you already have. That's a waste of electricity ;) I doubt the resulting nets would generalize at all: you're not even training them with actual day/night picture pairs. So, really, as usual: one of the most significant problems in machine learning is getting a good set of input data.

In your case, that's still pretty easy (just take a couple hundred pictures from the same positions at day and night – you can basically do that in a week, if you have a camera that you can reliably bring to the same positions). So, don't skip on that crucial step.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I think you haven't understood the question. I don't want the network to learn to convert day to night or vice versa. I want to only train it on night and my problem is that I don't have enough training data for it. For instance, a ResNet-101 pretrained on ImageNet is not a good starting point, as it doesn't almost contain night images. I want to produce a "night-ImageNet" pretrained network so I will use it as a starting point for training on my much smaller dataset. $\endgroup$ – SomethingSomething Feb 15 '20 at 10:21
  • $\begingroup$ again, you're not training an imagenet on night pictures, but on pictures that are just transformed day-time images. $\endgroup$ – Marcus Müller Feb 15 '20 at 10:29
  • $\begingroup$ ok, and what's wrong with that? Option 1: Start from regular ImageNet-pretrained network and train on some small dataset of night images with frozen backbone. Option 2: Start from night-like ImageNet-pretrained network and train on the same small dataset with frozen backbone. I would pick Option 2. $\endgroup$ – SomethingSomething Feb 16 '20 at 8:08
  • $\begingroup$ Anyway, my question was not whether this was a legitimate task or not, but whether there was a known image-processing flow that achieves that. $\endgroup$ – SomethingSomething Feb 16 '20 at 8:09
  • $\begingroup$ I understand that you're saying that the network will learn my image-processing flow, but I'm fine with that. I want it to be able to extract features that will work on night images and this requires a large dataset. Furthermore, if I add randomization of parameters and image manipulations during the day->night conversion, the network may not be able to just learn my flow. Additionally, performing the night->day conversion manually (undoing my manipulations) as a preprocessing is more a waste of electricity than a network that learns to do it in its forward flow, which will anyway run. $\endgroup$ – SomethingSomething Feb 16 '20 at 8:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.