Bounty Ended with 50 reputation awarded by tim occurred Dec 5 '13 at 17:55 5 Added the linear transfer example edited Nov 27 '13 at 15:45 Jazzmaniac 4,03911 gold badge77 silver badges1818 bronze badges Addendum: For sake of completeness let me quickly show how this model generalizes your approach as described in the question. Assume that the camera is in fact linear, and that the response function is therefore $$F(A)=s A$$ for some proportionality factor $$s$$. The derivative is then $$F'(A)=s$$ and the inverse $$F^{-1}(V)=\frac{V}{s}$$. With this special case, the gradient of the penalty becomes $$\frac{\partial P[V]}{\partial T}=\frac{8}{MN} \sum_{m,n}\left(V_{m,n}-\frac{1}{2} \right)\cdot \left( \frac{V_{m,n}}{T}\right)$$ and the intensity scaling factor $$s$$ cancels. Finding an extremum of this penalty function corresponds to choosing $$T$$ so that the gradient vanishes. $$\sum_{m,n}\left(V_{m,n}-\frac{1}{2} \right)\cdot V_{m,n}=0$$ Now let us quickly consider the trivial case where all $$V_{m,n}=V$$ are constant and not equal to zero. Then we can divide by $$V$$ and get that $$V-1/2=0$$ or $$V=1/2$$. With our convention of $$V$$ being in $$[0,1]$$ that means the solution is a constant flat image of 50% gray. Of course all $$V_{m,n}$$ being identical is a very non practical assumption. So let's look at the more general case and expand the product in the sum. $$\sum_{m,n}V_{m,n}^2-\frac{1}{2} V_{m,n}=0$$ or written slightly different $$\sum_{m,n} \left( V_{m,n}-\frac{1}{2} \right)^2 + V_{m,n} - \frac{1}{4} - \frac{1}{2} V_{m,n}=0$$ and after collecting and scaling $$\sum_{m,n} 2\left( V_{m,n}-\frac{1}{2} \right)^2 + V_{m,n} = \frac{1}{2}$$ We can express that with the mean brightness $$\langle V \rangle$$ and the quadratic deviation from the 50% brightness $$\Delta V$$ as $$2 \Delta V + \langle V \rangle = \frac{1}{2}$$ For very small deviations $$\Delta V$$ we get the solution you proposed in your question, namely $$\langle V \rangle = \frac{1}{2}$$. But the more brightness variations you have in the picture the bigger the (strictly positive) deviation becomes, and the mean brightness must drop below $$\frac{1}{2}$$. So this model generalizes your proposal for greater brightness variations, and that is very important. Because a picture that is black almost everywhere but has some small area with very high intensity would still be over exposed if you just make the mean brightness equal to 1/2. The huge variation will however make sure that the mean brightness will be chosen to be significantly lower, and save your result from over exposure! Addendum: For sake of completeness let me quickly show how this model generalizes your approach as described in the question. Assume that the camera is in fact linear, and that the response function is therefore $$F(A)=s A$$ for some proportionality factor $$s$$. The derivative is then $$F'(A)=s$$ and the inverse $$F^{-1}(V)=\frac{V}{s}$$. With this special case, the gradient of the penalty becomes $$\frac{\partial P[V]}{\partial T}=\frac{8}{MN} \sum_{m,n}\left(V_{m,n}-\frac{1}{2} \right)\cdot \left( \frac{V_{m,n}}{T}\right)$$ and the intensity scaling factor $$s$$ cancels. Finding an extremum of this penalty function corresponds to choosing $$T$$ so that the gradient vanishes. $$\sum_{m,n}\left(V_{m,n}-\frac{1}{2} \right)\cdot V_{m,n}=0$$ Now let us quickly consider the trivial case where all $$V_{m,n}=V$$ are constant and not equal to zero. Then we can divide by $$V$$ and get that $$V-1/2=0$$ or $$V=1/2$$. With our convention of $$V$$ being in $$[0,1]$$ that means the solution is a constant flat image of 50% gray. Of course all $$V_{m,n}$$ being identical is a very non practical assumption. So let's look at the more general case and expand the product in the sum. $$\sum_{m,n}V_{m,n}^2-\frac{1}{2} V_{m,n}=0$$ or written slightly different $$\sum_{m,n} \left( V_{m,n}-\frac{1}{2} \right)^2 + V_{m,n} - \frac{1}{4} - \frac{1}{2} V_{m,n}=0$$ and after collecting and scaling $$\sum_{m,n} 2\left( V_{m,n}-\frac{1}{2} \right)^2 + V_{m,n} = \frac{1}{2}$$ We can express that with the mean brightness $$\langle V \rangle$$ and the quadratic deviation from the 50% brightness $$\Delta V$$ as $$2 \Delta V + \langle V \rangle = \frac{1}{2}$$ For very small deviations $$\Delta V$$ we get the solution you proposed in your question, namely $$\langle V \rangle = \frac{1}{2}$$. But the more brightness variations you have in the picture the bigger the (strictly positive) deviation becomes, and the mean brightness must drop below $$\frac{1}{2}$$. So this model generalizes your proposal for greater brightness variations, and that is very important. Because a picture that is black almost everywhere but has some small area with very high intensity would still be over exposed if you just make the mean brightness equal to 1/2. The huge variation will however make sure that the mean brightness will be chosen to be significantly lower, and save your result from over exposure! 4 deleted 1 characters in body edited Nov 21 '13 at 18:07 Jazzmaniac 4,03911 gold badge77 silver badges1818 bronze badges If you know the exact response of the camera, you can convert the brightness samples of each pixel to a linear intensity scale and perform the averaging there. That will make your whole problem intensity linear and should solve all your problems. However, I would strongly recommend using a more advanced exposure algorithm. For example you could introduce a penalty function that gets large for very low intensities and very high intensities. Summing up the penalty of all pixels gives you a total penalty that you can minimize by changing the exposure time. If the penalty is nicely behaved (smooth, etc) then you can use fast converging optimization algorithms. As requested I will briefly describe a possible implementation of the penalty method. Your variables are the exposure time $$T$$, the pixel values $$V_{m,n}\in [0,1]$$ for $$m\in 1.. M$$ and $$n \in 1..N$$. You also have the invertible transfer function $$F$$ of the camera that maps real physical intensities to pixel values. This function is also differentiable and its derivative is $$F'$$. With this we can express the pixel values in terms of the physical intensities $$A_{m,n}$$ and the exposure time $$T$$ as: $$V_{m,n} = F\left(T A_{m,n} \right)$$ Next we introduce a penalty functional that assigns high scores to very dark and very bright regions and low scores to well lit regions. Note that you can have more than just one free parameter (here exposure time) and a more complicated penalty model. I'll just go for the simplest sensible one. The penalty functional is $$P[V] = \frac{4}{MN}\sum_{m,n} \left(V_{m,n}-\frac{1}{2}\right)^2$$ Now we can plug in our exposure model $$P[V] = \frac{4}{MN}\sum_{m,n} \left(F(TA_{m,n})-\frac{1}{2}\right)^2$$ and see how the penalty changes with the exposure time $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(F(TA_{m,n})-\frac{1}{2} \right)\cdot\left(F'(TA_{m,n})\cdot A_{m,n} \right)$$ This expression unfortunately depends on the unknown physical intensities $$A$$. But we can use the exposure model again, but inverted, to estimate those from the pixel values we have. That means with $$T A_{m,n} = F^{-1}(V_{m,n})$$ and therefore also $$A_{m,n}=F^{-1}(V_{m,n})/T$$ we get $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(V_{m,n}-\frac{1}{2} \right)\cdot\left(F'(F^{-1}(V_{m,n}))\cdot F^{-1}(V_{m,n})\frac{1}{T} \right)$$$$\frac{\partial P[V]}{\partial T} = \frac{8}{MN} \sum_{m,n} \left(V_{m,n}-\frac{1}{2} \right)\cdot\left(F'(F^{-1}(V_{m,n}))\cdot F^{-1}(V_{m,n})\frac{1}{T} \right)$$ Summing up, you can use the pixel values $$V_{m,n}$$ from just one measurement to calculate both the penalty and its gradient, using even just an approximate model of your camera. With value and slope you can then start to iteratively search for the minimum penalty by descending the gradient in some clever way. How exactly you do that is a matter of experimentation and taste. But with the square based model here I would suggest you use a parabolic approximation of the penalty function to guess the minimum. Also note that I chose the penalty so that it is normalized between 0 and 1. So you can use that scale to decide if a found minimum is well exposed absolutely or not. This could be useful as a quality indicator for your images. If you know the exact response of the camera, you can convert the brightness samples of each pixel to a linear intensity scale and perform the averaging there. That will make your whole problem intensity linear and should solve all your problems. However, I would strongly recommend using a more advanced exposure algorithm. For example you could introduce a penalty function that gets large for very low intensities and very high intensities. Summing up the penalty of all pixels gives you a total penalty that you can minimize by changing the exposure time. If the penalty is nicely behaved (smooth, etc) then you can use fast converging optimization algorithms. As requested I will briefly describe a possible implementation of the penalty method. Your variables are the exposure time $$T$$, the pixel values $$V_{m,n}\in [0,1]$$ for $$m\in 1.. M$$ and $$n \in 1..N$$. You also have the invertible transfer function $$F$$ of the camera that maps real physical intensities to pixel values. This function is also differentiable and its derivative is $$F'$$. With this we can express the pixel values in terms of the physical intensities $$A_{m,n}$$ and the exposure time $$T$$ as: $$V_{m,n} = F\left(T A_{m,n} \right)$$ Next we introduce a penalty functional that assigns high scores to very dark and very bright regions and low scores to well lit regions. Note that you can have more than just one free parameter (here exposure time) and a more complicated penalty model. I'll just go for the simplest sensible one. The penalty functional is $$P[V] = \frac{4}{MN}\sum_{m,n} \left(V_{m,n}-\frac{1}{2}\right)^2$$ Now we can plug in our exposure model $$P[V] = \frac{4}{MN}\sum_{m,n} \left(F(TA_{m,n})-\frac{1}{2}\right)^2$$ and see how the penalty changes with the exposure time $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(F(TA_{m,n})-\frac{1}{2} \right)\cdot\left(F'(TA_{m,n})\cdot A_{m,n} \right)$$ This expression unfortunately depends on the unknown physical intensities $$A$$. But we can use the exposure model again, but inverted, to estimate those from the pixel values we have. That means with $$T A_{m,n} = F^{-1}(V_{m,n})$$ and therefore also $$A_{m,n}=F^{-1}(V_{m,n})/T$$ we get $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(V_{m,n}-\frac{1}{2} \right)\cdot\left(F'(F^{-1}(V_{m,n}))\cdot F^{-1}(V_{m,n})\frac{1}{T} \right)$$ Summing up, you can use the pixel values $$V_{m,n}$$ from just one measurement to calculate both the penalty and its gradient, using even just an approximate model of your camera. With value and slope you can then start to iteratively search for the minimum penalty by descending the gradient in some clever way. How exactly you do that is a matter of experimentation and taste. But with the square based model here I would suggest you use a parabolic approximation of the penalty function to guess the minimum. Also note that I chose the penalty so that it is normalized between 0 and 1. So you can use that scale to decide if a found minimum is well exposed absolutely or not. This could be useful as a quality indicator for your images. If you know the exact response of the camera, you can convert the brightness samples of each pixel to a linear intensity scale and perform the averaging there. That will make your whole problem intensity linear and should solve all your problems. However, I would strongly recommend using a more advanced exposure algorithm. For example you could introduce a penalty function that gets large for very low intensities and very high intensities. Summing up the penalty of all pixels gives you a total penalty that you can minimize by changing the exposure time. If the penalty is nicely behaved (smooth, etc) then you can use fast converging optimization algorithms. As requested I will briefly describe a possible implementation of the penalty method. Your variables are the exposure time $$T$$, the pixel values $$V_{m,n}\in [0,1]$$ for $$m\in 1.. M$$ and $$n \in 1..N$$. You also have the invertible transfer function $$F$$ of the camera that maps real physical intensities to pixel values. This function is also differentiable and its derivative is $$F'$$. With this we can express the pixel values in terms of the physical intensities $$A_{m,n}$$ and the exposure time $$T$$ as: $$V_{m,n} = F\left(T A_{m,n} \right)$$ Next we introduce a penalty functional that assigns high scores to very dark and very bright regions and low scores to well lit regions. Note that you can have more than just one free parameter (here exposure time) and a more complicated penalty model. I'll just go for the simplest sensible one. The penalty functional is $$P[V] = \frac{4}{MN}\sum_{m,n} \left(V_{m,n}-\frac{1}{2}\right)^2$$ Now we can plug in our exposure model $$P[V] = \frac{4}{MN}\sum_{m,n} \left(F(TA_{m,n})-\frac{1}{2}\right)^2$$ and see how the penalty changes with the exposure time $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(F(TA_{m,n})-\frac{1}{2} \right)\cdot\left(F'(TA_{m,n})\cdot A_{m,n} \right)$$ This expression unfortunately depends on the unknown physical intensities $$A$$. But we can use the exposure model again, but inverted, to estimate those from the pixel values we have. That means with $$T A_{m,n} = F^{-1}(V_{m,n})$$ and therefore also $$A_{m,n}=F^{-1}(V_{m,n})/T$$ we get $$\frac{\partial P[V]}{\partial T} = \frac{8}{MN} \sum_{m,n} \left(V_{m,n}-\frac{1}{2} \right)\cdot\left(F'(F^{-1}(V_{m,n}))\cdot F^{-1}(V_{m,n})\frac{1}{T} \right)$$ Summing up, you can use the pixel values $$V_{m,n}$$ from just one measurement to calculate both the penalty and its gradient, using even just an approximate model of your camera. With value and slope you can then start to iteratively search for the minimum penalty by descending the gradient in some clever way. How exactly you do that is a matter of experimentation and taste. But with the square based model here I would suggest you use a parabolic approximation of the penalty function to guess the minimum. Also note that I chose the penalty so that it is normalized between 0 and 1. So you can use that scale to decide if a found minimum is well exposed absolutely or not. This could be useful as a quality indicator for your images. 3 added 229 characters in body edited Nov 21 '13 at 16:59 Jazzmaniac 4,03911 gold badge77 silver badges1818 bronze badges If you know the exact response of the camera, you can convert the brightness samples of each pixel to a linear intensity scale and perform the averaging there. That will make your whole problem intensity linear and should solve all your problems. However, I would strongly recommend using a more advanced exposure algorithm. For example you could introduce a penalty function that gets large for very low intensities and very high intensities. Summing up the penalty of all pixels gives you a total penalty that you can minimize by changing the exposure time. If the penalty is nicely behaved (smooth, etc) then you can use fast converging optimization algorithms. As requested I will briefly describe a possible implementation of the penalty method. Your variables are the exposure time $$T$$, the pixel values $$V_{m,n}\in [0,1]$$ for $$m\in 1.. M$$ and $$n \in 1..N$$. You also have the invertible transfer function $$F$$ of the camera that maps real physical intensities to pixel values. This function is also differentiable and its derivative is $$F'$$. With this we can express the pixel values in terms of the physical intensities $$A_{m,n}$$ and the exposure time $$T$$ as: $$V_{m,n} = F\left(T A_{m,n} \right)$$ Next we introduce a penalty functional that assigns high scores to very dark and very bright regions and low scores to well lit regions. Note that you can have more than just one free parameter (here exposure time) and a more complicated penalty model. I'll just go for the simplest sensible one. The penalty functional is $$P[V] = \frac{4}{MN}\sum_{m,n} \left(V_{m,n}-\frac{1}{2}\right)^2$$ Now we can plug in our exposure model $$P[V] = \frac{4}{MN}\sum_{m,n} \left(F(TA_{m,n})-\frac{1}{2}\right)^2$$ and see how the penalty changes with the exposure time $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(F(TA_{m,n})-\frac{1}{2} \right)\cdot\left(F'(TA_{m,n})\cdot A_{m,n} \right)$$ This expression unfortunately depends on the unknown physical intensities $$A$$. But we can use the exposure model again, but inverted, to estimate those from the pixel values we have. That means with $$T A_{m,n} = F^{-1}(V_{m,n})$$ and therefore also $$A_{m,n}=F^{-1}(V_{m,n})/T$$ we get $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(V_{m,n}-\frac{1}{2} \right)\cdot\left(F'(F^{-1}(V_{m,n}))\cdot F^{-1}(V_{m,n})\frac{1}{T} \right)$$ Summing up, you can use the pixel values $$V_{m,n}$$ from just one measurement to calculate both the penalty and its gradient, using even just an approximate model of your camera. With value and slope you can then start to iteratively search for the minimum penalty by descending the gradient in some clever way. How exactly you do that is a matter of experimentation and taste. But with the square based model here I would suggest you use a parabolic approximation of the penalty function to guess the minimum. Also note that I chose the penalty so that it is normalized between 0 and 1. So you can use that scale to decide if a found minimum is well exposed absolutely or not. This could be useful as a quality indicator for your images. If you know the exact response of the camera, you can convert the brightness samples of each pixel to a linear intensity scale and perform the averaging there. That will make your whole problem intensity linear and should solve all your problems. However, I would strongly recommend using a more advanced exposure algorithm. For example you could introduce a penalty function that gets large for very low intensities and very high intensities. Summing up the penalty of all pixels gives you a total penalty that you can minimize by changing the exposure time. If the penalty is nicely behaved (smooth, etc) then you can use fast converging optimization algorithms. As requested I will briefly describe a possible implementation of the penalty method. Your variables are the exposure time $$T$$, the pixel values $$V_{m,n}\in [0,1]$$ for $$m\in 1.. M$$ and $$n \in 1..N$$. You also have the invertible transfer function $$F$$ of the camera that maps real physical intensities to pixel values. This function is also differentiable and its derivative is $$F'$$. With this we can express the pixel values in terms of the physical intensities $$A_{m,n}$$ and the exposure time $$T$$ as: $$V_{m,n} = F\left(T A_{m,n} \right)$$ Next we introduce a penalty functional that assigns high scores to very dark and very bright regions and low scores to well lit regions. Note that you can have more than just one free parameter (here exposure time) and a more complicated penalty model. I'll just go for the simplest sensible one. The penalty functional is $$P[V] = \frac{4}{MN}\sum_{m,n} \left(V_{m,n}-\frac{1}{2}\right)^2$$ Now we can plug in our exposure model $$P[V] = \frac{4}{MN}\sum_{m,n} \left(F(TA_{m,n})-\frac{1}{2}\right)^2$$ and see how the penalty changes with the exposure time $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(F(TA_{m,n})-\frac{1}{2} \right)\cdot\left(F'(TA_{m,n})\cdot A_{m,n} \right)$$ This expression unfortunately depends on the unknown physical intensities $$A$$. But we can use the exposure model again, but inverted, to estimate those from the pixel values we have. That means with $$T A_{m,n} = F^{-1}(V_{m,n})$$ and therefore also $$A_{m,n}=F^{-1}(V_{m,n})/T$$ we get $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(V_{m,n}-\frac{1}{2} \right)\cdot\left(F'(F^{-1}(V_{m,n}))\cdot F^{-1}(V_{m,n})\frac{1}{T} \right)$$ Summing up, you can use the pixel values $$V_{m,n}$$ from just one measurement to calculate both the penalty and its gradient, using even just an approximate model of your camera. With value and slope you can then start to iteratively search for the minimum penalty by descending the gradient in some clever way. How exactly you do that is a matter of experimentation and taste. But with the square based model here I would suggest you use a parabolic approximation of the penalty function to guess the minimum. If you know the exact response of the camera, you can convert the brightness samples of each pixel to a linear intensity scale and perform the averaging there. That will make your whole problem intensity linear and should solve all your problems. However, I would strongly recommend using a more advanced exposure algorithm. For example you could introduce a penalty function that gets large for very low intensities and very high intensities. Summing up the penalty of all pixels gives you a total penalty that you can minimize by changing the exposure time. If the penalty is nicely behaved (smooth, etc) then you can use fast converging optimization algorithms. As requested I will briefly describe a possible implementation of the penalty method. Your variables are the exposure time $$T$$, the pixel values $$V_{m,n}\in [0,1]$$ for $$m\in 1.. M$$ and $$n \in 1..N$$. You also have the invertible transfer function $$F$$ of the camera that maps real physical intensities to pixel values. This function is also differentiable and its derivative is $$F'$$. With this we can express the pixel values in terms of the physical intensities $$A_{m,n}$$ and the exposure time $$T$$ as: $$V_{m,n} = F\left(T A_{m,n} \right)$$ Next we introduce a penalty functional that assigns high scores to very dark and very bright regions and low scores to well lit regions. Note that you can have more than just one free parameter (here exposure time) and a more complicated penalty model. I'll just go for the simplest sensible one. The penalty functional is $$P[V] = \frac{4}{MN}\sum_{m,n} \left(V_{m,n}-\frac{1}{2}\right)^2$$ Now we can plug in our exposure model $$P[V] = \frac{4}{MN}\sum_{m,n} \left(F(TA_{m,n})-\frac{1}{2}\right)^2$$ and see how the penalty changes with the exposure time $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(F(TA_{m,n})-\frac{1}{2} \right)\cdot\left(F'(TA_{m,n})\cdot A_{m,n} \right)$$ This expression unfortunately depends on the unknown physical intensities $$A$$. But we can use the exposure model again, but inverted, to estimate those from the pixel values we have. That means with $$T A_{m,n} = F^{-1}(V_{m,n})$$ and therefore also $$A_{m,n}=F^{-1}(V_{m,n})/T$$ we get $$\frac{\partial P[V]}{\partial T} = \frac{4}{MN} \sum_{m,n} 2\left(V_{m,n}-\frac{1}{2} \right)\cdot\left(F'(F^{-1}(V_{m,n}))\cdot F^{-1}(V_{m,n})\frac{1}{T} \right)$$ Summing up, you can use the pixel values $$V_{m,n}$$ from just one measurement to calculate both the penalty and its gradient, using even just an approximate model of your camera. With value and slope you can then start to iteratively search for the minimum penalty by descending the gradient in some clever way. How exactly you do that is a matter of experimentation and taste. But with the square based model here I would suggest you use a parabolic approximation of the penalty function to guess the minimum. Also note that I chose the penalty so that it is normalized between 0 and 1. So you can use that scale to decide if a found minimum is well exposed absolutely or not. This could be useful as a quality indicator for your images. 2 Added explanation for the penalty method edited Nov 21 '13 at 16:47 Jazzmaniac 4,03911 gold badge77 silver badges1818 bronze badges 1 answered Nov 20 '13 at 18:21 Jazzmaniac 4,03911 gold badge77 silver badges1818 bronze badges