# How to figure out or estimate the level of noise in a given data distribution

I have been given a data distribution which was synthetically generated by geostatistical methods such as variogram analysis. Without having the source of the technique which generated the data, is there any way to find out or at least estimate the level of noise (a parameter to quantify how noisy the data is or the amount of noise in the data) in the data? I have provided a sample of my data here. I know that without an underlying model, its very difficult to distinguish between noise and the signal, but given the nature of the subsurface rock property data which has to be of close values if located close to each other (same range without having outliers or data points that are suddenly too low or too large) for every range of the data, in other words, if are located close to each other, I was wondering is there is any way to describe or quantify the noise.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

noisy_data = [[0.0467251, 0.0529133, 0.0775945, 0.0640082, 0.0439774, 0.0915829,
0.0547699, 0.0826791, 0.0883616, 0.0977739],
[0.0875183, 0.0790368, 0.101146 , 0.0777582, 0.104289 , 0.0775479,
0.156569 , 0.0999909, 0.0905242, 0.161871 ],
[0.1427   , 0.1211   , 0.15661  , 0.113337 , 0.135986 , 0.1384   ,
0.181515 , 0.159102 , 0.200004 , 0.187535 ],
[0.163381 , 0.186425 , 0.174798 , 0.14191  , 0.1514   , 0.166044 ,
0.199063 , 0.146319 , 0.173303 , 0.144067 ],
[0.182485 , 0.16901  , 0.11     , 0.127162 , 0.0858656, 0.1564   ,
0.161187 , 0.0588985, 0.110706 , 0.0560944],
[0.103804 , 0.109948 , 0.0707144, 0.0493536, 0.0471329, 0.0708205,
0.048201 , 0.0487304, 0.069967 , 0.0947939],
[0.0382397, 0.0918   , 0.101197 , 0.102453 , 0.135943 , 0.0867575,
0.123071 , 0.0970329, 0.099906 , 0.101536 ],
[0.113361 , 0.167597 , 0.141526 , 0.0792396, 0.118    , 0.065764 ,
0.0915817, 0.1183   , 0.149376 , 0.118608 ],
[0.10647  , 0.1566   , 0.1303   , 0.139754 , 0.174465 , 0.148458 ,
0.187625 , 0.132805 , 0.171687 , 0.158161 ],
[0.15319  , 0.174204 , 0.161628 , 0.127377 , 0.173368 , 0.145292 ,
0.208068 , 0.168076 , 0.14961  , 0.124334 ]]

plt.imshow(noisy_data, cmap ="jet")
plt.colorbar()


• I think you have to create (make your best estimate) of what a similar pattern would look like if it was noise free, then based on that assumption, your deviation from ideal if statistically variant as independent deviations (in constrast to a constant offset or shift which could arguably be corrected) would be the noise. Commented Jan 19, 2023 at 16:38

No. Without more information it is impossible to know how much is noise or not. For example, in a very simple linear system, if $$y$$ is your measured variable, $$x$$ the real variable and if $$e$$ is the noise, you need $$x$$ to know $$e$$ and viceversa:

$$y(i,j)=x(i,j)+ e(i,j)$$

You can gather some model constraints for compensate your lack of knowledge.

I.e. if you say your real variable is constant, $$x(i,j)=x_0$$, then you immediately can calculate $$e$$:

$$e(i,j)=y(i,j)-x_0$$

Or you can say the measured variable is smooth under some known spatial linear smoothness $$y(i,j)=A(i',j')*x(i,j)$$ (such as $$y(i,j)=(1+4a)x(i,j)-ax(i\pm1,j)-ax(i,j\pm1))$$), and the noise is equally distributed, $$e(i,j)\sim N(0,1)$$, which is something often used, you can estimate $$x$$ by applying a spatial filter, just like for Image Processing techniques.

$$X=A^{-1}Y$$

Further assumptions are always possible, strictly tied to your modeling skills and of course, if they are pertinent and have some real meaning, and are not product of guessing caused by poor measuring and unrelated information.