Darktable: Noise profiles are wrong

[rawfiner]

Describe the bug
Noise profiles aim at having a constant variance of 1 after the generalized anscombe transform of denoiseprofiled, so that the denoising algorithms can work on gaussian noise of variance 1.
Yet, if we check the variance obtained on a flat image at 100% zoom with the compute variance feature, I get variance values of about 14-15 for fujifilm xt3, and about 10 for lumix fz1000.

The noiseprofiling tool assumes that noise is only fine grain, which is not true in practice.
Also, it uses an input color profile of REC709, which (I guess) modifies the values, so the noiseprofiles are not made on the real data we get after demosaic.

To Reproduce
Steps to reproduce the behavior:

1. Shoot a defocused picture of a flat uniform image (e.g. https://artphotoheritage.files.wordpress.com/2011/06/gris-neutre.png ) at several ISO values.
2. In darktablerc, set plugins/darkroom/denoiseprofile/show_compute_variance_mode to "true"
3. Launch darktable
4. For each of the considered pictures:
   4.1 Move denoiseprofile before colorin
   4.2 Put denoiseprofile in "compute variance" mode
   4.3 Zoom to 100% in the center of the image (to minimize the contribution of vignetting in the variance)
   4.4 The displayed variance values are not close to 1... however, it looks like they are almost the same for all ISO values.

Expected behavior
At step 4.3, we should see variances close to 1.

Screenshots

How bad is it to have wrong profiles?
It is bad for making presets that works well on different cameras if the variance values we get differ from one camera to another.
It does not change much in terms of denoising quality if the user uses the strength parameter to fit its need, as even with a wrong profile the generalized anscombe transform will succeed in stabilizing the variance.

Let me go into further details:
The generalized anscombe we do is (approximately) X -> 2*sqrt(X\/a)
The "a" number is supposed to be chosen such as a*mean = variance
V[ 2*sqrt(X\/a) ] is approximately equal to:
V [ 2*sqrt(mean\/a) + 1\/(a*sqrt(mean\/a)) * (X-mean) ] (Taylor expansion of sqrt around the mean, see https://en.wikipedia.org/wiki/Taylor_expansions_for_the_moments_of_functions_of_random_variables )
which gives V[ 2*sqrt(X\/a) ] is approximately equal to (a\/(a*a*mean))*V[X] = 1\/(a*mean)*V[X] = 1
Thanks to the fact that a*mean = variance, we can get a variance of 1 after the transform.

Now, what happens if we make mistakes and take a value which gives a*mean=c*variance?
We can go through the same computations, but we will get V[X] = 1\/c at the end.
So, the good new is that the variance remains constant!

The question is, is the factor between the wrong profile and a true one (i.e. the number "c") always the same? Is our current noiseprofiling tool consistent in the errors it makes?
If so, then it is not necessary to correct anything, we simply get profiles which allows to have a variance of 1\/c instead of 1, which is not problematic.
However, if the number "c" changes from one camera to another, or from one ISO value to another, then we have an issue: the denoiseprofile will behave differently from one camera to another.

From my first experiments, it _seems_ that the "c" does not change much from one ISO value to another, yet I get different values between my 2 cameras.
What I would like to also check is if all bayer cameras have the same "c", and all xtrans cameras have the same "c". In that case, we would still be able to fix the issue without changing the noise profiling tool, and without having to redo all noise profiles.
Otherwise, I guess the only option will be to do a new profiling tool, and to redo all noise profiles.

[rawfiner]

I'd like to assign the issue to myself but I don't see the option, maybe I do not have the required permissions

[edgardoh]

I have different numbers with my Nikon 7100, but the exposure is not the same as yours.

What do you mean by force parameter? I can't find it in the GUI.

[rawfiner]

Thanks for testing. So we will have to update the noise profiling tool and redo all the profiles. The exposure should not change a lot of things as the variance is supposed to be constant relatively to the exposure after the transform.

Oops sorry, I forgot it was called "strength" and not "force" in english

[edgardoh]

BTW, it is really better to have it before the colorin? The color denoise
on the equalizer does a better job after the colorin, maybe with a
different profile we can have better results with this one too. And the
image will have applied the icc profile, so it should be more accurate.

El sáb., 30 mar. 2019 a las 13:04, rawfiner ([e-mail])
escribió:

Thanks for testing. So we will have to update the noise profiling tool and
redo all the profiles. The exposure should not change a lot of things as
the variance is supposed to be constant relatively to the exposure after
the transform.

Oops sorry, I forgot it was called "strength" and not "force" in english

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/darktable-org/darktable/issues/2313#issuecomment-478259612,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APq1pW_GuYCaUBOlSgz17VQIlRwBGdyqks5vb4spgaJpZM4cTxQ5
.

[edgardoh]

Also, I can share my test shots if you want to play with it...

El sáb., 30 mar. 2019 a las 13:04, rawfiner ([e-mail])
escribió:

Thanks for testing. So we will have to update the noise profiling tool and
redo all the profiles. The exposure should not change a lot of things as
the variance is supposed to be constant relatively to the exposure after
the transform.

Oops sorry, I forgot it was called "strength" and not "force" in english

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/darktable-org/darktable/issues/2313#issuecomment-478259612,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APq1pW_GuYCaUBOlSgz17VQIlRwBGdyqks5vb4spgaJpZM4cTxQ5
.

[rawfiner]

Well, I don't know if it is better to have it before the colorin, but it seems that, when using color blending mode, results are better after colorin. Yet, it is hard to compare, as we do not have profiles made for after the colorin yet, and the variance of the different channels is affected by the colorin.

I guess we could leave denoiseprofile after the colorin if we can adapt easily the profile from one working space to another.
I think, but I have not tested yet, that working space would impact the variance values.
Thus, we need a way to change the profiles accordingly.

For instance, let's say we use working color space 1 for profiling.
If we have a picture with working colorspace 2, we need to convert our profile.
As long as the RGB channels in colorspace 2 can be written as an affine transform from colorspace 1, it will be easy to adapt the profile. But if that condition is not fullfiled, it may be way more complicated.
I am far from being an expert in colorspaces, so what do you think about this condition?

I would very much like to see your test shots to play with, thanks for sharing :-)

[rawfiner]

One drawback would be that user would have to stick with "standard color matrix". Otherwise, the profile would be wrong

[edgardoh]

this are the files:
https://1drv.ms/f/s!AqPOdeUNwMndkE_Xu3pAp2sgqB1O

Changing the colorspace is no problem, after the colorin the denoise
profile will work with the work profile, so we can change it to the
profiled colorspace (lin rec 709 for now), the transform is fast so
performance shouldn't be a problem if we get better results.
I'm just saying this because of what I have observed with the equalizer,
but its a different algorithm so maybe what I say makes no sense, and I
have no idea how much work can be to try it just to compare results, you
are the expect on denoise, so what you consider best is ok with me.

El sáb., 30 mar. 2019 a las 14:18, rawfiner ([e-mail])
escribió:

Well, I don't know if it is better to have it before the colorin, but it
seems that, when using color blending mode, results are better after
colorin. Yet, it is hard to compare, as we do not have profiles made for
after the colorin yet, and the variance of the different channels is
affected by the colorin.

I guess we could leave denoiseprofile after the colorin if we can adapt
easily the profile from one working space to another.
I think, but I have not tested yet, that working space would impact the
variance values.
Thus, we need a way to change the profiles accordingly.

For instance, let's say we use working color space 1 for profiling.
If we have a picture with working colorspace 2, we need to convert our
profile.
As long as the RGB channels in colorspace 2 can be written as an affine
transform from colorspace 1, it will be easy to adapt the profile. But if
that condition is not fullfiled, it may be way more complicated.
I am far from being an expert in colorspaces, so what do you think about
this condition?

I would very much like to see your test shots to play with, thanks for
sharing :-)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/darktable-org/darktable/issues/2313#issuecomment-478267102,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APq1pUnsVESywfOUvrML2zYpqZGVXrpYks5vb5xrgaJpZM4cTxQ5
.

[edgardoh]

True, I don't know about the luma, but the chroma can be very different
depending on the input color profile, so it should be the same on dt and
the profiler.

El sáb., 30 mar. 2019 a las 14:22, rawfiner ([e-mail])
escribió:

One drawback would be that user would have to stick with "standard color
matrix". Otherwise, the profile would be wrong

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/darktable-org/darktable/issues/2313#issuecomment-478267707,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APq1pcV13hJX-OpB5__gB2UKeXiALHPGks5vb51wgaJpZM4cTxQ5
.

[edgardoh]

If this restriction is too much how about adding a "pass through" option to the colorin and colorout? This way we can export in camera rgb and both process will have the same input image.

[rawfiner]

Thanks for the files.
Indeed, we can change it to the profiled colorspace, I did not think about that!
About the equalizer: I think the algorithm is the same as the wavelets of denoise profiled, but without the variance stabilizing transform, and in a different color representation.

I guess we will have to compare the results before and after colorin by making 2 profiles for testing, and we will decide depending on the result.
The good thing is that I think it won't be too much work to try both: a new noiseprofile tool can work this way:

1. export pfm file from a noisy blurred picture. This will give "noisy.pfm"
2. export pfm file of the same picture heavily denoised using raw denoise. This will give "smooth.pfm"
3. open both pfm files in the noise profiling tool, compute noisy-smooth to get only the noise part, and from this compute the variance
4. find the profile parameters from the variance values, i.e. find a straight line which fits our variance data

Basically, testing with denoiseprofile before or after colorin will only change steps 1. and 2. , and not the logic of the profiling tool. In other words, we will only need to change 2 xmp files, so it is easy to do.

And for these, yes, having passthrough options in colorin and colorout would help a lot :-)
Would you have time to implement these passthrough options?

[edgardoh]

>

And for these, yes, having passthrough options in colorin and colorout
would help a lot :-)
Would you have time to implement these?

Sure, it should be easy. I'll do a first version so you can use it in your
tests, then we can decide if we want to keep it.

[aurelienpierre]

Well, I don't know if it is better to have it before the colorin, but it seems that, when using color blending mode, results are better after colorin. Yet, it is hard to compare, as we do not have profiles made for after the colorin yet, and the variance of the different channels is affected by the colorin.

Always denoise before signal amplification. Exposure is an amplification, base curve too, profile gamma too. So, denoise before them no matter what.

[rawfiner]

@aurelienpierre I agree, but if both denoiseprofile and exposure are after the colorin, we don't have this problem: exposure can come after denoiseprofile

[aurelienpierre]

Exposure should go before colorin to retain physical consistency and be a true exposure compensation (in camera RGB). It doesn't make a difference as long as you use a simple matrice in your input ICC profile, but if you have a LUT ICC profile instead, the physical meaning of exposure (put after colorin) is undefined (it won't behave the same as lowering the shutterspeed in-camera).

[junkyardsparkle]

I think maybe some of the other modules that were before colorin in the fixed ordering might have been for good reasons. In general, anything that applies a profile should apply it to the same data that the profile was generated from, right? The lens correction module should also theoretically be before colorin for vignetting correction purposes.

[github-actions[bot]]

This issue did not get any activity in the past 30 days and will be closed in 7 days if no update occurs. Please check if the master branch has fixed it since then.

[github-actions[bot]]

This issue did not get any activity in the past 30 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.