Darktable: add RGB parade and vectorscope as histogram alternatives

[dtorop]

Is your feature request related to a problem? Please describe.
Video software tends to have an "RGB Parade" and "Vectorscope". Would these be useful for darktable? They would provide alternate ways to visualize color/lightness data besides the regular and waveform histogram.

Describe the solution you'd like
An "RGB Parade" would be trivial to implement as a generalization to the current waveform histogram code. There would be no changes needed to the pixelpipe math, only an additional type of drawing in libs/histogram.c. It should be fast to draw.

A Vectorscope would require more code in develop/pixelpipe_hb.c and would use a bit more memory for a buffer. It would involve adding a bit of code to libs/histogram.c. The polar coordinate math wouldn't be too slow so long as the scope was somewhat lo-res. It should be fast to draw once the color counts are calculated out to a buffer.

With that many histogram/scope alternatives, it would make sense to add keyboard accelerators to choose a specific scope, or to cycle through scopes.

The current histogram widget is a bit squashed for waveforms (or rgb parade), much less a circular vectorscope. It might make sense to allow the widget to resize vertically via the GUI, or to make it automatically resize depending on its contents.

I'd be happy to work on an RGB parade or vectorscope if it were of interest and there was some clarity about a good design.

Alternatives
The waveform histogram already has essentially the same data as an RGB parade, though it may take selecting/deselecting color channels to understand how a particular channel works.

A vectorscope came up in #2140 as a way to visualize how to balance color. It's possible that a home for a vectorscope would be in some sort of interactive color balancing iop, rather than in the histogram widget. Though I'd imagine the readout would be useful as a reference when using a variety of iops.

I'm also curious about whether a "power editor" would want to see multiple scopes at once, e.g. both waveform and vectorscope.

Additional context
Some articles about using scopes:

• https://www.cinema5d.com/primary-colour-correction/
• https://waondering.com/2015/05/18/continuity-of-colors-matching-exposure-and-color-balance/
• https://www.premiumbeat.com/blog/rgb-parade/
• https://www.bhphotovideo.com/explora/video/tips-and-solutions/introduction-waveforms-scopes-and-exposure
• https://thevideoproguys.com/read-histogram-vectorscope-color-correcting-tools/

There appears to be vectorscope code in qctools (https://github.com/bavc/qctools/wiki/Vectorscope). This could be helpful as a model.

I'm unclear if YUV is the standard/only vectorscope color model, or if there is one which is more preferred/useful as so much has happened since analog TV. I'm also not certain if the primary/secondary color targets on the vectorscope are that useful for photographers, except as a guidance to orient their sense of the graph.

There is also even more esoteric scopes, such as YC Waveform and YCbCr Parade. It would be interesting to hear if anyone has substantial experience with these, RGB parade, or vectorscope, and has a vision of how they might or might not be useful.

[chhil]

I miss the RGB parade and vectorscope. A lot of times I convert my raw to dng and edit/color grade it in resolve.

[GrahamByrnes]

Just a naive comment: these sort of tools seem very video-centric, since they display column-average intensities. That may be great when the frequency of portrait-mode cinema is about 1/10^7, the width to height ratio quite big and the story is in some sense laid out across the frame... but is it really useful in portrait mode?
At present, I find the waveform just a wasted step when changing from linear to log histogram. So if someone has time to implement this, great... but it would be nice to have a preset to include it or not in the set of histogram alternatives. Or, reduce the histogram to just linear & log histograms, and move this stuff into a new module.

[dtorop]

A lot of times I convert my raw to dng and edit/color grade it in resolve.

This is good to hear -- that someone uses these tools for still images!

video-centric, since they display column-average intensities

It's a fair point. It would be trivial to make row-average intensities, though it may confuse people used to the "classic" tools. A question is whether it would be better to implement the classic video forms of these, or imagine what the right added visual information could be for photographers.

preset to include it or not in the set of histogram alternatives

This would also be (fairly) trivial. Keyboard shortcuts to choose a scope could also help here.

[dtorop]

One other thought would be to add a "scope" iop. It would default to being at the end of the pixelpipe. Chances are it would be used in lieu of the histogram in the right panel (cf. hide the histogram window via ctrl-shift-h).

The interesting things about this could be:

• The iop could be duplicated, in case the user wished to see a variety of scopes.
• The iop could be dragged earlier in the pixelpipe, in case the user wished to get a sense of the output in intermediary steps, or even to get a sense of the initial raw file (as an alternative to the "raw overexposed indication").
• The iop could have a drop-down/buttons/accels to determine the type of scope (or alternately, there could be different iops for different sorts of scopes).
• The iop widget dimensions could be appropriate to different sorts of scopes, e.g. a vectorscope which looked better square, or a waveform which might benefit with more y-axis space).
• As it's a generalized version of the histogram panel, at some point it could replace this panel, or be displayed/treated specially to appear where that panel appeared.

The troubling things would be:

• It's a hack on the iop/pixelpipe, which is intended to change the image, not give readouts on the image.
• And, indeed, running the iop (even if it was a no-op everywhere but the preview pipe) would still require allocating an output buffer and copying data to it, and using cache space for that buffer, unless there was a further hack to make the pixelpipe treat readout-only iops in a special manner.
• Darkroom mode (depending on config darkroom/ui/single_module etc.) tends toward only one module open at a time, so there'd need to be another hack (or fiddling with config) to keep the scope iop open.
• There are iops such as the various curves which already show a background histogram when needed, and infrastructure to generate these in the pixelpipe. A quick implementation would end up duplicating some of this (though maybe there's some way to use some of that infrastructure).
• As the iop could be at an arbitrary spot in the pipe, the scope would probably need to be able to handle raw, Lch/Lab, and RGB.
• This is a lot of plumbing when the normal case is just to want to get a sense of the final output data.
• From what I can see, https://discuss.pixls.us/t/processing-that-sucks-less/13016 is trying to allow for output "taps" at various points in the pipeline. Better a system designed from scratch to do that well, rather than retrofit it.

[aurelienpierre]

I think an all-purpose scope IOP is a bad idea for the current pixel pipe. It's not only a matter of colour space (RGB/Lab) or data layout (mosaiced raw/demosaiced pic), but also of reference encoding (display or scene).

Just do that at the end of the pipe in display-RGB space.

[dtorop]

Just do that at the end of the pipe in display-RGB space.

Yes, that sounds wise. Anything else is over-engineering.

Maybe it's really a UI/usage problem: What sort of visualization do people need to tune an image? Are the existing visualizations (histogram/waveform) enough? Do more specialized visualizations interfere with the work of people who want the "classics"?

[aurelienpierre]

My understanding of the vector scope and waveform for video is they are primarily designed to help aligning different footages/rushes that might be shot from different cameras/angles at different exposures, in a way that doesn't create inconsistent exposure and colour grades bump over the duration of the video.

We don't have the same constraints in photography, although some photographers work on series (or books, catalogues, etc.) where visual consistency between pictures can be a goal and a challenge. In that case, a vector scope, for example to align skin tones, could be a real advantage.

But for that use case, that also triggers the need to be able to compare the scopes between 2 images (at least).

[dtorop]

But for that use case, that also triggers the need to be able to compare the scopes between 2 images (at least).

Yes, that's interesting. I hadn't thought this out that way. In which case, a vectorscope may be, er, out of scope for the project... I do find myself using the waveform -- I like having a spatial clue to where clipping is happening. Plus, it looks great.

I'm continually fascinated by the tools digital video people have which haven't yet made their way into still photography.

[GrahamByrnes]

As Aurelian said, matching skin tones can be important in a sequence... but
that seems something I can better relate to doing with a color-picker,
since eg a face is going to be surrounded by other things (clothes,
furniture etc) which will typically change between images. That brings me
back to the issue of column averaging (or row averaging): that would seem
to make matching skin more rather than less challenging.

For clipping, isn't that already taken care of with the two clipping
indicators? To go further in that sense would seem more likely to involve
soft-proofing for the display device, to capture out-of-gamut colours.

Le jeu. 23 janv. 2020 21:40, Dan Torop [e-mail] a écrit :

But for that use case, that also triggers the need to be able to compare
the scopes between 2 images (at least).

Yes, that's interesting. I hadn't thought this out that way. In which
case, a vectorscope may be, er, out of scope for the project... I do find
myself using the waveform -- I like having a spatial clue to where clipping
is happening. Plus, it looks great.

I'm continually fascinated by the tools digital video people have which
haven't yet made their way into still photography.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/darktable-org/darktable/issues/4149?email_source=notifications&email_token=AN2X2RQUI3LFYKBDLJKT5K3Q7H6EVA5CNFSM4KI3G6C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJYYYCA#issuecomment-577866760,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AN2X2RUT6YNGOK7GOTARTS3Q7H6EVANCNFSM4KI3G6CQ
.

[dtorop]

@GrahamByrnes All true. An option to select a portion of an image to produce a histogram could help with color-matching just a portion of an image (e.g. face). But as you write, color-picker works well already. Using an appropriate iop/mask could also selectively produce a histogram/waveform for the relevant portion of an image, though this is more awkward.

[aurelienpierre]

Color pickers show you the average, but not the distribution of hue/saturation/lightness. It's more limited than a vectorscope.

[chhil]

I use the RGB parade more often than the vectorscope.

In DT one can do the things with equivalent modules but with some effort.
e.g.
I prefer aligning the parade for white balance using individual RGB components to align it,
Just seeing the individual channels that make the waveform seems more beneficial to grasp the scene and move the sliders to adjust and keeping luminosity at 0 to not make other colors vary.

[dtorop]

Good to hear that RGB parade has some fans.

Here's a UI thought: Replace the single "type" button on histogram (which now cycles logarithmic -> linear -> waveform) with two buttons. A "type" button cycles histogram -> waveform -> vectorscope. Then a "mode" button cycles, depending on the type:

• histogram: logarithmic -> linear
• waveform: waveform -> RGB parade
• vectorscope: no-op, or YUV -> YCbCr -> etc. or 75% -> 100%, or to turn on/off overlays

Or alternately:

• histogram: _as above_
• waveform: column-binned -> row-binned
• RGB parade: column-binned -> row-binned
• vectorscope: _as above_

This would mean:

• people who only use histogram don't have to cycle through a bunch of other scopes to switch from logarithmic->linear
• there'd be room to experiment with variations for each scope type

The option to selectively show R/G/B channels could remain for all scope types, even if it is most useful for histogram users. Or the channel selector buttons could disappear unless the type is "histogram", and other buttons (e.g. 75%->100%) could appear as needed for each scope.

[kikoucalou]

I believe that the border between the technical processing of the still image and that of the industry of movies is held.

Resolve this powerful video editor with nodes used for color gradding was mentioned. It has a very interesting interface with many tools that could be implemented in Darktable for those who would like to use them, like vectorscope.

Vectorscope is certainly a very interesting and complementary tool to other analysis tools like parade ...

[johnny-bit]

I love this idea, since waveform display helped me greatly recently in edit.

What's more - I'd like to echo @aurelienpierre statement - sometimes I do need to keep "consistency" between image series and comparing scopes between 2 images would speed things up a bit ;)

Another thing to note - with being able to hide histogram I get more real-estate for space hungry modules (like color balance), but at the same time I'd love to see the histogram. Perhaps in a window, like darkroom preview? That could hold multiple different histograms (if anybody would need that).

[dtorop]

Glad to hear there's some continued interest in the idea. When I've a moment (this month?) I'll work on a branch with this.

@johnny-bit I've noticed the same thing about vertical space. I wonder if the histogram & other scopes could end up being more useful on the left panel.

[johnny-bit]

I wondered about this and IMO it's not a bad idea - image preview usually does just preview and helps in finding yourself around in zoom... so switching it to histogram display would/could help a bit when starved for space in right panel... I personally am a fan of preview window (which i can move to separate screen) and I'd love to be able to scale and put histogram into window too. My (only so far) usecase that i can think of is to enable horizontal and vertical weaveform while adjusting whitebalance (I have problem doing it by eye on image itself)

[dtorop]

@johnny-bit The argument against histogram in the top-left corner is that when someone is adjusting a module on the right panel and watching the histogram, that their eye has to travel all the way to the top-left to see the histogram, rather than to just above the module... Presumably this would be a whole other project, to make the histogram (and preview) into palettes that can be dragged around...

[dtorop]

FWIW, a first draft of RGB parade is here: https://github.com/darktable-org/darktable/compare/master...dtorop:more-scopes
I went down a bit of a wormhole since doing that work to get the blue in the waveform to paint more visibly. I haven't touched a vectorscope mode yet.
A bunch of the changes in the more-scopes branch is tidying up of src/libs/histogram.c to make all those different modes/types of scope make sense. I haven't done any profiling yet, so don't know if I've introduced performance problems. This branch should use less memory than what's in master, though.

[johnny-bit]

Um... you're creating PR? the link you've produced takes me to Pull Request creation... Shouldn't it just be link to your fork?

Anyway - I'm interested in this :) I hope you'll get something good!

[dtorop]

@johnny-bit It's pre-PR. I haven't tested it enough, and would like to get vectorscope working (and see if there's visual/speed improvements). Just linking to a fork (https://github.com/dtorop/darktable/tree/more-scopes) seems a bit less useful than the diff from master, as I'm always curious to see what the changes are. Am curious if there's a better way.

[github-actions[bot]]

This issue did not get any activity in the past 30 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[dtorop]

I'm finally (I hope) getting back to this. I've been using RGB parade for a few weeks and like it. I'll see if there are any speed regressions and if not submit a PR in hopes it's of interest. I haven't gotten to vectorscope yet, but still plan to.

While I've been using this, one other idea came up: What if dragging on the histogram/waveform not only altered exposure but (with modifier) color temperature as well? That way the two basic corrections for an incoming image could both occur in a large and usually visible area of the right panel without having to navigate tabs/scrolling to open modules at the bottom of the right pane.

Here is an example of how the interaction could work:

• on histogram, dragging left/right still modifies exposure or black point
• but alt-dragging left/right could modify temperature and alt-ctrl-dragging could modify tint
• very similar for waveform except the motion is up/down
• for RGB parade, could ditch modifiers and dragging blue could map to temperature, green to tint, and red to exposure
• there could be a similar sensitivity for scrolling w/in the iop, and possibly a fine adjustment modifier

There could also be GUI feedback as the histogram/waveform is now all drawn by Cairo as separate channels, by fading out the channels not being moved. This would also reduce the temptation to balance color simply by aligning the histograms/waveforms for each channel.

I can make a feature request with this idea -- and implement it -- as I get further along.

[dtorop]

Some benchmarks (all measured in elapsed, not CPU time), comparing master branch to the new code:

Time to calculate the waveform histogram (_pixelpipe_final_histogram_waveform()) has increased from 0.007 to 0.011 seconds. To compare, calculating the "regular" histogram takes 0.003 secs.

Time to draw the waveform histogram (_lib_histogram_draw_callback()) in the new code has increased from 0.004 to 0.008 secs. Drawing the new RGB parade takes 0.003 seconds. In comparison, drawing the "regular" histogram takes 0.002 seconds.

I'll see if there are easy optimizations. I'm not sure if these slowdowns are small enough so as to be negligible.

[dtorop]

It's possible to bring the waveform drawing code back to former performance by compositing the RGB channels in a loop rather than issuing draw calls to Cairo for each channel. I made that change.

On the other hand, I re-tested calculating waveform histogram and now can't replicate that it is faster than the code in my working branch. Both master and my new code appear to take about the same time (approx. 0.010 sec.) after some slight optimizations to the new code.

[chhil]

@dtorop
Just a comment to encourage you to continue with this NFR. I am looking forward to this getting into a DT release.

[dtorop]

@chhil Thank you! The RGB parade code should be in shape, and I made a PR for it. Other work (vectorscope, and maybe dragging histogram to change color) should fall out pretty easily, as much of the work so far is just internal to make it easier to add other histogram modes.

[sobotka]

Vectorscope is problematic all the way down due to the representation of specific encoded primaries assumptions (BT.709 or BT.601, for example) and the encoding format it relies on to express the image (YCbCr, for example).

It would be extremely challenging to represent it properly, and worse, even if it did, it would become nonsensical given the context.

If someone is keen to follow down this, it is worth considering what the Vectorscopes convey (saturation) and consider other more useful and grounded models. One such might be CIE’s 1976 UCS. More contemporary approaches might be something like JzAzBz.

Bear in mind none of these are “ideal” due to the nature of psychophysical response to visible light spectra, as we don’t have a successful model, and may never have. Given the context of a digital still image tool from digital sensors however, there may be greater value in representing the native camera solved virtual primaries, the current working light primaries, and image emission values using such a contemporary model.

[dtorop]

@sobotka That all makes a lot of sense -- I appreciate you pointing this out. I've been wondering about this too -- if the goal is to reproduce a classic video vectorscope, or something relevant for a contemporary image-making. The only risk of the latter would be confusing people used to reading a traditional vectorscope.

Given the context of a digital still image tool from digital sensors however, there may be greater value in representing the native camera solved virtual primaries, the current working light primaries, and image emission values using such a contemporary model.

This makes a lot of sense to me. As the scope would be visualizing what's at the end of the pipeline, probably the current working light primaries would be the most meaningful. I'm seeing this, as you write, mainly as a way to understand saturation over a range of colors.

[sobotka]

I appreciate you pointing this out. I've been wondering about this too -- if the goal is to reproduce a classic video vectorscope, or something relevant for a contemporary image-making. The only risk of the latter would be confusing people used to reading a traditional vectorscope.

I'm glad it wasn't read as "party pooping" as it wasn't the intention. Doing a Vectorscope properly is more challenging in RGB systems than people would like to believe. Even after that, it becomes more of a question of "What are we doing and for whom?", as all design things are.

I'd point at contemporary tools like Baselight and such that offer proper colour science based plots, such as the standardized CIE xyY plot. It's tricky given that while it is perceptually non-uniform, it does have a useful 2D representation of area of gamut with respect to the visible light spectral locus. The perceptually uniform equivalent would be the CIE 1976 UCS, in u'v'.

Of course, with each, comes the need for education to explain what people are looking at. I'm partial to having the spectral locus present on a plot as it shows the limit, in no uncertain terms.

This makes a lot of sense to me. As the scope would be visualizing what's at the end of the pipeline, probably the current working light primaries would be the most meaningful.

There's three states that are extremely useful for some image makers:

1. The input encoding primaries. How dd the image start out life? What is valid data within that? What is invalid introduced by error of camera virtual primaries solving or math such as resampling an image such that negative lobes result?
2. The working reference primaries. How did 1. end up in 2.? Are there negative and therefore nonsensical values in the working space? Should a gamut mapping pass be applied before work, between 1. and 2.?
3. The output primaries rendition. From a selection of gamut mappings, is one required between 2. and 3.?

Those seem rather topical questions for stills work, but also require nuanced tools.

I'm seeing this, as you write, mainly as a way to understand saturation over a range of colors.

Always remember that "saturation" is a human model. That is, it's one that literally does not exist. We can talk about it in loose terms, and even discuss it using various encoding models, but it can be said with utmost certainty that such an axis does not exist in the human experience in isolation. Intensity, particular wavelength, mixture, spatial proximity, illuminant and adapted illuminant, etc. all play a role here, and no such perfectly separated model exists.

Scopes and "information" are terrific, but don't tell the whole story. I'd push forward with your ideas as they are terrific, and always anchor them in "Who is using this, specifically?" and "What the hell are they are trying to achieve, specifically?"

2 pennies.

[dtorop]

@sobotka This is fascinating and helps to answer my initial questions from https://github.com/darktable-org/darktable/issues/4149#issue-551975220:

I'm unclear if YUV is the standard/only vectorscope color model, or if there is one which is more preferred/useful as so much has happened since analog TV. I'm also not certain if the primary/secondary color targets on the vectorscope are that useful for photographers, except as a guidance to orient their sense of the graph.

Part of the interest in vectorscope is to learn what tools digital video people have which could be useful to still image work but haven't yet widely crossed over.

And you write

Scopes and "information" are terrific, but don't tell the whole story ... always anchor them in "Who is using this, specifically?" and "What the hell are they are trying to achieve, specifically?"

This sounds right. My thinking is to hack together some sort of basic (and somewhat correct) implementation, then use it for a bit and see what turns out to be useful. Then the answer to these questions should fall out.

If you're able to stick around as questions come up -- and once I get an implementation going -- I'd be quite curious for more thoughts.

[sobotka]

I'm unclear if YUV is the standard/only vectorscope color model, or if there is one which is more preferred/useful as so much has happened since analog TV. I'm also not certain if the primary/secondary color targets on the vectorscope are that useful for photographers, except as a guidance to orient their sense of the graph.

YUV was the compression schema for analog. In digital, it is discredited under a YCbCr model. The coefficients to peel the RGB into YCbCr varies with the primaries. Conventionally, that typically means the BT.709 normalized luminance weights.

If you're able to stick around as questions come up -- and once I get an implementation going -- I'd be quite curious for more thoughts.

Absolutely. That Baselight video is quite useful. Perhaps starting with the CIE 1931 / CIE 1976 UCS is the most directly useful visualization. The two together are an excellent one-two punch, and given the data available in Darktable, also feasible to visualize the areas of the primaries easily.

[houz]

Just two general remarks:

It might make sense to allow the widget to resize vertically via the GUI, or to make it automatically resize depending on its contents.

The preview on the left and the histogram on the right should have the same size to keep the GUI balanced.

alt-dragging left/right could modify temperature

Alt-dragging moves the window. We can't use that.

[dtorop]

I've been roughing out a vectorscope, as a proof-of-concept. See https://github.com/darktable-org/darktable/compare/master...dtorop:vectorscope.

Things that work:

• There's a vectorscope display.
• It's possible to scale the scope, particularly useful for low-saturation images.
• I find the display to be helpful to understand color cast and intensity/saturation, as well as particular things, e.g. how "preserve chrominance" works.

Things that are ambiguous:

• Colorspace situation is a mess. This is because of everything which @sobotka writes (https://github.com/darktable-org/darktable/issues/4149#issuecomment-616211332, https://github.com/darktable-org/darktable/issues/4149#issuecomment-616238200, https://github.com/darktable-org/darktable/issues/4149#issuecomment-616270752). There's options to choose Rec.601/709/2020, but as the input data is in an arbitrary colorspace, this is meaningless. The right thing would be to mind the histogram profile. Right now, the "traditional" histogram does this, but not waveform! So part of this work would be making sure that waveform (and vectorscope) are profile-dependent. The other thing to do seems to be to ditch YCbCr (the sort of hand-wavey colorspace it's in) and follow up on what @sobotka writes, and choose a meaningful colorspace. The assumption here is that no one cares if this looks like a YUV vectorscope for analog video. The goal is to learn something about the image.
• The default is that the vectorscope is drawn in white, with an alpha channel based on counting the binned pixels with the given chroma. There are options to draw the vectorscope in color, using the chroma of the binned pixels. The luminance can be 50%, or the average/min/max of the binned pixels. These are too many choices. I tend to like the minimum, as it gives me the best sense of the color.
• There's an option to determine which axis is not graphed. The default is to graph chroma and show luminance via transparency. It's possible to graph luminance and a chroma channel and show the other chroma channel via intensity. I'm not sure if this is useful or just confusing.
• This is really pushing the edge of how the UI of the histogram widget is coded. It may be worth further GTK-izing the widget (particularly making the buttons actually be GTK buttons), so it's more flexible and so fewer things are coded from scratch.
• There's no graticule showing where primary/secondary colors would be. I don't miss this. It seems far more important for video work than photography. And dt already has other good tools for color matching. But it would be possible to add this once the color profile situation was less of a mess. Much more interesting would be to do something in reference to output primaries (https://github.com/darktable-org/darktable/issues/4149#issuecomment-616238200).
• There probably should be a drag action, as with the other scopes. Making drag adjust white balance could be appealing.

Other things to do:

• There's no optimization work.
• Icons for buttons need attention.
• Fix lots of hacky code.

Related things which this code does:

• Waveform histogram no longer needlessly zeros out a buffer that it writes to later.
• The histogram widget doesn't redraw every time the mouse moves over it.
• Other minor fixes throughout.

Also, belatedly, regarding what @houz wrote:

The preview on the left and the histogram on the right should have the same size to keep the GUI balanced.

This makes a lot of sense. Especially as zooming the vectorscope is pretty necessary, there seems to be no harm in having a square vectorscope show up in a horizontal widget.

[dtorop]

Thinking this through a bit more: The interesting thing isn't making a vectorscope-like scope. It's making a readout which shows the chroma range of a given image. And -- perhaps this self-evident -- that really is just a view down the lightness-axis of a 3D plot with chroma on the other two axes. (And views looking down other axes would also be interesting.)

So the actual dream would be some sort of 3D plot of the image, generated in a useful colorspace. Ideally with an overlay of the primaries/gamut of the working/input/output colorspace (see https://github.com/darktable-org/darktable/issues/4149#issuecomment-616238200). I'm not sure if darktable will be linking in OpenGL anytime soon (nor if we want to hold a perhaps 64MB voxel buffer of the raw data), but barring a 3D plot, a choice of 2D sections -- appropriately colored -- through such a plot could still be helpful -- and quick to calculate/draw.

[agittins]

Sounds like some very interesting progress!

The idea of a 3D pointcloud of the colourspace is certainly an interesting idea of how to visualise the image.

For my purposes, I'm primarily interested in the vectorscope, and very specifically in the graticules for the I "fleshtone" line and to a lesser extent, the primary and secondary colours. Everything else just seems like yet another way to perceptually visualise vague colours, but the graticuled vectorscope can give actual accurate data. I'd probably rather green-on-black for the rendering, or white - not the actual pixel colour, as that can make it very hard to see, and can hide the varied luma values of a given hue.

I don't have calibrated displays and I work on a laptop most of the time and in varied lighting conditions, so not leaving critical colour choices to a pair of soggy analog meatbags of protein (my eyes) is I think a worthwhile pursuit. I don't really understand why photography has never really embraced this, but we do embrace calibrated displays, colour profiles, controlled lighting etc etc. Maybe it's because they're profitable. And given the popularity of photography among, shall we say, persons more likely to have some form of colour blindness, I don't see why we haven't adopted a more quantifiable colour tool like the vectorscope more readily.

For those unfamiliar with the I axis (often called the "fleshtone" line) on a vectorscope, it's not specific to caucasian skin, but it is centered there. Typically asian tones will tend a few degrees to one side, and darker tones may tend to the other, but the majority of variation between humans is along the I line (ie, the luma or lightness value), as the colour of our skin is determined primarily by the red blood that runs in all our veins, while the melanin primarily affects the lightness. It can make for a very quick way to get realistic skin tones without having to mess around with colour swatches and colour pickers etc - which typically instead just end up having you target some pre-conceived notions of "ideal" skintones.

I do understand that colourspace conversions (and the complexities of colour in general) make objective measures tricky, but frankly, trusting my eyes is dumber. If there are caveats to its accuracy I really don't mind - when half the planet can't tell if a dress is white and gold or blue and black, I think a tool that can tell us where in a perceptual colourspace a collection of tones are is a huge step forward, even if that step doesn't hit the exact mark.

[sobotka]

Ignoring the L’a’b’ metric here, this is why a singular “axis” for “skin tone” is largely complete rubbish. This is the b’ axis, which is along a loose notion of blue / yellow.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/bjd.12529

Ignoring of course the vast problems with the idea of a singular axis during varying creative looks, colour temperature, filters, etc.

There are much more contemporary lenses to view this under that can likely use the _idea_ of a vector scope without all of the downside trappings.

@dtorop’s comment most eloquently sums it up with:

The goal is to learn something about the image.

[agittins]

I'm not very knowledgable re colourspaces and the like (nor biology, for that matter!) but I am not surprised that skin tone shows wide variability on the b' axis, and I'd guess they chose it for exactly that reason, yes?

I'm totally for any work that finds better tools for representing and matching skin tone, especially those that would be more inclusive.

If such things exist now, then I'd be happy to see them implemented and learn to use them.

If they don't yet exist, do you have a specific objection to implementing an I-axis other than it being a non-ideal tool? The video/film industry seems (from what I can tell) to find it useful for both absolute and relative adjustments of image colour with human subjects.

[agittins]

Hmm... just spit-balling here... but I wonder if the researchers who gathered the colorimetry data cited above would be willing to offer the measured values in order that some work could be done in that area? It seems to be covered here: https://onlinelibrary-wiley-com/doi/pdfdirect/10.1111/j.1600-0846.2007.00223.x (sorry, I don't know if there's an open access version somewhere).

I'm not aware of the existing research so I don't know if such a carefully measured corpus is already available, but it might help build more representative tools. Maybe the I-line can be permanently replaced by a more statistically-representative shape such as a triangle - or perhaps it will show the degree of (in)accuracy of the I-line in a way I've not seen (I have tried to find info on this previously, but the only stuff I was able to find tended to be anectdotal or so old as to be... monoculturally biased).

[sobotka]

Again, that's all well and good except it is exceptionally rare that someone is creatively grading to a colourimetrically ideal value for "skin".

It's a creative grade, so fixed positions aren't helping anyone.

[dtorop]

As an update, https://github.com/darktable-org/darktable/compare/master...dtorop:vectorscope is coming along. It's now working in Lab, which isn't the final plan, but is so much better than a Yuv-variant (and is easier for me to understand) and means that the color actually refers to something. I'll try out CIELUV.

The code is also now cleaner.

It's interesting to read the recent comments. Fig. 2 from the Bino/Bernerd paper is pretty compelling in suggesting that a skin tone line would be very tricky territory to get into for a newly built graticule.

For me, despite having spent a bit of time recently thinking about scopes, I'm not convinced that they could ever be enough. I've found visualizations of the numbers to be a helpful reality check. And scopes are great as instructional tools, to give a dynamic sense of certain attributes of the image. But the way the color works between two images, even those made moments apart with the same camera settings, is so specific to each image, that there has to be other sorts of perception to do good work on an image.

[sobotka]

It's now working in Lab, which isn't the final plan, but is so much better than a Yuv-variant (and is easier for me to understand) and means that the color actually refers to something. I'll try out CIELUV.

All models tend to have some upsides and downsides of course. CIE 1976 has the upside in that it doesn’t deviate too far from the classical spectral locus plot when plotted along u'v', which at least allows people to see the visible light spectral horizon.

It's interesting to read the recent comments. Fig. 2 from the Bino/Bernerd paper is pretty compelling in suggesting that a skin tone line would be very tricky territory to get into for a newly built graticule.

That’s before we even discuss creative grades at all! Imagine putting a blue or green tint to your image.

Perhaps custom peg positions on a plot can help? For example, if someone is going for a Fight Clubby green drugstore look, with CFL vibes, they can sample some skin points in an image they prefer, and carry those points across to other photos as reference anchors?

[johnny-bit]

Perhaps custom peg positions on a plot can help? For example, if someone is going for a Fight Clubby green drugstore look, with CFL vibes, they can sample some skin points in an image they prefer, and carry those points across to other photos as reference anchors?

I think I mentioned similar but it would be great if we could:

• mark some points on any of the scopes as "reference"
• display scope(s) in separate window
• if separate window would be possible: display same set of scopes for "reference" image.