Back

For comparison, here's more or less what it should look like. Super crappy photos of the screen, but the HDR content here is true HDR (yes, on Linux), and you can get the idea.

@rfnix In my code it's just a standard decode call, as in their example in the readme:

heif_decode_image(handle, &img, heif_colorspace_RGB, heif_chroma_interleaved_RGB, nullptr);

The images I show are either viewed in gwenview, or converted to jpeg with ImageMagick. Regardless of the method, the output is the same.

@wolfpld if you're not saving the ICC profile or CICP/nclx it's definitely going to get displayed wrong. You're still getting an HDR signal here AFAICT.

(Iirc libheif has RRGGBB or something for 16-bit signals?)

@rfnix I just downloaded a picture from the internet and none of the tools I have can display it properly. This is not a "me" problem.

@wolfpld ah sorry that came off as "this is your fault", that was not my intention, I was interrupted and finished the message too quickly...

What I wanted to add is : the state of HDR support in OSS libraries is very very young, most of them concern themselves with getting the raw RGB pixels and leave you hanging there trying to figure out how to interpret them. And unlike SDR where one will get something close when forgetting the colorimetry metadata of a Display P3 image and may not notice it, for HDR it's absolutely necessary.

@wolfpld technically at this point in time you need libheif+LittleCMS if you want to interpet colors correctly in HDR, or reimplement part of the standards e.g. what mpv does in its rendering shaders.

@wolfpld another solution might be OpenColorIO but I'm way less familiar with it...

@rfnix The color theory is definitely a headache generator for me. For now, the plan is to get the HDR data and put it into a tone mapper, which I think expects BT.2020, and this will be vastly superior to what anyone else does, even if incorrect.

libheif exposes color space information, and what you say sounds like the way to do things, but it may be a bit much, at least for now.

@wolfpld it your tonemapper supports BT2020 you should be good, but it probably also has to know about the transfer function (otherwise it will very likely result in under-contrasted images if your input is PQ).

I'm curious about which libs you're thinking of for tonemapping (or are you thinking of implementing one?) - I admittedly don't know that area as well as I'd like. I know the Blender community have stuff like Filmic/AgX but I don't know how accessible they are from outside of Blender.

@wolfpld and yeah, it can be headache inducing, especially since some of the metadata that's required in HDR (CICP notably) is not yet out of draft state for many formats other than HEIF/AVIF so it's very easy to loose track of that when doing conversions...

@wolfpld got it, thanks! Simple and "good enough" until we need more right? ;)

@wolfpld what command line did you use to convert it with ffmpeg?

@rfnix ffmpeg -i in.avif out.png

The resulting PNG is 16 bpc.

@wolfpld ah yes that will definitely not convert/tonemap the image, just dump the BT2020 PQ HDR RGB data (see first screenshot) and thankfully the metadata since the PNG draft spec for CICP is implemented in ffmpeg (second screenshot in exiftool)

(yes, I spent an inordinate amount of time figuring this stuff out...)

@rfnix So the tl;dr is that this color space stuff is quite important actually, just as you said? ;)

I would expect that libheif would do the right thing when it's asked to output 8 bit RGB though...

@wolfpld yes, for HDR it is. PQ and BT2020 are too different compared to stuff that exists in SDR (sRGB/BT709) so they're going to give you completely distorted colors and contrast if you don't account for that somehow.

The unfortunate answer is that there is no "one single" way of downmapping HDR to SDR so libheif doesn't bother. There's a lot of people far clever than me who developed dozens of algorithms for that, and there are none that work in all the cases.

See: https://ffmpeg.org/ffmpeg-filters.html#Tone-mapping (you can use libplacebo in a recent ffmpeg version to do tonemapping, I'm trying to figure out how the colorspace handling work b/c as usual this is very difficult...)

@wolfpld tangentially this is why the HDR peeps are so excited about gain maps (now your SDR image has extra stuff inside to upmap it to HDR in a deterministic way), and why proprietary formats like Dolby Vision have made looots of money (they provide a deterministic way to do SDR <-> HDR conversions between different kinds of displays so that you, as a developer, just tell it "give me the image for _this_ type of display")

@rfnix Ah, something like what iOS does? I read that this particular implementation is largely undocumented and you get different results even if you stick to Apple devices.

@wolfpld Apple has been doing this for a long time, Google recently introduced UltraHDR which works the same way and there are others.

The idea is slowly getting standardized in ISO 21496-1 and the latest draft is already supported if you have the latest iOS. I think libavif has a few experimental flags that let it read the extra metadata even though nothing's finalized yet.

(and yes, the irony of a "standard" behind a paywall is something that irks me to no end each time...)

@rfnix I can just screenshot the mpv window, where mpv does all the heavy lifting to process the color data into something that looks on the screen.

Then KDE does tone mapping to SDR and a lot of detail is lost (e.g. the yellow area is not flat at all) and the colors are off, but it's night and day compared to the flat image I originally uploaded.

@wolfpld having a default tonemapping is actually something that's being discussed, so I have hopes that in a year or two the dust will have settled and this will be implemented ;)

You can also try this, you can experiment with several options for tonemapping and other parameters from libplacebo:

ffmpeg -i input.avif -vf libplacebo=tonemapping=spline:range=full:color_primaries=bt709:color_trc=iec61966-2-1:colorspace=bt709 output.png

@rfnix This looks good.

@wolfpld OCIO is definitely way bigger, maybe the package also brings predefined transformations (e.g. LUTs, which can take some space).

LittleCMS just focuses on reading, writing and executing ICC profiles, but you have to bring your own profiles (or create one on the fly).

- The simple way is one cmsCreateRGBProfile with BT.2020 primaries/whitepoint and PQ tone curve, one cmsCreate_sRGBProfile(), then cmsCreateTransform between the two.

Since you probably need to insert tonemapping in the mix, so you would possibly use a tonemapped version of PQ instead of the real one or something like this.

@wolfpld another option is to implement PQ yourself (it's pretty straightforward actually) to go to linear, then do the tonemapping you want, then go from (tonemapped) BT2020 linear to sRGB using the transform mechanism.

(But that's possibly slower than if you can implement the whole thing using one single cmsCreateTransform.)

@rfnix I recognize some of the terms you are using :|

I'm looking at OpenEXR now, which is always linear. It may define chromacities, and defaults to BT.709 if unspecified. I assume that I can do some transform with lcms between the color spaces, so that the floating point values I output will be well defined, for example in BT.2020?

With the PQ data I will additionally need to linearize it first, before doing the color space transformation.

Does that sound right?

@rfnix Oops, the PBR Neutral tone mapper is designed for BT.709, so I'd want that instead.

@wolfpld exactly!

lcms lets you choose to use float values, and they can be unbounded for each channel (>1). In general the intent should be "relative colorimetric" if you just want to do simple colorimetric conversions (not perceptual/saturation).

I often prototype in Python with colour-science (https://www.colour-science.org), it's slower but the math inside the code is very nice to read and you can use the console!

For example for converting linear BT2020 to linear BT709:

>>> import colour
>>> colour.RGB_to_RGB((0.5, 0.1, 0.5), colour.RGB_COLOURSPACES["ITU-R BT.2020"], colour.RGB_COLOURSPACES["ITU-R BT.709"])
array([ 0.73505646, 0.04684004, 0.54023156])