Wayland (the protocol and architecture) is still lacking proper consideration for color management. Wayland also lacks support for high dynamic range (HDR) imagery which has been around in movie and broadcasting industry for a while now (e.g. Netflix HDR UI).
While there are well established tools and workflows for how to do color management on X11, even X11 has not gained support for HDR. There were plans for it (Alex Goins, DeepColor Visuals), but as far as I know nothing really materialized from them. Right now, the only way to watch HDR content on a HDR monitor in Linux is to use the DRM KMS API directly, in other words, not use any window system, which means not using any desktop environment. Kodi is one of the very few applications that can do this at all.
This is a story about starting the efforts to fix the situation on Wayland.
This is a great article to read – and an important topic, too. Colour management and HDR should be a core aspect of Wayland, and these people are making it happen.
> Wayland (the protocol and architecture) is still lacking proper consideration for color management.
How is this possible? It’s not like color management is some cutting-edge feature they couldn’t have anticipated. I would have expected it to be a core part of the system from the beginning.
What other major oversights did they miss in the design of Wayland?
I’d expect that compatibility is the reason.
For a new OS design you can choose a standard color representation for everything (e.g. when I went through this last, I ended up choosing linear/”not gamma encoded” CIE XYZ with D65 as the white point, partly because it’s additive but has a much wider gamut than any variation of RGB can handle). Then you can say “all software for this OS must use the standard color representation” and have video drivers convert into whatever color space the monitor happens to use (and whatever pixel format/”bits per pixel” the video mode happens to want); including taking HDR and a “virtual iris” into account, and including doing gamma encoding properly last; possibly using GPU for the color space conversion, and possibly doing dithering to reduce “color banding”.
For an ancient OS design (Windows, *nix) where almost all existing software already uses “random who-knows-what RGB” you’re mostly screwed before you start. The best you can do is say “everything should assume sRGB (with no HDR) even though it’s probably wrong and isn’t additive”; and then add an extra layer of horrible hackery (a special purpose API) for software that needs to get colors and/or HDR right.
Note: for “additive”, I mean that you can add 2 colors and get a correct result with pure addition alone; which matters for things like anti-aliasing and blurs (e.g. you can do “result_red = red1 * alpha + red2 * (1.0 – alpha); result_green = …” and get a correct result). To get a correct result for sRGB you need to do something like “result_red = gamma_encode( gamma_decode(red1) + gamma_decode(red2) ); result_green = ….” which is awful (expensive and has rounding/precision loss concerns) so most people don’t do it and just accept an incorrect/”slightly wrong” result (and often programmers don’t even realize that their code creates incorrect/”slightly wrong” results and merely assume its additive when it isn’t).
Compared to a “clean slate redesign without any concessions for backward compatibility”?
I’d say that using “surface described as array of pixels” as the basis of their interface/protocols is extremely bad to begin with. I’d want a higher level interface (where objects are described by lists of commands for the renderer; like “object = draw_box, x1, y1, x2, y2, colour1; set_font “arial”; draw_text x3, y3, “Hello word”; …” ) that provides resolution independence; where renderer does occlusion tests before doing all the rendering itself (using GPU as much as possible), and where renderer can decide to use whatever resolution it thinks is a good compromise for all intermediate surfaces.
I’d also say that 2D should be an afterthought (treated as “3D with constant Z”); and that it’s a mistake to assume that the display device is a flat rectangle (and will never be a device like Hololens, Google Glass, Occulus Rift, etc).
Gamma correction is a good way to allocate more bits to the dark colours (where the human eye is more discerning) and allows for getting life-like colours with just 8-bits per channel (not just in RGB but also YCbCr, where not all combinations correspond to valid RGB colours so the bits are more precious). Then there is RGB 15-235 (an actual ugly hack), where not all RGB combinations correspond to unique colors, so again your BTW bits are more precious. Bits are always precious in video, even in 10-bit color.
But since we live in an era of “simplicity for the coder above all” and with a complicit hardware industry cheering for that attitude, I don’t think the people of today can understand what a piece of good engineering digital gamma correction was. Use a library to add colors. Also yes it takes more time computationally, but it does it on what is heavily compressed images and videos (most of the time) in need of decompression anyway. Your intermediate uncompressed format can be high-bit-depth and non-gamma corrected.
If you want to write code that handles image or video bits, you have to learn the details, such as what those bits represent. If dealing with video, learn what interlacing is (hint: it’s not a cute way to lay the lines of a frame in memory, also all the native 50i and 60i content won’t magically go away)
your BTW bits = your bits
All current display technologies including Hololens are a flat panel of square pixels. Tough. Live with it. You certainly can code EGL operations onto a Wayland surface and when it is ready to display the compositor will flip it for you.
If you want higher level layers of abstraction you currently use a web browser. There’s even VR layers for HTML/CSS.