Palette 0.7.7

Somewhere, deep in the forest, stands a boulder. Its surface covered in moss and lichen, with mushrooms growing on top of it in the late summer and fall. Some say it was there already before the forest grew around it, gently left behind by melting glaciers. Other believe it had been thrown there by a giant. All anyone knew for sure, however, was that it had not changed much at all in recent memory.

Then, one day, something did change. The mossy boulder opened its eyes and spoke to itself: “Oh, I must have dozed off…”

It’s time for another Palette update and it has been quite a while. This library has even had the time to turn 10 years old! The first commits were made 2015-11-29 and added the RGB, Luma, XYZ and L*a*b* color spaces, plus a dynamic Color type that was later scrapped. Humble beginnings.

For those who aren’t familiar with it, Palette is a Rust library for working with colors, including color conversion and color management. Its features range from common tasks, like color blending or converting HSL to RGB, to more advanced topics. It leverages Rust’s type system, by encoding color space information as type parameters, to be able to prevent mistakes and to keep it open to extension and customization.

The State of the Crate

Much like the mossy boulder in the little story above, Palette isn’t a fast-moving project. Like most other open source projects, it’s susceptible to life situations. Such as variations in one’s day job, or one’s desire to go outside and look for dragonflies instead. Luckily, the concepts it implements don’t really change much either. RGB is still RGB.

What does change over the course of a couple of years is the crate ecosystem, similar to how to forest grew around the boulder. This means the third party integrations with crates like rand and wide will be a bit outdated. But fear not, for there’s a plan!

The focus in the nearest future will be to update dependencies, shed some unnecessary ones, maybe remove the deprecated features, then release that as Palette 0.8.0. No new features, maybe a couple of bug fixes, otherwise just a set of low-friction breaking changes to bring things up to date. Exactly what will change will be detailed when we get there.

Looking further ahead, there are more intrusive changes that need to be done. That means the next focus after “modernizing” things in 0.8.0 will be an immediate jump to version 0.9.0. This version will be more focused on refactors and allowing the library to grow in a more sustainable way. There may of course be bug fix releases after 0.7.7 and 0.8.0 along the way, but they will not be prioritized for other changes.

But now, let’s look at what has been in the slow cooker since last time.

MSRV Increased to 1.71.0

Due to the MSRV of some dependencies changing, Palette’s MSRV has been increased to 1.71.0. That should hopefully still be old enough for the more conservative environments.

Gamma Encoding LUTs And RGB Spaces

One of the core features of Palette, and the reason why it exists to begin with, is the tools for converting RGB values between linear and non-linear representations. This shows up time and time again, in photo editing software, video game graphics, graphical user interfaces, and so on. And since it’s such an important operation, it’s also good to be able to do it fast.

Palette has so far relied on fast-srgb8 for speeding up this operation for sRGB. It provides a lookup table that’s indexed by the gamma encoded (non-linear) u8 values and contain the corresponding floating point values. But that’s the easy part. It does also provide a reverse lookup table that converts floating point values back to u8. There’s also support for converting multiple values at once with SIMD. A very neat crate. Go check it out after reading this post!

use palette::{LinSrgb, Srgb};

// Changing both the component type and gamma at the same time is faster than
// changing them separately:
let srgb_u8 = Srgb::<u8>::from_linear(LinSrgb::new(0.3f32, 0.8, 0.1));

The only downside in the case of Palette is that fast-srgb8 is limited to sRGB. What about all the other RGB standards? While adding support for a few of those other RGB standards to Palette, GitHub user @Kirk-Fox also put in a ton of work to add lookup tables for them. These tables use a variant of the same reverse lookup algorithm as in fast-srgb8, based on the original C++ code by Fabian “ryg” Giesen. Unfortunately, this means fast-srgb8 has been dropped as a dependency, as it didn’t fit into this generalized set of tools.

use palette::{LinDciP3, DciP3, Srgb, ProPhotoRgb, encoding, math::lut::VecTable};

// This is now possible and just as fast as for sRGB:
let p3_u8 = DciP3::<u8>::from_linear(LinDciP3::new(0.3f32, 0.8, 0.1));

// The pre-generated lookup tables can also be accessed directly:
let lut = encoding::Srgb::get_u8_to_f32_lut();
let linear = lut.lookup_rgb(Srgb::new(23, 198, 76));

// ProPhoto RGB uses `u16` for its wider gamut, so its tables aren't pre-generated.
// We can however generate our own table, here allocated as a `Vec`:
let lut = encoding::IntoLinearLut::<_, f32, _, VecTable>::new_u16();
let linear = lut.lookup_rgb(ProPhotoRgb::new(23, 198, 76));

The newly added RGB standards are Rec. 709 and Rec. 2020, Adobe RGB (a98-rgb in CSS), DCI-P3, DCI-P3+, Display P3, and ProPhoto RGB. This makes Palette match CSS Color Module Level 5, apart from the lack of CMYK.

LMS Color Space

Support for the LMS color space has been added, as the Lms type. It’s often used as a model of the stimuli the eye’s cone cells get, but should be taken with a grain of salt. The word “model” is load bearing in that sentence and its accuracy varies a lot from case to case. Those cases can be anything from white balancing to color vision deficiency simulation. It’s not something everyone needs but having a representation of it is useful.

Palette provides two conversion matrices (VonKries and Bradford) and corresponding type aliases, similar to the RGB standards:

use palette::{lms::VonKriesLms, white_point::D65, Srgb, FromColor};

// Von Kries LMS from sRGB:
let lms_from_srgb = VonKriesLms::<D65, f32>::from_color(Srgb::new(0.3f32, 0.8, 0.1));

Reworked Chromatic Adaptation

A tie-in to the LMS color space is the new implementation of chromatic adaptation, which for example is used in white balancing. This process changes the color values to match a different reference white point. Essentially, adapts the color to which light spectrum is considered “white”. For those who remember the blue/black or white/gold dress debate, this is very related!

For example if you have a color that’s adapted for reference white point A and want it to appear the same relative to reference white point C, you may use:

use palette::{
    Xyz, white_point::{A, C},
    chromatic_adaptation::AdaptFromUnclamped,
};

let input = Xyz::<A, f32>::new(0.315756, 0.162732, 0.015905);

// Will convert Xyz<A, f32> to Xyz<C, f32> using Bradford chromatic adaptation:
let output = Xyz::<C, f32>::adapt_from_unclamped(input);

So far, this isn’t much different from the old system. What’s new is the generalized adaptation_matrix function. It’s both more flexible than the AdaptFromUnclamped and AdaptIntoUnclamped traits and it constructs a reusable Matrix3. Using this function, the above example would become:

use palette::{
    chromatic_adaptation::adaptation_matrix,
    lms::matrix::Bradford,
    convert::Convert,
    white_point::{A, C},
    Xyz,
};

// Adapts from white point A to white point C:
let matrix = adaptation_matrix::<f32, A, C, Bradford>(None, None);

// Explicit types added for illustration.
let input: Xyz<A> = Xyz::new(0.315756, 0.162732, 0.015905);
let output: Xyz<C> = matrix.convert(input);

A bit more verbose, but lets you call matrix.convert(input) over and over, without recalculating the matrix values. adaptation_matrix can also accept arbitrary reference white points. This becomes useful when the color space stays the same and we instead want to change the color’s appearance:

use palette::{
    chromatic_adaptation::adaptation_matrix,
    lms::matrix::Bradford,
    convert::{FromColorUnclampedMut, Convert},
    Srgb, Xyz,
};

/// Just an example, don't expect it to be good!
fn simple_white_balance(image: &mut [Srgb<f32>]) {
    // Temporarily convert to Xyz:
    let mut image = <[Xyz<_, f32>]>::from_color_unclamped_mut(image);

    // Find the average Xyz color:
    let sum = image.iter().fold(Xyz::new(0.0, 0.0, 0.0), |sum, &c| sum + c);
    let average = sum / image.len() as f32;

    // Construct a matrix that makes `average` grey/white and adjusts the
    // other colors accordingly:
    let matrix = adaptation_matrix::<_, _, _, Bradford>(Some(average), None);

    for pixel in &mut *image {
        *pixel = matrix.convert(*pixel);
    }
}

This deprecates the AdaptFrom, AdaptInto and related items. Note that the new AdaptFromUnclamped and AdaptIntoUnclamped traits are currently only implemented for Xyz, to avoid surprising side effects from color space conversion.

Matrix3, Convert And ConvertOnce

As demoed above, there’s now a Matrix3 type that represents a 3-by-3 matrix that can be applied to color values. It’s statically typed, meaning it knows what it converts from and to, and multiple matrices can be composed into one:

use palette::{
    Xyz, Srgb,
    convert::{Convert, Matrix3},
};

// Multiple matrices can be combined into one:
let matrix = Xyz::matrix_from_rgb()
    .then(Matrix3::scale(0.5, 0.5, 0.5));

let rgb = Srgb::new(0.8f32, 0.3, 0.3).into_linear();
let scaled_xyz = matrix.convert(rgb);

It comes with the Convert and ConvertOnce traits, for performing the actual conversion. The idea is, as demonstrated, to have a system for converting values in bulk without having to rely on the optimizer figuring out what can be reused. For example, it’s also integrated with the gamma lookup tables and the CIE Cam-16 color spaces.

Named Color Iterators

Iterators have been added for the colors in palette::named. They can be iterated as names, colors or name-color pairs, in arbitrary order.

use palette::Srgb;

let red = Srgb::new(255u8, 0, 0);
let red_entry = palette::named::entries().find(|(name, color)| *color == red);

To make things simpler, the from_str function has also been made part of the "named" Cargo feature, instead of being toggled by the separate "named_from_str" feature. The lookup structure for from_str is also the backing collection for the new iterators, so it didn’t make as much sense to have a separate feature for it anymore. To compensate for the added cost of having the lookup structure, it’s now pre-generated instead of using a build script and a proc-macro. More on that in the next topic.

This makes the separate "named_from_str" Cargo feature redundant and it will eventually be removed. Use "named" instead.

Code Generation And palette_math

The addition of the lookup tables prompted a few other changes. First, the build script has been removed in favor of ahead-of-time code generation. What would have been the build script is instead an internal tool that you, the users of Palette, don’t need to interact with. You get the generated code files packaged and ready.

The initial reason for this change was that the gamma tables were a bit too heavy to generate in their initial iteration. This changed later and today’s version may have worked as a build script, but there’s not really any reason for Palette to require everyone who uses it to recalculate the same numbers. That time and power is better spent on other things.

The pilot feature for the code generation tool was to build the lookup structure for the palette::named colors. Its impact as a build script was relatively small, but not having a build script at all is a better situation. Especially among concerns for supply chain attacks. The same goes for the proc-macro that generated the table. It was also removed and the phf_codegen crate is used in its place.

To solve the chicken-and-egg problem of generating the gamma tables using Palette itself, the palette_math crate is being introduced with this release. It’s currently very bare-bones but it will later be where the low-level math and algorithms are implemented. The idea is also that those who don’t need all the fancy types should be able to depend on it instead of palette. But we aren’t there yet. For those who want to use both palette and palette_math, it’s re-exported as palette::math.

Other Changes

• encoding::gamma has been deprecated and will be removed. It’s error prone and incorrectly implemented.
• Rgb::from_hex has been added as a more explicit way of parsing hexadecimal strings. Support for parsing u16, u32, f32, and f64 components have also been added.
• Hue ::from_degrees and some palette::cast::* functions are const fn.
• normalize_unsigned_angle should behave better now for very small numbers.
• Conversion into HSLuv when L is near or below 0, or near or above 100 should no longer produce strange values.
• Documentation improvements.

Wrapping Up

I want to give a big thank you to everyone who contributed or took part in discussions. Your help is always appreciated!

Palette can be found on GitHub, and on crates.io.

Thank you for reading! And waiting!