IMPORTED libraries' CONFIG search order is obscure

2021 March 20

Background

An IMPORTED library is a target that represents a pre-existing dependency. It is not compiled, just provides a convenient name for further use in target_link_libraries().

There are various types of imported libraries, mainly:

  • INTERFACE which only specify include paths, compile definitions, etc.
  • regular (mainly, STATIC or SHARED) libraries which correspond to a compiled library file on a disk: .so, .a, .lib, .dll, etc.

This sadness impacts regular IMPORTED libraries.

They have similar properties to regular, non-IMPORTED libraries, but those properties are prefixed with IMPORTED_*, and they exist in multiple variants:

  • config-less, IMPORTED_PROPERTY (e.g. IMPORTED_LOCATION)
  • and config-specific, e.g. IMPORTED_PROPERTY_RELEASE.

The problem is their search order: it is obscure, undocumented, and unreliable by default.

TL;DR: if you want to check the solution: scroll down to MAP_IMPORTED_CONFIG_<CONFIG>.

The default search order of properties

CMake looks for the IMPORTED_* properties in this order by default for e.g. RelWithDebInfo build type:

  • IMPORTED_LOCATION_RELWITHDEBINFO,
  • IMPORTED_LOCATION,
  • if not found at this point, takes hint from IMPORTED_CONFIGURATIONS, e.g. DEBUG;RELEASE
  • IMPORTED_LOCATION_DEBUG,
  • IMPORTED_LOCATION_RELEASE.

This search order is not documented anywhere, see below.

First, the base property is searched, which is

  • IMPORTED_LIBNAME for INTERFACE libraries,
  • IMPORTED_OBJECTS for OBJECT libraries,
  • any of IMPORTED_LOCATION or IMPORTED_IMPLIB where applicable (mainly for DLLs),
  • IMPORTED_LOCATION for regular libraries.

A minor inconveniece is if e.g. IMPORTED_IMPLIB_RELEASE is applicable, and it isn't found, but IMPORTED_LOCATION_RELEASE is found, then CMake will happily use the RELEASE properties.

The search will stop at RELEASE with success, and RELEASE will be the active configuration for this library. Regardless that CMake needs to use only the IMPLIB at link-time (which is missing), and has nothing to do with the LOCATION, which is a requirement only at run-time. So it will try to link to a property value like foo-NOTFOUND, causing build-time error (!) without a single warning. See sadness #2 for more information.

Then, the IMPORTED_* properties will be applied of the same configuration as the base property. Whichever variant of the base property is found, the same variant is applied, so e.g. IMPORTED_LOCATION_RELEASE implies IMPORTED_SONAME_RELEASE [1].

IMPORTED_CONFIGURATIONS is unreliable

This search order is fragile as f*ck, because IMPORTED_CONFIGURATIONS is treated with order as priority [2] while its order is unreliable.

Normally, if you install a target with CMake, it will generate fooTargets-<CONFIG>.cmake files, and those files add <CONFIG> to IMPORTED_CONFIGURATIONS. The fooTargets-*.cmake files are globbed from fooTargets.cmake. Starting from CMake 3.6, the GLOB order is lexicographical, giving you this order if all configs are present: DEBUG;MINSIZEREL;RELEASE;RELWITHDEBINFO.

So in case of RelWithDebInfo build type, and IMPORTED_CONFIGURATIONS DEBUG;RELEASE (co-installed), the DEBUG will take precedence, which doesn't look right: Release seems more fitting.

But this is just a side effect of the auto-generated config files and GLOB behavior, and does not apply if the IMPORTED_CONFIGURATIONS is filled in any other way.

An other way would be Find modules, which are hand-written, mostly in CMake's system modules, but is routinely written and used by CMake users.

A concrete example, showing that this is not just theoretical, is FindZLIB.cmake [3]. As of now (2021, CMake 3.20) if both DEBUG and RELEASE libraries are available, fills it in RELEASE;DEBUG order, so if MODULE is found, the RELEASE takes precedence. But if find_package(ZLIB) happens to find the CONFIG module, the lexicograhpical order will give you the DEBUG libraries.

This behavior dates back to 2.6 [4] and is still present [5]. Minor detail: prior to 3.6, in case of CONFIG mode, this was fully undeterministric, because GLOB order was fully undeterministic [6], probably kept the order that the filesystem gave, so a butterfly's wing flaps decided whether you linked DEBUG or RELEASE zlib.

Solution: use MAP_IMPORTED_CONFIG_<CONFIG>

Luckily for those who want some reliable behavior despite using CMake, the MAP_IMPORTED_CONFIG_<CONFIG> property overrides this search order.

It lets you customize the search order described above, but if you specify ANYTHING in there, IMPORTED_CONFIGURATIONS will be fully ignored [7], so it will suddenly become deterministic.

Note that as of now (2021, CMake 3.20) this is still undocumented, including the full search order and the ignoring, appears neither on the documentation page of MAP_IMPORTED_CONFIG_<CONFIG> [8] nor of IMPORTED_CONFIGURATIONS [9].

The value of the global variable CMAKE_MAP_IMPORTED_CONFIG_<CONFIG> (at the time of add_library()) is the default value for MAP_IMPORTED_CONFIG_<CONFIG>, and the property is applied for IMPORTED targets only.

Note that this is a LIST with order as priority.

Let's say we'll set this to "Release;MinSizeRel;RelWithDebInfo;".

set(CMAKE_MAP_IMPORTED_CONFIG_RELEASE Release MinSizeRel RelWithDebInfo "")

The empty string item in the list stands for the configless property: IMPORTED_PROPERTY. This means when the CMake "generating step" is looking for e.g. the imported library's location, it will consider the location properties in this order:

  • IMPORTED_LOCATION_RELEASE,
  • IMPORTED_LOCATION_MINSIZEREL,
  • IMPORTED_LOCATION_RELWITHDEBINFO,
  • IMPORTED_LOCATION.

Note that if you omit Release from the list, then the *_RELEASE property will be ignored, even if the build type is Release. Yes, f*ck logic, and the documentation for this is terrible. The effective default value for this property is "Release;" (note the empty string in the end). Of course, this isn't documented either, and the actual default is undefined.

If you need to read this article, please check the official documentation, probably they have updated it (since 3.20) to be human consumable: https://cmake.org/cmake/help/latest/prop_tgt/MAP_IMPORTED_CONFIG_CONFIG.html

This effectively instructs CMake that if we're building Release, we can deterministically consume packages that were built as MinSizeRel or RelWithDebInfo or a unknown config too.

Note that this is not always enough: if you use a multi-config generator (like Visual Studio solutions), by default it will generate for all four config types, even if you stick to the one config, one build folder workflow. And, if you want a Debug one, and your package manager fetches Debug binaries, you'll get warnings because the Release is also generated, and there's no suitable IMPORTED_LOCATION for the Release config, because you explicitly excluded Debug.

You have two options to solve this:

  • define CMAKE_CONFIGURATION_TYPES, for a Debug build folder, set it to Debug only,
  • or enable linking Debug libraries in Release, by adding Debug to the map list.
set(CMAKE_MAP_IMPORTED_CONFIG_RELEASE Release MinSizeRel RelWithDebInfo "" Debug)

You should always use MAP_IMPORTED_CONFIG_<CONFIG>

Another minor detail is: other properties relevant to IMPORTED libraries that don't aren't prefixed with IMPORTED_ don't have a variant for _<CONFIG>, so if you want e.g. different (imported) compile definitions per configs with INTERFACE_COMPILE_DEFINITIONS [10], your best try is a generator expression depending on CONFIG [11], but, as you probably expected by now, the $<CONFIG:...> is based on CMAKE_BUILD_TYPE, and it will not always be the one that corresponds to the actually picked binaries (LOCATION).

The $<CONFIG:...> honors MAP_IMPORTED_CONFIG_<CONFIG> but not IMPORTED_CONFIGURATIONS. So you might get IMPORTED_LOCATION_RELEASE with -DFOO_DEBUG [12], possibly breaking ABI, causing hard-to-debug crashes. For example:

struct FooData
{
  int a[64];
#ifdef FOO_DEBUG
  int a_checksum;
#endif
  int b;
};

void bar(const FooData&);

If you are a library consumer, you should always define MAP_IMPORTED_CONFIG_* for this reason.

If you are a library producer, you'd better not (publicly) use generator expressions depending on CONFIG, because if you export those, it works reliably only if the consumer defines MAP_IMPORTED_CONFIG_*. You really shouldn't define it yourself, you can't force the consumer to define it (because the consumer might set the property, and not the global variable, after your export code is processed), and it is unreliable by default.

This property is the only reliable way (as of 2021, CMake 3.20) that CMake lets you consume a package that is built with different build type than the consumer's build type, because the (automatically generated) exported fooTargets-relwithdebinfo.cmake only sets RELWITHDEBINFO properties, so by default, consuming with other build types can fail because e.g. the IMPORTED_LOCATION_RELEASE is empty, and the fallback mechanism is fragile.

References:

Note that many of these are RTFS (Read The F***ing Source). In those cases, I linked a pinned reference (that is pinned to the state at the time of writing), and noted method name, for better future trackability if you want to check latest master.

  1. [1] cmGeneratorTarget::ComputeImportInfo: cmGeneratorTarget.cxx#L6943
  2. [2] cmTarget::GetMappedConfig: cmTarget.cxx#L2135
  3. [3] ZLIB::ZLIB find MODULE: FindZLIB.cmake#L137
  4. [4] cmTarget::ComputeImportInfo: cmTarget.cxx:2938
  5. [5] cmTarget::GetMappedConfig: cmTarget.cxx#L2240
  6. [6] file(GLOB): cmGeneratorTarget.cxx#L6943
  7. [7] cmTarget::GetMappedConfig: cmTarget.cxx#L2208
  8. [8] MAP_IMPORTED_CONFIG_<CONFIG>
  9. [9] IMPORTED_CONFIGURATIONS
  10. [10] INTERFACE_COMPILE_DEFINITIONS
  11. [11] $<CONFIG:...>
  12. [12] Issue #15142