Skip to content

Conan

Overview

  • Install Conan via pipx / (WinGet / Chocolatey / Homebrew / ...)
  • Create Conan profile
  • Define dependencies in conanfile.py
  • Execute conan install
    • Builds dependencies in ~/.conan2/p/
    • Installs files in <build>/generators/
  • Execute CMake configure
  • Execute build

See README for more details.

Example dependency from conanfile.py

def requirements(self):
    if self.options.with_client and not self.options.use_system_qt:
        self.requires('qt/[^6.6]', options={
            'qtnetworkauth': True,
            'qtsvg': True,
            'qttranslations': True,
        })

Packages can be found on ConanCenter.

Profiles, settings, config, options

Conan has multiple types of configurations.

Kind Purpose Profile section Command line switch Example
Setting Setting OS, CPU, build type, compiler, ... [settings] --setting/-s build_type=Release
Configuration Conan or tool-specific (e.g. CMake) configuration [conf] --conf/-c tools.cmake.cmaketoolchain:generator=Ninja
Option Package-defined option [options] --option/-o shared=True

Can be specified per context

  • Host context: For the executable that you are building, so the target machine
    • E.g. --setting:host/-s:h (default, so equivalent to --setting/-s)
  • Build context: For your own machine, so build tools
    • E.g. --setting:build/-s:b
  • Both: E.g. --setting:all/-s:a

Can be specified per package

  • openssl/[~3]:no_deprecated=True (option)
  • pep/*:build_type=Debug (setting)
  • &:build_type=Debug (setting, & = consuming package, in our case pep)

Combine: E.g. build Protobuf compiler as release: --setting:build "protobuf/*:build_type=Release"

Profiles

All can be specified in a Conan profile, default ~/.conan2/profiles/default.

[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=17
os=Linux

[conf]
tools.build:compiler_executables={'c': 'clang-17', 'cpp': 'clang++-17'}
tools.cmake.cmaketoolchain:generator=Ninja

[options]
qt/*:shared=True

Profiles can be chosen on command line via --profile/-pr (=--profile:host) or --profile:build/-pr:b, etc.

Please note that compiler is descriptive, not prescriptive: you indicate to Conan which compiler it is using, not which compiler it should use; that's what tools.build:compiler_executables is for. E.g. if you default compiler (cc/c++) is GCC but you tell Conan it's using clang, it will try to use GCC as if it's Clang, which will cause problems.

Profiles can contain Jinja2 templates, e.g. arch={{ detect_api.detect_arch() }}.

Recipe: conanfile.py

Our conanfile.py is in docker-build/builder/conan/conanfile.py, but symlinked to core repo root for easy use.

Options

class PepRecipe(ConanFile):
    name = 'pep'
    settings = 'os', 'compiler', 'build_type', 'arch'

    options = {
        # Build GUI (with Qt)
        'with_client': [True, False],
        # Build dependencies as shared libraries
        'shared_libs': [True, False],
        ...: ...
    }
    default_options = {
        'with_client': True,
        'shared_libs': False,
        ...: ...
    }

Specified using --option and variants.

Building PEP

def layout(self):
    ...
    cmake_layout(self)

def generate(self):
    tc = CMakeDeps(self)
    tc.generate()

    tc = CMakeToolchain(self)
    ...
    tc.cache_variables['BUILD_CLIENT'] = self.options.with_client
    tc.generate()

def build(self):
    cmake = CMake(self)
    ...
    cmake.configure()
    cmake.build()

Where and how to build PEP. generate creates files for CMake.

Requirements (dependencies)

def requirements(self):
    if self.options.with_client and not self.options.use_system_qt:
        self.requires('qt/[^6.6]', options={
            'qtnetworkauth': True,
            'qtsvg': True,
            'qttranslations': True,
        })

def system_requirements(self):
    apt = Apt(self)
    ...
    if self.options.with_client and self.options.use_system_qt:
        apt.install(['qt6-base-dev', ...])

Build requirements (tools)

def build_requirements(self):
    # Add these to PATH
    self.tool_requires('protobuf/<host_version>')  # protoc

What happens on conan install

  • Conan checks if existing packages in home folder are compatible with those required by conanfile.py.
    • If --lockfile=... is passed, recipes must be in that lockfile.
    • If --update is passed, it updates to the most recent compatible revision.
  • It fetches missing recipes from the remote, default ConanCenter
  • For missing packages, it checks if prebuilt packages are available from ConanCenter, and downloads those
    • Often this is the case for build tools
    • Configurations with prebuilt packages can be found in the 'Packages' tab on a package in ConanCenter
  • It builds missing packages in the Conan home folder
  • Finally, it creates the generators folder in the build folder, and creates/updates CMakeUserPresets.json, aggregating all build configurations

Conan home folder structure

  • ~/.conan2/
    • profiles/
    • p/: package sources & prebuilt packages
      • 5-char-packagename + random hash /
      • b/: packages built on this system
        • 5-char-packagename + random hash /

generators folder

When running conan install, it creates a generators folder in your build folder, which contains:

  • packagename -config.cmake / packagename Config.cmake and/or Find packagename .cmake for all dependencies
    • Used by CMake's find_package( packagename )
  • conan_toolchain.cmake
    • Set compiler options
    • Set CMAKE_*_PATH to the right folders
    • Set CMAKE_VS_DEBUGGER_ENVIRONMENT so that Visual Studio can find DLLs when running a program
    • Set CONAN_RUNTIME_LIB_DIRS to library binaries
  • conanbuild.sh: Source the script (.) to add build tools to path, e.g. macdeployqt:
    • . ./generators/conanbuild.sh; macdeployqt cpp/pep/assessor/pepAssessor
  • conanrun.sh: Source the script to add runtime libraries to path, e.g. QtCore:
    • . ./generators/conanrun.sh; cpp/pep/assessor/pepAssessor
  • CMakePresets.json
    • Set toolchain to conan_toolchain.cmake
    • Set some cache variables like BUILD_CLIENT corresponding to with_client recipe option
    • Set PATH for build tools
    • Set LD_LIBRARY_PATH/DYLD_LIBRARY_PATH for libraries
    • ...
    • This file is included from /CMakeUserPresets.json

The actual binaries and library include files of dependencies stay in ~/.conan2.

Adding a new dependency

  • Find recipe in ConanCenter
  • Use the information provided to add to our conanfile.py in requirements/build_requirements depending on if it's a build tool or not. Use a sensible version range (e.g. pin major version)
  • Look at the recipe's conan graph info (--requires=package/version) or the conanfile.py on GitHub. Are there options we should specify? Put those in our conanfile.py (optional options disabling features we don't need should go though _optional_opts)
  • Add the find_package from the bottom of the ConanCenter page to our CMakeLists.txt. Case is important here! If we require specific components, you can require these using find_package's COMPONENTS as well. If this info is missing on the web page, look at what conan install prints at the end.
  • Link to our target in CMakeLists.txt using the target_link_libraries from the bottom of the ConanCenter page, or specify more specific components if we only need those. Note that the target name can differ from the package name, and case is also important here! If this info is missing on the web page, look at what conan install prints at the end.
  • Re-run conan install, CMake configure, build
  • When pushing, make sure to update docker-build as well

Versioning

  • Exact recipe can be identified with: (most fields are optional)

    plaintext name/version@user/channel#recipe_revision%recipe_timestamp

    • E.g. in conan.lock: qt/6.7.1#3ce06ebc401937e53710f271ce89be6d%1717666193.215
    • Often versions can be ranges, e.g. qt/[~6] (6.x) or qt/[^6.7] (6.x but at least 6.7)
    • Revisions are changes without the underlying source (e.g. qt) changing, e.g. a change in the build method or available options
    • Exact built package can be identified by appending suffix to recipe:

    plaintext :package_id#package_revision%package_timestamp

    • E.g. logged by conan install for a cached package: qt/6.7.1#3ce06ebc401937e53710f271ce89be6d:1167fcab096cf9e0f7e72334b61a4bc00b23eaf4#1eb51fdab5617db31950c7d7de5886f4
    • Depends on settings/config/options
    • Versions can be pinned using lockfiles
    • Exact dependencies can differ per platform/configuration
    • Our Conan lockfile is at docker-build/builder/conan/conan-ci.lock and not automatically used on devboxes

docker-build

Our conanfile.py is in the docker-build submodule, which builds Docker images containing the profile and built dependencies in the home folder. These images are linked to from core via RUNNER_IMAGE_TAG in /ci_cd/docker-common.yml, which should correspond with the submodule commit.

Scheduled pipelines for updates run on master/stable/release of docker-build. master/stable/release of core should thus always refer to the corresponding docker-build branches, to pick up the updated images. Keep in mind that merging in docker-build may create an extra merge commit, and core should then refer to that commit. If new revisions are available, the docker-build CI commits the updated conan-ci.lock and updates core as well.

Dynamic linking & CMake install

Use -o 'pkg/*:shared=True' to link pkg dynamically. cmake --install can be used to copy our binaries plus required dynamic libraries (excluding Qt plugins).

Platform requires

Use [platform_tool_requires] in the profile to prevent Conan from installing tools already present on your system, such as CMake or automake. This can be easily done via ./docker-build/builder/conan/conan_platform_tool_requires.sh, but it can give problems when using lockfiles and sometimes versions conflict or recipes won't work.

Show info on dependencies

Use conan graph info to show info on all dependencies. E.g.:

conan graph info ./ --format=html >./conan.html

Inherit from presets

You can inherit from presets generated by Conan in your CMakeUserPresets.json file. E.g.:

{"configurePresets": [
 {
  "name": "ppp-acc-debug",
  "displayName": "PPP Acc + Debug",
  "inherits": "conan-debug",
  "cacheVariables": {
   "PEP_INFRA_DIR": "~/pep/ppp-config/config/acc"
  }
 }
]}

Reference & issues

Conan docs: https://docs.conan.io/2/index.html.
Full reference: https://docs.conan.io/2/reference.html.

File issues with the Conan tool at https://github.com/conan-io/conan/issues.
File issues with ConanCenter packages at https://github.com/conan-io/conan-center-index/issues.