Bazel Design Notes - Toolchain APIs

This is the start of a series of dev notes on Bazel. Today’s topic: what I wish I had done differently in the Starlark Toolchains APIs.

This is, to be clear, very inside baseball for the Bazel dev community: if you aren’t deeply versed in Configurability and Toolchains this probably isn’t for you. On the other hand, I’ve tried to put in links where it makes sense, so maybe it will be interesting.

Background

I implemented Bazel’s platforms and toolchains systems, and designed the Starlark APIs that rule authors use to control those. I did the best I could at the time (with a lot of help and advice from the rest of the team and the Bazel community), but there’s one bit of the toolchains API I wish I’d done differently.

Currently, to declare that a rule wants to use a toolchain type, it looks like this:

my_rule = rule(
    implementation = _my_rule_impl,
    ...
    toolchains = [ "//some/toolchain:type" ],
    ...
)

Where the //some/toolchain:type label refers to a toolchain_type target.

When the rule actually accesses the concrete toolchain, it looks like this:

def _my_rule_impl(ctx):
    concrete_toolchain_provider = ctx.toolchains["//some/toolchain:type"]["provider"]
    ...

There are a few nuances here.

I’m entirely ingoring exec_groups. They complicate the explanation but don’t change the actual impact.
The toolchain type labels I’ve given are strings, but this can cause problems when external repositories are in play: it’s usually safer to explicitly use Label to make these into label objects and avoid errors with implicit repository names.
The ctx.toolchains accessor returns a ToolchainInfo provider, which allows access to whatever the toolchain rule put into it but is entirely untyped. Some rules just stcik their own provider in there at a well-known name, some put all sorts of data into it like a map, it’s very unstructured.

Is There A Better Way?

Maybe? (As a note, my recollection is that Lukacs Berki advocated for something similar to what I am about to propose. I no longer recall my reasons for disagreeing, maybe they were very good.)

Instead of using the toolchains parameter to rule, instead we could define toolchains as a new attribute type:

my_new_rule = rule(
    implementation = _my_new_rule_impl,
    attrs = {
        ...
        "_some_toolchain": attr.toolchain(
            toolchain_type = "//some/toolchain:type",
            cfg = "toolchain",
            mandatory = True),
    },
)

And then, to use it:

def _my_new_rule_impl(ctx):
    concrete_toolchain_provider = ctx.attr._some_toolchain[SomeToolchainInfo]
    ...

This syntax has a number of advantages (and probably some disadvantages).

Regularity

Regularity is important: this allows accessing toolchain providers in the exact same way as other dependencies’ providers. Instead of using a different system (either ctx.toolchains[type] or the possibly worse ctx.exec_groups[exec_group].toolchains[type]), the same ctx.attr.attr_name syntax is used.

However, the default for mandatory should be True, not False as it is for other attributes, to match the default for config.toolchain_type.

Provider Type Safety

This removes the need for the untyped ToolchainInfo provider: toolchain rules simply return a provider (like every other rule) and the rule using the toolchain gets the provider by type (like every other rule). Much simpler, and Bazel can give more useful errors when the wrong provider type is used.

Exec Groups Become Simpler

I guess I am discussing the exec group APIs, also. Rules would still need to declare the exec groups they use (or use automatic exec groups), but there is an existing system for attributes to declare what exec group they are part of: the cfg attribute. We would probably want to make the cfg attribute mandatory to reduce potential confusion.

I’m not sure what values should mean, though:

Using "exec" seems natural, but isn’t actually correct: toolchains have a special dependency mode where they keep the same target configuration but also force the exec platform to match the original rule’s: this isn’t really the exec configuration.
Using "target" is probably more correct, but doesn’t express the special situation for the toolchain’s exec platform.
The new value "toolchain" is very clear, but this value shouldn’t be allowed for non-toolchain attributes.
Specifying an exec group with cfg = config.exec("link") does seem very clear, however, and should be supported.

Is This Worthwhile?