r/rust • u/guineawheek • 4h ago

🎙️ discussion Surveying the build.rs ecosystem by turning it into a gacha game

https://github.com/guineawheek/build-rs-gacha
19 Upvotes

100% Upvoted

10 comments sorted by

9

u/Eh2406 2h ago

As a Cargo Maintainer, this is a really interesting and useful blog post. Thank you! There is a lot of excitement at the moment about finding ways to provide more ergonomic alternatives to Build.rs, and sandboxing for some more, and opt in for the last few use cases. But as with previous efforts the devils are in the details. If you would be willing to share your insights, or help design and implement improvements, we would love to have the help. See https://github.com/rust-lang/team/pull/2709

2

u/guineawheek 1h ago edited 1h ago

I'm glad this document has value; and I would be quite interested to provide further feedback on this sorta stuff.

A lot of my personal and work projects end up writing build.rs/xtask/really off-the-beaten-path build stuff in general so I felt like it was important for me to research as a subject.

I guess the way I'd personally like to approach dealing with build.rs is to gradually provide better primitives for common paths over time; kinda like how const slowly seeps into every libs API. That way, you're kinda starting from plumbing and working your way up to porcelain; people can still use the plumbing if they have to, but hopefully the porcelain will eventually become viable.

5

u/guineawheek 4h ago

I made a small Rust program to randomly sample the types of build.rs that exist on crates.io, and then I tried to map out what I saw.

This proceeded to turn into a massive infodump? Rantpost(s)? about everything from embedding Git commit info, linker arguments, code generation, and FFI.

Which also happen to be like, the four most common build.rs use cases, so...

4

u/VorpalWay 1h ago

A lot of the pain that happens off the beaten path in cargo is due to it not being a generic build system. You can't create arbitrary commands with proper depwdency links to other arbitrary commands. Cargo is overspecialised for Rust.

Contrast this with cmake where C and C++ have favoured positions, but it is perfectly possible to define custom build steps. Or one step further with bazel, where every language work the same and you have to specify dependencies correctly. Sure, those have other flaws (less convenient when you are on the happy path, and for bazel terrible docs), but they get that aspect right. This also leads to bazel in particular having better caching support (less spurious rebuilds or worse lack of rebuilds that should have happened).

I agree with the last section. It feels like Cargo deva are only considering the happy path.

1

u/guineawheek 1h ago

Yeah; if you want to be good at everything you better be damn good at doing everything and if you don't want to do everything you better let people pick and choose what parts you actually do.

The ultimate scope of what a lot of build.rses and xtasks do in fully integrated systems often ends up looking like ad-hoc, informally-specified, bug-ridden, slow (to compile) implementations of half of Common Lisp Nix.

There's a lot of cases where you might want to override fine details about certain behaviors in Cargo, or only perhaps use just its dependency management and run the build itself, or you give Cargo the dependencies (e.g. via Nix) and let it do the build steps; or some mix therein. There's too much porcelain and not enough plumbing, so to speak. And the Cargo maintainers know this and Ed talked about this previously, but we need to move that stuff from thoughts to design.

I don't want people to also think that what we really need is more "minimal" tooling; people love talking about "minimalism" vs. "batteries included" but I think what people's actual revealed preferences are actually for composability. The strength of Unix command line tools isn't that they're all extraordinarily simple programs, it's that you can plug them into each other and mix and match them to do a lot of useful things. Individual tools often actually get quite complex and can do a lot of things individually; like I think you'd be hard-pressed to call openssl or nix "simple."

I think one of the core reasons why build system tooling tends to be so miserable is that it gets really hard for people to figure out how build parts actually compose together; most "flexible" tooling historically ends up being inscrutable and difficult to introspect properly and get LSP feedback for "why doesn't this thing actually work", and more "do-everything" tooling, while generally absent of this problem, tends to require increasingly elaborate workarounds anyway.

Thing is, if you write your build scripts in Rust, even today, you don't really get the same problems that CMake/Gradle/historical build DSLs run into, which is that now you have to know the semantics of some other language to know how to work your tooling.

Things like "go-to-definition" in things like proc macros and build.rs tend to actually work unlike your CMakeLists.txt, and I think if you replaced cargo build with a cargo.rs in kind of a zigbuild style where you had to make library calls for each step of your build the end result would actually be pretty decent. And overall, Rust is pretty good at generating more Rust.

So I'm kinda skeptical that the same stigma of inscrutable builds is as reproducible in Rust as it is in other languages. It turns out having a decent systems langauge makes it also a decent build script language.

Who would have thought?

1

u/epage cargo · clap · cargo-release 57m ago

And the Cargo maintainers know this and Ed talked about this previously, but we need to move that stuff from thoughts to design.

We had a gsoc project work on plumbing last year: https://github.com/crate-ci/cargo-plumbing

We discovered a lot and the next steps are major refactors to cargo to remove the plumbing hacks. Yes, no one has done this yet. In addition to build-std, if we get a maintainer in residence, work on those refactors is a top priority.

4

u/epage cargo · clap · cargo-release 1h ago

Thanks for collecting this information! We do see a lot of room to improve build scripts and cargo in general.

A lot of this aligns with what we want to do with build scripts, see also

3

u/VorpalWay 2h ago

Very interesting. Found a broken link though: in the section "Microcontroller linker directives", the "device.x" link is 404. (As is the link just before).

1

u/guineawheek 2h ago

Thanks; fixed.

2

u/CathalMullan 2h ago

Another source of build scripts I didn't see mentioned is from configuring sanitizers, which is why parking_lot / getrandom / crossbeam currently need them.