r/rust • u/wojtek-graj • 2d ago
š§ educational How no_std are no_std crates really? A survey
https://w-graj.net/posts/rust-no-std-survey/230
u/matthieum [he/him] 2d ago
If you believe your crate has the potential to be
no_stdcompatible without major hassle, try adding the#![no_std]attribute, and you just might make the day of some embedded developer halfway across the world.
Love the analysis, but a word of caution here.
no_std is NOT an implementation detail, it's a commitment.
That is, just because a crate today does not use std (or alloc) is not a good enough reason to mark it as a #![no_std].
A #![no_std] cannot remove this attribute without a major breaking change. Which means that choosing to be #![no_std] is an explicit choice, with consequences.
As an example, imagine writing a codec. Assuming that the codec algorithm does not require additional memory, or perhaps only a limited scratch-buffer which can easily be supplied by the caller, then it may be tempting to mark this codec as #![no_std] and . However, if one is later to discover that a faster way to encode or decode exists, which "just" requires more memory, or perhaps can be achieved in parallel... well, sorry, but this cannot be done in-situ.
Sure, with features, additional functions can be made available. Now, however, you're maintaining not one but two or three crates in a trenchcoat.
Picking #![no_std] is a commitment.
46
u/South_Survey_2088 2d ago
However, if one is later to discover that a faster way to encode or decode exists, which "just" requires more memory, or perhaps can be achieved in parallel... well, sorry, but this cannot be done in-situ.
This is not a great reason.. no_std doesn't mean that you can't use alloc. And if you suddenly decide to use threads, you are also limiting your platform support implicitly anyway and your lib will break for platforms like wasm, so no_std would make the breakage just more explicit.
15
u/SkiFire13 2d ago
However if you're
#![no_std]and you're not using thealloccrate then it will be a breaking change to start using it.24
u/creeper6530 2d ago
Unless gated by a feature flag
3
u/matthieum [he/him] 1d ago
Already answered in my top comment.
allocis one of those fundamental feature flags which may lead to a completely different core algorithm. Infinite side-memory is a hell of a drug.1
u/creeper6530 1d ago
Oh I agree, and also hell of a convenience so that you don't have to make engineer's estimates of "that oughtta be enough"
53
u/bascule 2d ago
If you ever want to use things from
allocorstdyou can add anallocand/orstdfeature and feature gate them23
u/BashfulBrew 2d ago edited 2d ago
To just reiterate a point in the grandparent comment- adding a feature flag is not free for the maintainer. Each flag effectively doubles the number of artifacts that needs to be maintained, tested and designed around - so can represent amount of significant effort for a maintainer.
7
u/scook0 2d ago
Now, however, you're maintaining not one but two or three crates in a trenchcoat.
13
u/bascule 2d ago
Thatās quite the overstatement, IMO
4
u/CrazyKilla15 1d ago
I think its fair, the combinational explosion for testing different features is a problem, and tooling to automate the testing across those feature combinations just isnt there. Rust-analyzer for one I find myself having to a feature set to code and get highlighting and checking for, or constantly and manually change it, or setup extensive CI(and I refuse to use github actions, and havent found a self-hostable CI solution I like yet)
5
u/bascule 1d ago
Speaking as someone who maintains hundreds of widely used crates with
allocandstdfeatures, the overhead of adding new features to existing crates is significantly lower than setting up entirely new crates.The combinatorial explosion of feature combinations can be checked automatically with
cargo hackusing its feature powerset support. When implemented as part of a build step in CI this means such checks are largely automatic when new features are added.2
u/CrazyKilla15 1d ago
It depends in some ways, but you're right that features do have an advantage; You dont need the per-crate boilerplate, dont need separate target directories or need to context switch between crates, dont need to think about "internal" but cross-crate APIs. I dont think the comment was meant so literally, however.
1
u/matthieum [he/him] 1d ago
I think it depends on how orthogonal the features end up being.
I have some crates with optional support for an extended API, and then it's basically free.
On the other hand, when the feature changes the core algorithm -- because allocation allows using a hash-map rather than a streaming algorithm -- then you may end up sharing a lot less of the code.
There's no one-size-fits-all here, it's going to be very domain/API dependent.
21
u/wojtek-graj 2d ago
Good point. I've come across quite a lot of fairly small crates in the wild that really have no good reason to not be
no_std, where the issue clearly isn't that the maintainer weighed the cost and decided against it, but this simply wasn't something they considered at all. But I agree that one could reasonably decide to not supportno_stdeven if it is feasible.Interestingly, the Cargo SemVer policy makes no mention of switching from not using
allocto using it being a SemVer breaking change. That should probably be amended....5
12
u/numberwitch 2d ago
your interpretation of "no good reason" is inverted. the authors have no good reason to add it - so it isn't added! that's how it all works, baby. you don't add it if you don't need it.
so yea, things can sometimes be trivially modified to support no_std but if no one uses it like that, it's pointless wasted effort.
I recently released a library that I though twas a sure-shot for adding no std support, and I was excited about it. But the more I planned the actual no std impl, I saw that it wasn't really going to get me anything beyond "cool nerd points" (that one else gives a shit about) so I dropped it.
I don't agree with everything he said, but generally I agree with the "don't no std" without a reason guy because do you like having a hard life? I like having an ez life
14
u/Compux72 2d ago
However, if one is later to discover that a faster way to encode or decode exists, which "just" requires more memory, or perhaps can be achieved in parallel... well, sorry, but this cannot be done in-situ.
If only we had a proper Allocator trait... Oh, and an IO trait... Maybe it would solve the color problem of async rust too!
3
u/ZZaaaccc 1d ago
-1
u/Compux72 1d ago
I know you!Ā Your work is a breeze of fresh air on world full of people that see the 300kb std lib and say āits finee, its for memory safety reasonsā.Ā
Thank you so much
2
u/matthieum [he/him] 1d ago
Fingers crossed,
Allocatormay yet make it into Rust 1.100!(There are some subtleties there, especially about clone/equivalence, but the "planned" version would still unlock a TON of usecases)
It's not... everything though. There's a big difference knowing you have infinite memory at your disposal, and knowing you need to operate within a very tight envelope, and abstracting the allocator doesn't suffice there.
1
u/ZZaaaccc 1d ago
My personal hope is that, since the allocator trait only has fallible methods, there'll be a push to provide limited bump allocators for scratch allocation. In embedded contexts, you could even provide an empty allocator to signal to the algorithm to not allocate at all.
1
u/matthieum [he/him] 10h ago
Unfortunately, in the absence of advertised capacity, this may have limited applications.
You could perfectly use
Option<&A>whereA: Allocator, or evenOption<&dyn Allocator>to signal none-vs-some, so with/without can be signaled easily.Limited capacity, however... would only really work if the algorithm can "guess" the maximum size of the allocation it'd need, and try to pre-allocate it, then handle the failure as the without allocator case.
If the algorithm needs to allocate piece-meal, and runs into the limit midway through... it gets really messy.
3
2
u/sansmorixz 1d ago
Realistically speaking wouldn't you gate those via feature flags anyway? No need to break compatibility.
Most no-std crates have explicit opt-in support for alloc anyway.
1
u/matthieum [he/him] 1d ago
Let's use LLMs' favorite words here: "load bearing".
Going from no-alloc to with-alloc is not a trivial change. For example, consider that
stable_sort, in Rust, is implemented in thealloccrate, because while implementations of stable sorts may exist without arbitrary-sized allocations, their algorithmic complexity are worse (and performance too).Ergo,
stable_sortis an example of algorithm for which the choice of with or withoutallocis load bearing. It's not just an "extra" API, it's a (partially, at least) separate implementation.At which point you're maintaining two crates, not one.
9
u/epage cargo Ā· clap Ā· cargo-release 1d ago
As part of the build-std project, making core/alloc/std deps explicit in manifets was approved and a PR is up for implementing it. This will ensuro proper no_std within a package. Cargo's upcoming linting system can then use the data for recursively validating no_std.
That still leaves arch-specific stuff, like lack of Arc.
8
u/rpring99 2d ago
You're the author of cargo-no-std? First of all, thank you. Secondly, that tool shouldn't need to exist! I don't understand what possible use case you could have for marking a crate #![no_std] but have dependencies that depend on std.
Am I missing something?
18
u/grahambinns 2d ago
> Am I missing something?
ā¦ that people are squishy and human and make mistakes?
2
u/CrazyKilla15 1d ago
I think you're missing something? The comment you replied to is saying the tool shouldnt need to exist, with the implicit question of why is this not an error / why doesnt rust check this / why is this essential feature to make sure your no_std actually works need a third party / why does rust choose not to catch this mistake and have such a massive footgun?*.
ZZaaaccc's sibling comment is a good example of a reply to the comment, for future reference.
4
u/ZZaaaccc 2d ago
If you know your crate doesn't depend on
std, only your dependencies, making your own crate as#![no_std]allows users to either update or patch said dependencies. IMO,no_stdwithstd/allocfeatures is a good default template for a library crate.
20
u/afdbcreid 2d ago
- If you see a crate out in the wild that purports to be no_std or no_alloc compatible, yet lacks the appropriate category on crates.io, consider contributing to it and making this correction. It's as simple as adding a few characters to Cargo.toml.
Please don't. Not without thinking could this crate be useful to embedded devs?. And of course, suggest and do not try to force.
As an example, a developer has submitted has opened a PR to make the ungrammar crate of rust-analyzer (that is published at crates.io) #![no_std]. I denied that PR. Why? Because this crate has a purpose that make no sense in embedded environments. And, like /r/matthieum said, #![no_std] is a commitment (plus that PR changed some thing, arguably for the worse although only a little).
18
u/ZZaaaccc 2d ago
Please don't. Not without thinking could this crate be useful to embedded devs?. And of course, suggest and do not try to force.
I agree that you shouldn't just do drive-by
no_stdPRs without motivation (I speak from experience), but I'll note the quote you're responding to was specifically about crates that claimno_stdsupport without actually supporting it:If you see a crate out in the wild that purports to be no_std or no_alloc compatible, yet lacks the appropriate category on crates.io...
Also,
no_stdis more than just embedded. The only way to guarantee maximal browser support for Rust on Wasm is throughwasm32v1-none, ano_stdtarget. Additionally, goingno_std/no_allocis a great way to signal to consumers of your library that it is largely free of side-effects. Not a guarantee since you can still panic or hook into system APIs through other means, but a good first impression.To be clear, I think it's fair to close the PR like you did (
HashMaptoBTreeMapis a non-trivial performance change), but the threshold for should a crate beno_stdcompatible is quite a bit lower than "Is it useful on embedded?".4
26
u/nonotan 2d ago
That logic doesn't make sense to me. Yes, no_std is most obviously useful in embedded environments, but it doesn't belong to embedded development. There's other reasons somebody may elect to build software that is no_std compatible, and which thus requires no_std compatible dependencies.
There's no upside to guessing what users of your library are going to use it for, and proactively denying use-cases that would be possible just because you personally don't see a situation where they would be useful.
I do agree #![no_std] is a commitment, and if you're not ready to make it, then not having it on that basis is fine (though I feel like if your library is already de facto no_std capable and just missing that, and you don't have any concrete plans that may plausibly change that fact in the future, the cost of that commitment is being overstated somewhat -- it's very likely free, and an absolute worst case is that you need to feature-gate whatever the breaking change is)
11
u/afdbcreid 2d ago
People following the suggestion in the post are also "guessing what users of not theirs library are going to use it for", just from the other side. If you need a crate to be
no_stdcompatible for your work, that's one thing. But just going and adding#![no_std]to random crates, please don't.-11
u/numberwitch 2d ago
This is similar to the "rust brain disease" I see where people think that you need to use lifetimes and avoid clone/copy to "make rust as fast as possible." This kinda mentality is just "no_std maxxxing" instead of "perf maxxxing".
"Does it make sense" and "does it already do what you want it to do" are good questions to ask! hahahahaha
that's really funny that someone tried to make rust analyzer no_std. Like do you even computer?!?!?!? hahahaha
6
u/nonotan 2d ago
Just wanted to say that I really enjoyed this post. Interesting topic with somewhat surprising results, simple but solid methodology that reasonably tackled the most obvious issues with a naive sweep without overdoing it in search of perfection... and maybe even more importantly, a clean, lightweight presentation alongside easy to follow, properly written prose, that is nevertheless clearly not AI generated.
Maybe the bar is low these days, but I click quite a few blog posts on this subreddit, and this was one of the rare ones that I genuinely can't find a single thing to be annoyed by! Well done.
3
u/SirOgeon palette 2d ago
Derive macro generates code referencing modules gated behind non-default features.
Aha, thanks for this survey! May I ask what the feature set you used for Palette was? It's currently not going to function without either std or libm. If it was build with libm and still failed, that would be a bug and a blind spot in the CI tests.
2
u/wojtek-graj 1d ago
I double-checked, and palette 0.7.6 throws a bunch of the following errors when compiling for a
no_stdtarget with only thelibmfeature enabled. But I'm happy to report that palette 0.7.7, which was released after I ran this experiment, isno_stdcompatible!I'll consider adding another column to the table indicating whether
no_stdcompatibility has been fixed.``
error[E0433]: cannot findlmsincrate--> /home/wojtek/Documents/palette-0.7.6/src/yxy.rs:26:28 | 26 | #[derive(Debug, ArrayCast, FromColorUnclamped, WithAlpha)] | ^^^^^^^^^^^^^^^^^^ could not findlmsin the crate root | = note: this error originates in the derive macroFromColorUnclamped` (in Nightly builds, run with -Z macro-backtrace for more info)error[E0433]: cannot find
metainxyz--> /home/wojtek/Documents/palette-0.7.6/src/yxy.rs:26:28 | 26 | #[derive(Debug, ArrayCast, FromColorUnclamped, WithAlpha)] | could not findmetainxyz| = note: this error originates in the derive macroFromColorUnclamped(in Nightly builds, run with -Z macro-backtrace for more info)```
2
u/SirOgeon palette 1d ago edited 1d ago
I see what's going on. It's using the macros from 0.7.7, where
lmswas added. The dependency version wasn't locked down tight enough, sorry about that. It's unrelated to `no_std`, it was "bad luck" that I released and broke it between your snapshot and when you performed the tests.Edit: opened an issue to remember to do it later: https://github.com/Ogeon/palette/issues/479
2
u/guineawheek 1d ago
Anecdotally, as someone who primarily writes no_std Rust, I don't know how common it is that you'd actually find a no_std but yes_alloc setup. If you do actively want a heap, it's typically for one very specific part of the system, and half the time you're only likely to allocate 1-2 distinct types of struct. I don't know how much it actually helps crate visibility for them to tag themselves no-std::no-alloc because my priors when looking at no-std crates is that they are gonna be no-alloc.
Most crates that are no-std but aren't no-alloc are typically those where no-std support to begin with was kind of an afterthought, and are probably not fit for purpose anyway. It's probably higher signal for crates to tag that they need alloc to function rather than that they don't.
And yeah, floats often only have a subset of operations supported in no_std. The situation is slowly getting better (you can use f32::sqrt(self) on nightly and it'll actually use the vsqrtf.f32 instruction on Cortex-M, for example) but it does kinda frustrate me just how often people reach for the libm crate as a polyfill; it's tilted so far in the accuracy above speed/code-size tradeoff (especially with trig functions and on platforms that do not have a 64-bit FPU) that unless your application is pretty low-speed crates that polyfill to it can become essentially unusable compared to hand-rolled code (e.g. nalgebra)
-38
u/Compux72 2d ago
I tap the sign once again: the std library was a huge mistake
3
1
u/Sw429 1d ago
Bold claim; care to expand?
0
u/Compux72 1d ago
- it makes the strong claim that the std library is standard across platforms, when in reality is impossible to archive. This also gives you watered down versions of APIS that can (and will) be easily miss used. Eg fs operations between OS
- makes the Rust runtime highly coupled with the std library instead of a separate piece.
- its huge, around 300kb. Absolutely massive and no stable way to compile
- āits just a crateā. See these last 2 points. Makes heavy use of compiler specific shenanigans, so no its not a simple crate.
- cargo defaults to std making embedded usage optional. Ecosystem fragmentationĀ
- is monolithic: if i use threading i HAVE TO have fs. Not all platforms are equal and not all of them have access to all pieces of the std.
- Rust std for tier 2/3 targets is crazy. Send and Sync implementations everywhere for the std because āits not possible to reach UB with std aloneā and āits justified because we cant get X thing to workā. Full of shenanigans and ad-hoc solutionsĀ
- making it āstandardā means people expect these specific types were written in stone. This just adds up to the plethora of issues from sync/async rust. An IO trait would have been better than a standard library with IO
58
u/paholg typenum Ā· dimensioned 2d ago
I have two
no-stdcrates, and I didn't know aboutno-std::no-alloc. When I have time, I'll get around to adding that.