FreeBSD is discussing adding Rust to the FreeBSD base system.
In a recent thread on src-committers, we discussed the costs and benefits of including Rust code in the FreeBSD base system. To summarize, the cost is that it would double our build times. imp suggested adding an additional step after buildworld for stuff that requires an external toolchain. That would ease the build time pain. The benefit is that some tools would become easier to write, or even become possible.
Warner Losh on the freebsd-hackers mailing list
From everything I’ve read and what you, the readers, have told me, someone who isn’t a programmer, languages like Rust really are a big improvement over older languages, and it’s probably not a good idea for a major, important project like FreeBSD to isolate its base system from such progress. Now, I’m not at all qualified to say whether Rust, specifically, is the right choice, but a language like Rust should probably be part of the base system.
A big issue is FreeBSD’s architecture support. Rust is not well-supported or even supported at all on all the various platforms FreeBSD supports, which might prove to be a road block for now. That being said, letting barely used ISAs hamper your progress too much might not be a good idea either. Rust has already become a supported language for the development of the Linux kernel.
Keep an eye on rustc_codegen_gcc for a solution that should do for all platforms that FreeBSD supports and is closest to being ready. (Not all platforms, because platforms exist that don’t even have GCC or which use some weird in-house fork of it that hasn’t been upstreamed.)
…for something that’s further from being ready, but should also do the trick eventually, check out gccrs.
Bear in mind that it’s currently only been accepted for drivers and other things which need not be mandatory for all supported platforms.
ssokolow,
I also think that using rust syntax to access unsafe C kernel structures limits the value of using rust in a C kernel. To really benefit from rust, the kernel APIs and structures really need to be designed for and validated by rust. which in linux they’re not. On the bright side, it’s still a symbolic milestone that rust modules are allowed, but my gut feeling is that it is unlikely that proper linux kernel structures will fully commit to using rust.
I imagine it varies. Asahi Lina certainly found it helpful:
Yes,
Writing new code in a modern language might make sense. It might even make the process faster even though you have to worry about interfacing with the legacy structures all the time. (At least you can isolate the interface, and worry only on the new parts).
However replacing an existing system is entirely a new kind of endeavor.
Specifically when Rust only handles certain kind of errors, but can trigger other ones. And yes, rewriting code usually resurfaces old bugs cleaned up long time ago. (“Why is this structure not optimal? Let me write it in a better way… Ooops… That old NE2000 card in our server is now crashing”). That scenario is not entirely made up, because those old cards really have to be initialized in some non-intuitive manner, for example.
So, according to Google there are over 27 million lines of code in the kernel, which is pretty much unlikely to be rewritten in any other language.
(“The industry average is 325–750 LOC per month per developer”)
sukru,
I agree with you, as a practical matter, replacing all the existing code opens up all sorts of cans of worms, things that have long been settled and the willpower isn’t there. It’s not even that everything in linux was done the best way possible (it wasn’t), but re-designing and re-testing everything against real hardware is a non-trivial & costly endeavor. I think the only practical approach is to use safer languages for new code. I find it unfortunate yet inevitable that this means we will continue using C kernel APIs for the foreseeable future.
Interesting data point, although it sounds very low and and that made me wonder where I stand. I checked a recent project that took a few days and it’s 4467 lines. Admittedly some weeks I don’t write code, and that would pull down an “average”, but for the average to be < 1kLOC per developer month seems very little to me, are there other factors?
Alfman,
The data came from
https://dev.to/polterguy/loc-is-an-important-metric-to-measure-developer-productivity-21no
Which came from a Google search.
It might be off of course. But probably not by too much.
I was for example able to write 2,000+ line, per day!, when I was in high school computer club.
Today, if I can do 1/10 of it, I feel extremely productive.
sukru,
I followed your link, but the author asserts it without a citation either.
I checked more of my projects and I have to say the statement that “The industry average is between 325 and 750 LOC per month per developer” is not even close to the ballbark for myself. Either the author’s estimate is just off, or I write a hell of a lot more code than average, but this doesn’t feel true because I don’t even write code every day. So until I see more evidence, I’m inclined to believe the claimed LOC per month is wrong. Any other devs care to chime in?
I know that saying this is exactly the opposite of what your link claims, but I don’t think LOC is a meaningful metric for productivity except in an artificial sense. And IMHO using it as a performance metric can actually be harmful and counterproductive.
If ten devs get judged by LOC, that may inadvertently be creating a fitness function with regressive properties in the long run. A dev who outputs far more more code obviously wins at LOCs, but might loose at quality, efficiency, debugging, more maintainable code, etc. I would argue reaching milestones is FAR more important than LOC. Having developers who can express problems more concisely with fewer LOC (while still solving the problem) may be the better choice,. Writing concise code is a legitimately valuable programming skill, yet LOC punishes it.
I agree the ergonomics of rust are a big step forward for OS level programming.
The ergonomics of rust with a more approachable syntax and a team which valued stability would be nice.
Flatland_Spider,
Yeah. I am behind the rust safety philosophy, but I feel the syntax is a put off for some devs. I know we can’t have everything, but ideally I would have liked to see rusts compile time verification applied to a language like Dlang, which I find syntactically pleasant to use in the lineage of C.
I don’t know I mean I understand the difficulty, but at the same time there will always be dev advocates or some such that say the new thing is too different and difficult to learn. See the transition from VB6 to C#.
In this case, the FreeBSD team would probably start by applying rust to the userland rather then the kernel.
They’re looking at it as a means to get better testing frameworks in place, as writing tests in C for some features isn’t going to be easy. At least, that seems to be the consensus coming out of the mailing list discussions.
Start with something small that would let them work on the infrastructure to support Rust (or Language X or Y or Z, as the case may be) without requiring it for a normal buildworld/installworld process. Once all the infrastructure is there and working, then they can look at using it to write small userland utilities, then larger utilities, then maybe rewrite some utils, and expand it’s use.
There’s not even the hint of a suggestion to add Rust support to the FreeBSD kernel at this time.
ssokolow,
I’m sure this depends from project to project, but the kernel drivers I’ve written had to interact with the rest of the kernel’s structures at a low level and it was the whole point of writing the driver. If the kernel were using safe primitives native this would be fine, but mixing and matching safe and unsafe code compromises the benefits of a safe language.
Don’t get me wrong, I feel rust is up to the task for developing a bona fide safe kernel from the ground up, but as with any other alt-OS it faces a critical mass problem – no one will use it. Linux falls well short of rust’s potential to verify memory and threading behavior across kernel structures, but since it already has critical mass and mind-share, we’ve got to take what we can get I suppose.
Yeah, C can be extremely tedious. Even for an unsafe language, we’d have much better choices.
That doesn’t help FreeBSD since they removed the GCC from base several years ago, and they aren’t interested in pulling it back in.
It is a surprise to me to learn that Rust does not work on all platforms that GCC supports. They use Clang / LLVM as their base compiler and Rust is also LLVM based. I would have though that FreeBSD would have been limited to LLVM supported platforms already.
Linux is another story of course as GCC is the assumed compiler there.
I need to look into FreeBSD platform support.
Rust uses a special LLVM version, and rust also has spotty OS support simply because the rust team doesn’t care that much about OSes outside of the big 3.
FreeBSD limits themselves mostly to modern server gear (x86 (T1), Arm (T1), PowerPC (T2)). They’ve never had a wide variety of ISA support. MIPS was the 4th, but it’s been cut as of v14.
Last I heard, Rust can build against stock LLVM but official builds are against a following fork containing any patches they’re waiting for LLVM upstream to merge at the time.
As for “doesn’t care that much about OSes outside of the big 3”, I think you’re being misled by Rust’s tiered support list. “Tier 1” literally means “We have access to a build farm where every push to the main development repo can be gated on a successful CI build of the toolchain and regression suite run, natively, on that platform”.
The thing constraining Tier 1 to “the big three” is that “natively, on that platform” clause. They just don’t have affordable access to cloud providers who offer VMs on the more niche architectures to extend their CI farm to them.
Since the release of LLVM 8.0, Rust can be built using stock LLVM releases. There are some caveats around which crates are used, and which Rust Editions are targeted. But (according to posts on the mailing list) it’s doable.
FreeBSD is much more judicious in the platforms it supports. NetBSD and OpenBSD target high portability, but FreeBSD mainly targets modern server hardware. x86-64 and aarch64 are the two tier 1 platforms, so LLVM isn’t inhibiting FreeBSD.
FreeBSD Platform Support matrix: https://www.freebsd.org/platforms/#_supported_platforms
Rust is probably not the correct choice, but yes, OS projects like FreeBSD should figure out a more ergonomic and compiled programming language which can fit into base.
Rust itself is not the answer, in the case of FreeBSD. Linux is baseless, so it can pull in whatever. The BSDs have a base though, and they need to make base as self contained as possible. In the case of Rust, it requires it’s own special LLVM version. If Rust was free with the LLVM in base, it might make more sense, but it’s currently more work.
FreeBSD has C, C++, Lua, and Shell as it’s programming languages. The gap is between Lua and C, C++. Something compiled with memory safety without a GC would be ideal, but I’m not sure what that is. LLVM is in base, so the language should be able to work with LLVM stable.
Since the release of LLVM 8.0, Rust can be built using standard LLVM versions. The LLVM in FreeBSD base, though, isn’t a complete “whole-hog” install of LLVM; it’s tailored to the parts they need/use in the base OS. That could make it harder to compile Rust from scratch, and/or lead to more parts of LLVM being imported into the base OS.
There’s also the whole discussion around which Rust Edition would be support for each stable branch of the OS (13.x, 14.x, 15.x, etc), as that should provide a more stable versioning for Rust that matches up better with the FreeBSD release cadence.
Then there’s the whole discussion of which crates to support and how to limit scope-creep via crates.
Several commenters on the mailing list have brought up other languages that could be used instead of Rust (Lua, which is already in the base install; Zig, which hasn’t hit 1.0 release yet; C++-17 or C++-20 with limits on which features are used; and a few others), as a strongly-typed, memory-safe systems programming language alternative to C.