situation
I use git worktrees (admittedly a bit more than necessary). My main project is a cargo workspace. In Cargo.lock, I see around 550 (direct and indirect) crate dependencies. Looking at the target/debug/build directory, that results in 400 rlib files. The workspace itself has most of its code in lib crates (mostly aranged to minimize dependencies for dependees which depend on me) and a few bins. In target/debug/incremental there are around 850 .o files from the workspace crates. A full build (cargo clean; unset RUSTC_WRAPPER; cargo build --offline) takes 41 seconds.
(You see right, I haven't looked too much into the organisation of the build directory. But if you know what's going on in there, I hope these convey some impression of what size of build I'm talking about.)
The different worktrees are often different feature branches from main, though sometimes lack rebasing to the most recent main or are development of patches on top of old tags. (means: sometimes a dependency can be at different major versions in different worktrees. Most of the time, most .rs files are identical across the worktrees)
Means, one way or another, I do many compilations which should get cached.
test 1: sccache
As a test if sscache serves me, I ran the following. (I set basedir to basedirs = [ "/path/to/worktree1", "/path/to/worktree2"])
- clean sccache (
rm -rf ~/.cache/sccache) cd worktree1; git switch --detach main; cargo cleancd worktree2; git switch --detach main; cargo cleanexport RUSTC_WRAPPER=sccacheunset CARGO_BUILD_BUILD_DIRcd worktree1; sccache -z; cargo build --offline; sccache -s > fresh_build(~40s build)cd worktree1; sccache -z; cargo build --offline; sccache -s > rebuild(~20s build)cd worktree2; sccache -z; cargo build --offline; sccache -s > other_workspace(~30s build)
I.e. both worktrees should compile the same and I would hope near no cache misses in the second report.
In fact I see all three builds report
Compile requests 915 Compile requests executed 808 … Compilation failures 4 Non-cacheable compilations 0 Non-cacheable calls 103 Non-compilation calls 4 Unsupported compiler calls 0 … Non-cacheable reasons: crate-type (bin) 49 crate-type (proc-macro) 38 missing input 6 - 5 incremental 3 -o 2 The differences:
fresh_buildreportsCache hits 0(expected)rebuildreportsCache hits 804,Cache hit rate 100 %(nice, perfect)other_workspacereportsCache hits 723,Cache misses (Rust) 81
Naively, I would've expected near no cache misses (especially since anything non-cacheable is already counted as non-cacheable, and only cacheable cache misses count towards the 81 compilations).
test 2: share build directory
From the rust 1.100 release news, it occurred to me, that I might share the build dir between the git worktrees
As a test if sharing CARGO_BUILD_BUILD_DIR serves me, I ran the following.
cargo new asdf1 --name asdfcargo new asdf2 --name asdf; sed -i 's/world/Paul/' asdf2/src/main.rsunset RUSTC_WRAPPERexport CARGO_BUILD_BUILD_DIR=$(pwd)/../shared_build_dircd asdf1; cargo run; cd -(printHello, world!)cd asdf2; cargo run; cd -(printHello, world!)cargo clean; cargo run(printsHello, Paul!)
So … that doesn't work the way I intended.
what I had expected to end up with
What I had expected is that the two worktrees would just compile fine but back and forth. Anytime I would switch worktree, I would expect that most of the compilations (dependencies, stable files) are already done, but the different files would need to get re-compiled on every switch (kind of defeating incremental builds). And based on that, I would've tried combining the two approaches and getting from the cache what I overwrote by sharing the build directory.
The question(s)
- I think test 2 clearly shows I'm doing the build dir sharing wrong. How is that intended? I saw this blog post but that sounds more complicated than what I would've expected from the release notes
- Is there any beginners' mistake I'm making with ths
sccachesetup? Or is that just as good as it gets? - Do people maybe have other setups / advice how to have fast builds whilst being able to jump between worktrees and branches?
1 post - 1 participant
