This is a *fascinating* exploration into the problems with VCS - LLVM moved lots of things to the monorepo (so they all move in lockstep), but now the sheer size of the monorepo and the fact that people changing APIs have to fix a lot more projects is becoming a bottleneck to progress.

@sheredom needs m0ar bors (well, and the investment necessary to run CI fast enough that bors is usable)

@sheredom Many years ago, before the monorepo, I was maintaining the LLVM interface of an out-of-tree frontend. It was a lot of work whether I ported to one LLVM release at a time, or whether I tried to track trunk weekly. APIs and debug info representation changed all the time. It often felt more like churn than progress.

I’ve never worked with the LLVM monorepo. It doesn’t sound like fun to maintain so much code you don’t really know well.

@sheredom I think the cost of LLVM’s quickly changing APIs has always been there, but it used to be externalized.

Other open source projects providing a library have spent more energy on “interface engineering”.

@stoklund right - and honestly (I just commented on the thread actually) it was by design to keep the momentum up for core contributors!

@sheredom Yeah, that’s always been Lattner’s priority. The review-after-commit policy for trusted developers had great velocity benefits too. I don’t know if they still do that.

@stoklund @sheredom It almost inhibited uptake of LLVM by other open source projects. Coupled with the clang-is-the-only-frontend-that-matters attitude.

@sheredom 100% predictable and part of why I was against the monorepo.

@sheredom Personally I've found the monorepo has made it really challenging to deal with downstream merge conflicts etc. When there were multiple repos, a merge conflict in clang wouldn't stop us from merging + testing upstream LLVM code. Everything is a lot slower now. :(

@sheredom ... OTOH bisecting Clang is a lot easier which is nice I suppose. Sparse checkouts help with the size a bit.

@sheredom Its a big mess for LLVM I guess, because it refuses to decide whether it's a monolithic C toolchain, or a compiler library. If it's a toolchain then monorepo is great. If it's a library you want to add friction to people changing the API constantly so others can actually use it. Dreadfully tricky balance.

@sheredom feels extremely similar to "are we making a game, or are we making an engine", which usually makes me think sharing engines with upstream development is a fool's errand, and the copy paste model just works out better.

@dotstdy yeah LLVM is in such a weird spot in that it was made as a compiler library but then Google in particular started to push for it to become a toolchain.

Monorepo pushes the problems of a few on the many (you change an API you need to fix all the various users), but I think without some semver API approach it was inevitable.

Sign in to participate in the conversation
Gamedev Mastodon

Mastodon server focused on game development and related topics.