Storing large files. LFS is a shitty hack that barely works.
Well, git is for source control, not binary artefacts. There are indeed projects whose size is not a good match to git, but not everyone is Google or CERN.
Integrating other repos. Git submodules are a buggy hack, and Git subtree is… better… but still a hack that adds its own flaws.
What are your requirements? What do you need this for? And why do you think everyone else needs the same?
It’s quite possible you are doing it wrong. What you want as a FOSS project are probably libraries which are build, versioned, and packaged separately. Perhaps using Debian packaging tools or Guix. Splitting it into real libraries with a concise API ensures that the API surface does not becomes too large, that the components stay relatively compact and maintainable, and that other parts of the FOSS community can re-use that library.
Companies - especially large companies - sometimes promote vendoring instead. But this promotes their interests, not those of the FOSS community on which creations they are building on.
Yes, git is designed to match the needs of the Open Source community! If you have a deeply intertwined multi-billion code base for a commercial product, a smartphone with closed firmware, or yet another TV , it might not be the best match. But who cares? Is the open source community obliged to meet such needs?
Well, git is for source control, not binary artefacts
Only because it is bad at binary artefacts. There’s no fundamental reason you shouldn’t be able to put them in version control.
It’s not much of an argument to say “VCSes shouldn’t be able to store binaries because they aren’t good at it”.
What are your requirements? What do you need this for?
Typically there’s a third or first party project that I want to use in my project. Sometimes I want to be able to modify it too (soft fork).
And why do you think everyone else needs the same?
Because I’ve worked in at least 3 companies who want to do this. Nobody had a good solution. I’ve talked to colleagues that also worked in other companies that wanted this. Often they come up with their own hacky solutions (git subtree, git subrepo, Google’s repo, etc. etc. - there are at least half a dozen of these tools).
It’s quite possible you are doing it wrong.
No offence, but your instinctive defence of Git and your instant leap to “you’re holding it wrong” are a pretty dead giveaway that you haven’t stopped to think about how it could be better.
Just to jump in here, git submodules and similar are a terrible design pattern that needs killed, not expanded. Create a library properly and stop cutting corners that will bite you in the ass.
Three seperate companies wanting to do it the lazy, wrong way doesn’t suddenly make it a good idea.
Libraries are now always a suitable solution. You just haven’t worked on the same projects I have and you can’t imagine all the things submodules are used for.
On top of that, I can’t force all third party projects to turn their repos into nice easily installable packages. Especially if they’re using a language that doesn’t have a package manager.
Because I’ve worked in at least 3 companies who want to do this. Nobody had a good solution
There are good solutions: Use proper package managers with automated build support like dpkg, pacman, pip or perhaps uv, or even better Guix. Companies not doing that are just cutting corners here.
That can work in some cases, but it’s usually not that great for first party projects where you want to be able to see and edit the code, and most package managers are OS or language specific so they don’t work well with multi-language project or projects using a language that doesn’t have a good package manager (SystemVerilog for example).
Git was Made for the Linux kernel. The kernel is pretty much only text files. For the complete decentralisation git achieves an easy diffing and merging operations needs to be defined. It is working for what it was made.
Large files don’t work with git, as it always stores the whole history on your drive.
For files that are large and not mergeable SVN works better and that is fine. You need constant online connectivity as a trade of though.
Some build tools for software being the option to define a dependency as a git path +commit or a local path. That works quite well but is in the end just a workaround.
Large files don’t work with git, as it always stores the whole history on your drive.
Not any more. That’s only the default. Git supports sparse checkouts and blobless checkouts both of which only get a subset of a repo. And actually it has supported --depth for as long as I remember.
For files that are large and not mergeable SVN works better and that is fine.
This. I have worked for a large research organization where a single SVN checkout took more than 24 hours. And they knew what they were doing.
BTW jujutsu, being created by an engineer who happens to work at Google, supports alternative backends which are meant for very large repos. But as said, I think that these do not align with the needs of the FOSS community.
Well, git is for source control, not binary artefacts. There are indeed projects whose size is not a good match to git, but not everyone is Google or CERN.
What are your requirements? What do you need this for? And why do you think everyone else needs the same?
It’s quite possible you are doing it wrong. What you want as a FOSS project are probably libraries which are build, versioned, and packaged separately. Perhaps using Debian packaging tools or Guix. Splitting it into real libraries with a concise API ensures that the API surface does not becomes too large, that the components stay relatively compact and maintainable, and that other parts of the FOSS community can re-use that library.
Companies - especially large companies - sometimes promote vendoring instead. But this promotes their interests, not those of the FOSS community on which creations they are building on.
Yes, git is designed to match the needs of the Open Source community! If you have a deeply intertwined multi-billion code base for a commercial product, a smartphone with closed firmware, or yet another TV , it might not be the best match. But who cares? Is the open source community obliged to meet such needs?
Only because it is bad at binary artefacts. There’s no fundamental reason you shouldn’t be able to put them in version control.
It’s not much of an argument to say “VCSes shouldn’t be able to store binaries because they aren’t good at it”.
Typically there’s a third or first party project that I want to use in my project. Sometimes I want to be able to modify it too (soft fork).
Because I’ve worked in at least 3 companies who want to do this. Nobody had a good solution. I’ve talked to colleagues that also worked in other companies that wanted this. Often they come up with their own hacky solutions (git subtree, git subrepo, Google’s
repo, etc. etc. - there are at least half a dozen of these tools).No offence, but your instinctive defence of Git and your instant leap to “you’re holding it wrong” are a pretty dead giveaway that you haven’t stopped to think about how it could be better.
Just to jump in here, git submodules and similar are a terrible design pattern that needs killed, not expanded. Create a library properly and stop cutting corners that will bite you in the ass.
Three seperate companies wanting to do it the lazy, wrong way doesn’t suddenly make it a good idea.
Libraries are now always a suitable solution. You just haven’t worked on the same projects I have and you can’t imagine all the things submodules are used for.
On top of that, I can’t force all third party projects to turn their repos into nice easily installable packages. Especially if they’re using a language that doesn’t have a package manager.
deleted by creator
There are good solutions: Use proper package managers with automated build support like dpkg, pacman, pip or perhaps uv, or even better Guix. Companies not doing that are just cutting corners here.
That can work in some cases, but it’s usually not that great for first party projects where you want to be able to see and edit the code, and most package managers are OS or language specific so they don’t work well with multi-language project or projects using a language that doesn’t have a good package manager (SystemVerilog for example).
There is a fundamental reason: You can’t merge them.
So what? You can manually merge them. File locking is also a common solution (LFS supports that).
The level of “you’re holding it wrong” here is insane.
Git was Made for the Linux kernel. The kernel is pretty much only text files. For the complete decentralisation git achieves an easy diffing and merging operations needs to be defined. It is working for what it was made.
Large files don’t work with git, as it always stores the whole history on your drive.
For files that are large and not mergeable SVN works better and that is fine. You need constant online connectivity as a trade of though.
Some build tools for software being the option to define a dependency as a git path +commit or a local path. That works quite well but is in the end just a workaround.
Yes I’m aware where Git came from.
Not any more. That’s only the default. Git supports sparse checkouts and blobless checkouts both of which only get a subset of a repo. And actually it has supported
--depthfor as long as I remember.This. I have worked for a large research organization where a single SVN checkout took more than 24 hours. And they knew what they were doing.
BTW jujutsu, being created by an engineer who happens to work at Google, supports alternative backends which are meant for very large repos. But as said, I think that these do not align with the needs of the FOSS community.