Why am I not surprised that GitHub is dragging its heels on sha256? I assume they just aren't able to change fundamental parts of their system now. So no sha256, no IPv6 etc. They can only sprinkle bits around the edges.
> It is a binary file optimized for both space efficiency and quick access. Since then, it has been possible to create a repository that uses a reftable rather than the old file-based mechanism,
Good, are there (m)any other plans to ditch the slow files and use proper database? Or is it only reserved for various post-git competitors?
> Good, are there (m)any other plans to ditch the slow files and use proper database?
The filesystem is a proper database, just not a relational one.
Linus focused heavily on performance when he wrote git; he used the filesystem because, as the main Linux kernel maintainer, he knew that the Linux VFS and filesystems were fast enough for these use cases.
(It's the use cases that have changed; it was not expected back then to have more than a few hundred refs in a single repository.)
Ah, yeah, "you're holding it wrong", though use cases haven't changed, it's closer to the expected common case of expectations turning out wildy wrong (Why would you ever expect people to stop NAMING things at scale???)
But also the core property of the filesystem database has always been low performance for a bunch of tiny things
Does it mean that when switching trop sha1 to sha256 you need to forcepush and rewrite all history? Wouldn’t that be a massive source of potential vulnerabilities?
i guess that for now only the default will change for new repositories. support for sha1 is not going to be dropped, so most existing repositories won't switch any time soon. if you want to switch then yes, it sounds like a force push might be needed, although it could also be that simply switching is not possible, but that instead you have to create a new repo and import the history from the old repo, forcing everyone to clone the new repo intentionally.
Couldn't you write something that checks every commit's content and message is byte equal to the old tree? One scan through the history to verify it should be relatively simple if not cheap. Should be built into git.
I wouldn't agree with all of those reasons, but it's very definitely not "just for the sake of it." One of the better reasons so many people look to writing some things in Rust is that we now have pretty ample evidence than trying to write a binary file format parser in C is a cornucopia of CVEs that are just simply absent in Rust, and the excuse of "well, but a sufficiently smart programmer doesn't write bugs in C" doesn't cut it anymore.
Somehow we have binary file format parsers written in C everywhere, so the real world shows it is possible and we do have programmers capable of doing it.
Somehow we also have memory safety bugs everywhere, too. So real world shows bugs in C code are possible. What even is your argument? Real men write asm?
Sure, we can write a binary file format parser in C. We just can't figure out how to write one that isn't buggy and lets someone infect your computer if you give it sufficiently inventive garbage.
The issue isn't whether it's possible to have parsers, but whether it's possible to have them be secure, and periodic CVEs "everywhere" suggest we don't
Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations (so not GCC or Clang upstream), which CVEs specifically would have been ameliorated by a parser written in Rust instead of C?
> Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations
I don't see how that's possible without turning the language into something that isn't C, either by adding significant new functionality (e.g. fat pointers) or subtracting enough functionality that it's a much less capable language (e.g. disallowing dynamic memory allocation).
Of the codebases i know that have adopted Rust, it has always been because some of their maintainers wanted to do so.
Maybe git's case is different though. Do you have more info about it? Are you a git maintainer who was coerced to use Rust, or do you know of such cases?
If that's the case, they should stop using git and Linux right now, because it's everything written in C. Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
> Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
Just because something does provide an immediate perfect solution does not mean it isn't not worth investigating and/or pursuing.
Also consider that bugs tend to be more prevalent in new code (e.g., [0]) as a result, you are likely to see more of a benefit from writing new code in a memory-safe language than raw line count proportions would indicate.
You don't believe in slowly and iteratively improving a codebase over time? Should git stick with its weird mishmash of C and perl and shell scripts forever, for tradition's sake, performance and maintainability be damned?
Good, are there (m)any other plans to ditch the slow files and use proper database? Or is it only reserved for various post-git competitors?
The filesystem is a proper database, just not a relational one.
Linus focused heavily on performance when he wrote git; he used the filesystem because, as the main Linux kernel maintainer, he knew that the Linux VFS and filesystems were fast enough for these use cases.
(It's the use cases that have changed; it was not expected back then to have more than a few hundred refs in a single repository.)
Ah, yeah, "you're holding it wrong", though use cases haven't changed, it's closer to the expected common case of expectations turning out wildy wrong (Why would you ever expect people to stop NAMING things at scale???)
But also the core property of the filesystem database has always been low performance for a bunch of tiny things
Failing that, have a kind of git object that wraps another and says hey this is in sha1 don't mess with it
I wouldn't agree with all of those reasons, but it's very definitely not "just for the sake of it." One of the better reasons so many people look to writing some things in Rust is that we now have pretty ample evidence than trying to write a binary file format parser in C is a cornucopia of CVEs that are just simply absent in Rust, and the excuse of "well, but a sufficiently smart programmer doesn't write bugs in C" doesn't cut it anymore.
I don't see how that's possible without turning the language into something that isn't C, either by adding significant new functionality (e.g. fat pointers) or subtracting enough functionality that it's a much less capable language (e.g. disallowing dynamic memory allocation).
An important improvement over rust is that "Fil-C has no unsafe statement."
Maybe git's case is different though. Do you have more info about it? Are you a git maintainer who was coerced to use Rust, or do you know of such cases?
Just because something does provide an immediate perfect solution does not mean it isn't not worth investigating and/or pursuing.
Also consider that bugs tend to be more prevalent in new code (e.g., [0]) as a result, you are likely to see more of a benefit from writing new code in a memory-safe language than raw line count proportions would indicate.
[0]: https://security.googleblog.com/2024/09/eliminating-memory-s...