A few years ago I designed a way to detect bit-flips in Firefox crash reports and last year we deployed an actual memory tester that runs on user machines after the browser crashes. Today I was looking at the data that comes out of these tests and now I'm 100% positive that the heuristic is sound and a lot of the crashes we see are from users with bad memory or similarly flaky hardware. Here's a few numbers to give you an idea of how large the problem is. 🧵 1/5
The problem is that ECC is used to permit price discrimination between server (less price sensitive) and PC (more price sensitive) users. Like, there’s a significant price difference, more than cost-of-manufacture would warrant. There are only a few companies that make motherboard chipsets, like Intel, and they have enough price control over the industry that they can do that.
Also…I’m not sure that ECC is the right fix. I kind of wonder whether the fact is actually that the memory is broken, or that people are manually overclocking and running memory that would be stable at a lower rate at too high of a rate, which will cause that. Or whether BIOSes, which can automatically detect a viable rate by testing memory, are simply being too aggressive in choosing high memory bandwidth rates.
Also…I’m not sure that ECC is the right fix. I kind of wonder whether the fact is actually that the memory is broken, or that people are manually overclocking and running memory that would be stable at a lower rate at too high of a rate, which will cause that.
Some of it is cosmic rays, right? I think ECC is still worth it even at JEDEC speeds.
My last Intel motherboard couldn’t handle all four slots filled with 32GB of memory at rated speeds. Any two sticks yes, four no. From reading online, apparently that was a common problem. Motherboard manufacturers (who must have known that there were issues, from their own testing) did not go out of their way to make this clear.
Maybe it’s not an issue with registered/buffered memory, but with plain old unregistered DDR5, I think that manufacturers have really selling product above what they can realistically do.
The problem is that ECC is used to permit price discrimination between server (less price sensitive) and PC (more price sensitive) users. Like, there’s a significant price difference, more than cost-of-manufacture would warrant. There are only a few companies that make motherboard chipsets, like Intel, and they have enough price control over the industry that they can do that.
Also…I’m not sure that ECC is the right fix. I kind of wonder whether the fact is actually that the memory is broken, or that people are manually overclocking and running memory that would be stable at a lower rate at too high of a rate, which will cause that. Or whether BIOSes, which can automatically detect a viable rate by testing memory, are simply being too aggressive in choosing high memory bandwidth rates.
Some of it is cosmic rays, right? I think ECC is still worth it even at JEDEC speeds.
My last Intel motherboard couldn’t handle all four slots filled with 32GB of memory at rated speeds. Any two sticks yes, four no. From reading online, apparently that was a common problem. Motherboard manufacturers (who must have known that there were issues, from their own testing) did not go out of their way to make this clear.
Maybe it’s not an issue with registered/buffered memory, but with plain old unregistered DDR5, I think that manufacturers have really selling product above what they can realistically do.