SchwertImStein@lemmy.dbzer0.com to Programmer Humor@programming.devEnglish · 2 days agomuskrat's data eng expert's hard drive overheats while processing 60k rowslemmy.dbzer0.comimagemessage-square303fedilinkarrow-up1959arrow-down113
arrow-up1946arrow-down1imagemuskrat's data eng expert's hard drive overheats while processing 60k rowslemmy.dbzer0.comSchwertImStein@lemmy.dbzer0.com to Programmer Humor@programming.devEnglish · 2 days agomessage-square303fedilink
minus-squareeasily3667@lemmus.orglinkfedilinkEnglisharrow-up141·2 days ago60k is laughably, embarrassingly small. It’s still sqlite-sized.
minus-squaremorrowind@lemmy.mllinkfedilinkarrow-up49·2 days agoSqlite can easily handle millions of rows. Don’t sell it short
minus-squareThe_Decryptor@aussie.zonelinkfedilinkEnglisharrow-up8·1 day agoHow about a 6.4TB sqlite database? https://boyter.org/posts/searchcode-bigger-sqlite-than-you/
minus-squareKorhaka@sopuli.xyzlinkfedilinkEnglisharrow-up4·1 day agoI have an sqlite db that is a few GB in size, game saves using the format. Sadly almost all blob data, would love to play with it if it was a bit more readable
minus-squarenaught@sh.itjust.workslinkfedilinkarrow-up57·2 days agoi mean its even excel sized depending on how many columns. This is seriously sad and alarming
minus-squareeasily3667@lemmus.orglinkfedilinkEnglisharrow-up11·edit-21 day agoHey now that’s real close to the 65,535 16-bit limit (from 20 years ago)
minus-squarewise_pancake@lemmy.calinkfedilinkarrow-up3·1 day agoHoly shit if this ids lm issue that’s too funny
60k is laughably, embarrassingly small. It’s still sqlite-sized.
Sqlite can easily handle millions of rows. Don’t sell it short
I’m not
How about a 6.4TB sqlite database?
https://boyter.org/posts/searchcode-bigger-sqlite-than-you/
Should be enough to hold 60k rows
I have an sqlite db that is a few GB in size, game saves using the format. Sadly almost all blob data, would love to play with it if it was a bit more readable
60k is single json file
i mean its even excel sized depending on how many columns. This is seriously sad and alarming
Hey now that’s real close to the 65,535 16-bit limit (from 20 years ago)
Holy shit if this ids lm issue that’s too funny