There exists a peculiar amnesia in software engineering regarding XML. Mention it in most circles and you will receive knowing smiles, dismissive waves, the sort of patronizing acknowledgment reserved for technologies deemed passé. “Oh, XML,” they say, as if the very syllables carry the weight of obsolescence. “We use JSON now. Much cleaner.”
{ "columns": ["id", "name", "age"], "rows": [ [1, "bob", 44], [2, "alice", 7], ... ] }There ya go, problem solved without the unparseable ambiguity of CSV
Please stop using CSV.
Great, now read it row by row without keeping it all in memory.
Wdym? That’s a parser implementation detail. Even if the parser you’re using needs to load the whole file into memory, it’s trivial to write your own parser that reads those entries one row at a time. You could even add random access if you get creative.
That’s one of the benefits of JSON: it is dead simple to parse.