There are a couple I have in mind. Like many techies, I am a huge fan of RSS for content distribution and XMPP for federated communication.
The really niche one I like is S-expressions as a data format and configuration in place of json, yaml, toml, etc.
I am a big fan of Plaintext formats, although I wish markdown had a few more features like tables.
AV1 video codec !
I’ll add JXL if we’re doing codecs
SqLite for office formats.
I’m not quite following this. Can you please elaborate?
I read this somewhere. Since i didn’t find it anymore and don’t remember all the advantages aside from concurrence (don’t have to unpack a zip archive) i asked chatgpt:
Using SQLite instead of ZIP for office formats can offer several advantages, particularly in terms of data management, performance, and functionality. Here are some key benefits:
- Structured Data Storage: SQLite is a relational database management system, which means it can store data in a structured format with relationships between different data entities. This allows for more complex data queries and relationships compared to a flat ZIP file structure.
Query Capabilities: With SQLite, you can use SQL queries to retrieve and manipulate data efficiently. This is particularly useful for applications that require complex data retrieval, filtering, and aggregation, which would be cumbersome with a ZIP file.
Integration with Other Tools: SQLite can be easily integrated with various programming languages and tools, making it a versatile choice for applications that require data storage and manipulation.
- Concurrency: SQLite supports multiple readers and a single writer, allowing for better concurrency when accessing and modifying data. This can be beneficial in collaborative environments where multiple users may need to access or update the data simultaneously.
- Data Integrity: SQLite provides features like transactions, which ensure that a series of operations either complete successfully or leave the database unchanged. This helps maintain data integrity, especially in scenarios where multiple operations are performed.
- Ease of Updates: Updating specific pieces of data in an SQLite database can be more efficient than modifying a ZIP file, which may require decompressing, altering, and recompressing the entire file.
- Reduced File Size: While ZIP files compress data, SQLite can store data in a more compact format, especially for structured data. This can lead to smaller file sizes for certain types of data. Built-in Data Types: SQLite supports various data types (e.g., INTEGER, TEXT, BLOB), which can be beneficial for storing different kinds of data in a more organized manner compared to the generic binary format of ZIP files.
- Versioning and History: SQLite can be used to implement versioning and history tracking for documents, allowing users to maintain a history of changes and revert to previous versions if necessary.
I put related points together. One point was moot, removed.
- Structured Data Storage: SQLite is a relational database management system, which means it can store data in a structured format with relationships between different data entities. This allows for more complex data queries and relationships compared to a flat ZIP file structure.
XMPP, RSS, …
XMPP is not a good protocol though. There’s a reason nobody uses it anymore.
I think it’s going to be interesting when the EU tries to enforce interoperability between the major messaging platforms. What are they going to do? They have some ridiculous targets like interoperable end-to-end encrypted group video calls in 5 years!
There’s a reason nobody uses it anymore.
Yeah, Google and Faceebook EEE’d it.
XMPP is not a good protocol though.
Do elaborate.
XMPP is very old and was created when nobody knew about mobile phones. It worked more like true messaging app less than messages store ( unlike matrix ).
Requirement of permanent tcp ip connection doesn’t work well for mobile + pretty much useful feature in xmpp ( like message history ) is optional. If something doesn’t work in xmpp most people would blame xmpp / jabber rather than the lack of feature support in their server
XMPP is very old
Seriously? That’s your argument? So is the wheel.
Requirement of permanent tcp ip connection doesn’t work well for mobile
I was under the impression PubSub was created for that.
Still, it’s an open extensible protocol.
XMPP is very old
Seriously? That’s your argument? So is the wheel.
They elaborated how that relates; usage scenario changed with mobile phones. XMPP is a bad match.
XMPP is a bad match.
The X is for extensible, so are a whole bunch of other protocols and people haven’t stopped using them, they get improved upon (for the most part).
The mentioned permanent tcp ip connection (which you don’t neccessarily have on mobile) too?
Seriously, if you do take one verse from the whole response, you get straw men you fighting with.
I just told you that jabber / xmpp was created in the times almost nobody knew or believed mobile phones can be a thing. Thus it got created in that way: many similarities of xmpp and e-mail, irc or icq which didn’t stand the passage of time.
Of course, you’re right xmpp evolved to get PubSub extension as an “optional feature” but because of its availability (or rather lack) - most servers didn’t support it even the client did support, xmpp didn’t win the acceptance of the end-users. It got some attention in the business world (cisco jabber) but not in the retail.
Business cannot work forever without clients willing to pay or at least use, so it died off even in the business.
End of story, try not to fighting with the straw men you created.
Of course, you’re right xmpp evolved to get PubSub extension as an “optional feature” but because of its availability (or rather lack) - most servers didn’t support it even the client did support, xmpp didn’t win the acceptance of the end-users. It got some attention in the business world (cisco jabber) but not in the retail.
That XMPP’s extensibility is in itself a strength and a weakness is indeed a valid argument, as you’ve exemplified. I was expecting you’d criticize OMEMO though…
Business cannot work forever without clients willing to pay or at least use, so it died off even in the business.
No, it didn’t die off, it’s still used. IRC is still used as well, probably more or less at the same level. But if you define usage as “used in business” well then probably just a few cases, yes.
I hadn’t heard of Cisco Jabber but i’ve heard of Google and Facebook - both companies’ messengers were, initially, based on XMPP but they EEE’d it once they got enough users and walled their gardens, dealing a major blow to the protocol.
End of story, try not to fighting with the straw men you created.
Can i fight my inner daemons at least? Please?
It worked more like true messaging app less than messages store ( unlike matrix ).
Can you please elaborate this point? I don’t understand what you mean by “true messaging app” and why that would be a bad thing?
Requirement of permanent tcp ip connection
Are you sure this is the case? Maybe back in the day, but my understanding is this isn’t true anymore
useful feature in xmpp ( like message history ) is optional
Why is user choice a bad thing? There’s a wealth of clients that implement the features you want
If something doesn’t work in xmpp most people would blame xmpp
This may not be an important point, but from my experience, people always blame the client and not the underlying protocol. If I face an issue with my browser, I’d likely blame the browser before I blame http.
I use xmpp. It happens to be a great fit for a private family messaging service. Good interoperability between modern clients. I get that “nobody uses it” is hyperbole, but the internet is a big place and there is room for services without mass market appeal to thrive.
There’s a reason nobody uses it anymore.
I and many others use it! And Google, meta, etc. Have used it but decided to lock it down.
Yes you’re right, there’s a reason people don’t use it as much, which is because these corporations embraced it, dominated it, then extinguished it.
But XMPP is honestly my favorite comm protocol and the most impressive imo.
For RSS I honestly don’t see a point, at least for me. What’s the use for having update feeds in a unified format when I still have to go to each fucking site to view the full text? I completely see the point of RSS when all I need is in the feed. But I hate going from different UI to different UI to get the full content. I want something like inoreader.com for self-hosting.
The content of the feed depends on the content creator, not on RSS.
I know that. But RSS is like 95% used for news feeds and that’s what I’m talking about. The way RSS is overwhelmingly used is making the whole thing useless (to me).
well, then just consider those giving shitty support for it as if they wouldn’t be supporting it at all
RSS works great for me though.
I have an app on my not-so-smart phone to read news when commuting. It is not a long journey so I just want to have a quick glance at the headlines and read the actual articles that I want to. There are only 6 sites that I am interested, but still will take quite some work to crawl from the proper websites. RSS in turn is unified so I don’t need to worry about their website layouts, formats, etc. It also gives me an URL to the actual content which I can use readability/reader mode library to parse and further reduce unnecessary contents.
Quite the opposite, I hope more informational sites offer/keep RSS! (Some removed RSS typically after a revamp, design change)
Mastodon offers rss for both keywords and users
What’s the use for having update feeds in a unified format when I still have to go to each fucking site to view the full text
This has nothing to do with RSS, it is the author’s choice. It’s like someone who posts links to their articles on Twitter / Facebook / Reddit, same thing. The platform doesn’t prevent you from putting the entire content there, and in fact, many do, especially with RSS.
One benefit of RSS though is that because it is an open protocol, the problem you mention already has solutions, which auto fetch the articles for you. That wouldn’t be possible without an open protocol like RSS
Moreover, I’d argue even with that, RSS is still a huge plus. To have all your content’s headlines in one UI, and potentially you can filter or sort them however you want, that’s pretty awesome.
Miniflux is likely to tick most of your boxes. It’s self hostable and can download the full article without extra clicks / having to visit the source.
Thanks, I’ll take a look. These days Inoreader also shows only the summary, making it useless for me.
Zigbee or really any Bluetooth alternative.
Bluetooth is a poorly engineered protocol. It jumps around the spectrum while transmitting, which makes it difficult and power intensive for bluetooth receivers to track.
I agree Bluetooth (at least Bluetooth Classic) is not very well designed, but not because of frequency hopping. That improves robustness and I don’t see why it would cost any more power. The hopping pattern is deterministic. Receivers know in advance which frequency to hop to.
About s-expressions, what i read about it: https://web.archive.org/web/20120206034439/https://shinkirou.org/blog/2010/06/s-expressions-the-fat-free-alternative-to-json/
Seems rather niche, for non-key-value-pair data structures (aren’t no-sql databases good for that?), considering that lightweight markup fulfills that role for readable document source.
The appeal for json and yaml is readability, and partially ease of parsing. I say s-expressions win over both in both aspects.
Can you please expand on your references to no-sql and your reference to “lightweight markup”? I don’t quite understand what you meant there.
S-expressions are basically directly writing the AST a compiler would normally generate. They can be extremely flexible. M-expressions were supposed to be programming part of Lisp, and S-expressions the data part. Lisp programmers noticed that code is just another kind of data to be manipulated and then only used S-expressions.
Logo is arguably a Lisp with M-expressions. But whatever niche Logo had is taken by Python now.
Is ipfs usage growing? Stagnant? No idea… Diatributed serving of content seems great
I never really quite understood IPFS and why it gets used where I see it today. What problem is it solving?
IPFS would replace Content Delivery Networks in present day.
It would also allow you to host software and other content from your own network again without the constraints modern Internet Service Providers pose on you to limit your self-hosting capabilities.
If applications are built for it, it could serve as live storage for your applications too.
We ran ipf-search. In one of the experiments we could show that a distributed search index on ipfs-search, accessible through JavaScript is likely feasible with the necessary research. Parts of the index would automatically be hosted by clients who used the index thus creating a fairly resilient system.
Too bad IPFS couldn’t get over the technical hurdles of limiting connection setup time. We could get a fast (ElasticSearch based) index running and hosted over common web technologies, but fetching content from IPFS directly was generally rather slow.
Would you be interested in a similar protocol that supports more things (and is IMO easier to set up)?
I’m not actively looking but please do share references! Other people may read this and they may want to know too. Perhaps I’ll jump back in the rabbit hole at some point too 😁
Okay here it goes!
Tenfingers sharing protocol & python implementation (your python needs cryptodomex, or use the frozen executables).
You share theirs, they share yours (all encrypted)! So no benevolent nodes or crypto and it’s 100% decentralised.
I’m working on a better documentation on how to set it up (just forward a port and run setup basically).
I had to read the overview and it looks nice. It reads like IPFS without some of the challenging cruft. Well written!
IPFS seemingly works small scale but not large scale. What makes tenfingers handle millions of files and petabytes of data better than IPFS? Perhaps that is not the goal. In what way do you think the tech scales? Why will discovery of the node which has the data be short?
I want to ask for benchmarks but you can’t do a full benchmark without loads of resources.
Thanks!
IPFS is static, whereas tenfingers is dynamic when it comes to the links. So you can update the shared data without the need of redistributing the link.
That said, its also very different tech wise, there is no need for benevolent nodes (or some crypto or payment).
Nodes do not need to be trustworthy either, so node discovery is very simple (basically just ask other nodes for known nodes).
The distribution part, where nodes share your data, is based on reciprocal sharing, you share theirs and they share yours. If they don’t share any more (there are checks) you just ditch the deal and ask for a new deal with another node.
With over sharing (default is you share your data with 10 other nodes, sharing their data) this should both make bad nodes a no problem, but also make for good uptime and takedown safety.
This system also makes it scalable infinitely node wise, as every node does not need to know all other nodes, just enough for their need (for example thousands out if millions of existing nodes).
To share lots if data, you need to bring enough storage and bandwith to the table because it’s reciprocal, so basically it’s up to your node how much it can share.
Big data sets are always complicated because of errors and long download times, I have done 300MB files without problems, but the download process sure can be made better (with parallel downloading for example and better error handling).
I haven’t worked on sharing way bigger datasets, even a simple terabyte is a pita to download on the regular internet :-) and the use case is more the idea of sharing lots of smaller data, like a website for example, or a chat.
What do you think, am I missing something important? Or of course if you have other questions please do ask!
Also, sorry I’m writing this on my mobile so it’s not very well written.
Edit: missed one question; getting the data is straight forward to use (a bit complicated how it’s handled because of the changing nature of things) but when you download, you have the addresses of the nodes sharing your data so you just connect to one of them and download it (or the next if the first one isn’t up etc and so on). So that should not be any kind of bottleneck.
file sharing between planets, obviously /s
Yeah it’s basically a benevolent-store-static-data, where static is you cannot change it (or you have to upload new data and make a new link to it).
Cool name though.
I wish standards were always open access. Not behind a 600 dollar paywall.
When it is paywalled I’m irritated it’s even called a standard.
DP >> HDMI
I’ll give my usual contribution to RSS feed discourse, which is that, news flash! RSS feeds support video!
It drives me crazy when podcasters are like, “thanks for listening to our audio podcasts. We also have a video feed for our YouTube subscribers.” Just let me have the video in PocketCasts please!
I feel you but i dont think podcasters point to youtube for video feeds because of a supposed limitation of RSS. They do it because of the storage and bandwidth costs of hosting video.
I’d think they’d get it back by not having to share their ad rev with Google. There’s something to be said for the economies of scale Google benefits from but with cloud services that’s not as relevant as it was.
I just wrote a YouTube scraper and exported to RSS and into my podcast client. Using YouTube any other way is masochism in comparison.
Since nobody’s brought it up: MQTT.
It got pigeonholed into IoT world, but it’s a pretty decent event pubsub system. It has lots lf security/encryption options, plus a websocket layer, so you can use it anywhere from devices, to mobile, to web.
As of late last year, RabbitMQ started suporting it as a supported server add-on, so it’s easy to use it to create scalable, event-based systems, including for multiuser games.
I spun up a MQTT/Aedes/MongoDB stack on my network recently for some ESP32 sensors.
Fantastic protocol and super easy to work with!
MQTT is great! There are clients available in Python, JS, etc
I’m currently on the ZeroMQ boat. What made you go to Rabbit Mq? I need the Pair socket for zeroMq for a project.
Installed RabbitMQ for use in Python Celery (for task queue and crontab). Was pleasantly surprised it also offered MQTT support.
Was originally planning on using a third-party, commercial combo websocket/push notification service. But between RabbitMQ/MQTT with websockets and Firebase Cloud Messaging, I’m getting all of it: queuing, MQTT pubsub, and cross-platform push, all for free. 🎉
It all runs nicely in Docker and when time to deploy and scale, trust RabbitMQ more since it has solid cluster support.
This isn’t exactly what you asked, but our URI/URL schema is basically a bunch of missed opportunities, and I wish it was better designed.
Ok so it starts off with the scheme name, which makes sense. http: or ftp: or even tel:
But then it goes into the domain name system, which suffers from the problem that the root, then top level domain, then domain, then progressively smaller subdomains, go right to left. www.example.com requires the system look up the root domain, to see who manages the .com tld, then who owns example.com, then a lookup of the www subdomain. Then, if there needs to be a port number specified, that goes after the domain name, right next to the implied root domain. Then the rest of the URL, by default, goes left to right in decreasing order of significance. It’s just a weird mismatch, and would make a ton more sense if it were all left to right, including the domain name.
Then don’t get me started about how the www subdomain itself no longer makes sense. I get that the system was designed long before HTTP and the WWW took over the internet as basically the default, but if we had known that in advance it would’ve made sense to not try to push www in front of all website domains throughout the 90"s and early 2000’s.
This is actually exactly what I asked for, thank you!!
Then don’t get me started about how the www subdomain itself no longer makes sense. I get that the system was designed long before HTTP and the WWW took over the internet as basically the default, but if we had known that in advance it would’ve made sense to not try to push www in front of all website domains throughout the 90"s and early 2000’s.
I have never understood why you can delegate a subdomain but not the root domain, I doubt it was a technical issue because they added support for it recently via
SVCBrecords (But maybe technical concerns were actually fixed in the decades since)Don’t worry, in 5 or 10 years Google will develop an alternative and the rest of FAANG will back it. It will be super technically correct but will include a cryptographic signature that only big tech companies can issue.
Problem Details for HTTP APIs - I have to work and integrate with a lot of different APIs and different kinda implementations of error handling. Everyone seems to be inventing their own flavor of returning errors.
My life would be so much easier if everyone just used some ‘global unified’ way to returning errors, all in the same way
Please guys, stop using line-breaks mid-sentence. It’s not the 90’s anymore, viewers generally can wrap.
Maybe a bad markdown viewer?
viewergenerator?No, in general the markdown format suggests using line breaks in the middle of paragraphs to make the code just as readable as the output. That’s why two line breaks is what creates a new paragraph. So it’s the viewer showing it incorrectly here.
The screenshot is of the website ietf.org , which doesn’t seem to be markdown.
That would be nice. I have implemented this in the past but never once encountered an API that used it.
Best is when the API doesn’t match a PDF and says “500: Internal Error”
Saving…
I made my first API at work last year (still making) and always saw myself looking for input on making a consistent way to return errors, with no useful input from the senior programmers or the API users. This is my second biggest problem, the first being variable and function names of course.
If I were to do anything related to HTTP, I now have something to look at.
TOML instead of YAML or JSON for configuration.
YAML is complex and has security concerns most people are not aware of.
JSON works, but the block quoting and indenting is a lot of noise for a simple category key value format.
TOML is not a very good format IMO. It’s fine for very simple config structures, but as soon as you have any level of nesting at all it becomes an unobvious mess. Worse than YAML even.
What is this even?
[[fruits]] name = "apple" [fruits.physical] color = "red" shape = "round" [[fruits.varieties]] name = "red delicious" [[fruits.varieties]] name = "granny smith" [[fruits]] name = "banana" [[fruits.varieties]] name = "plantain"That’s an example from the docs, and I have literally no idea what structure it makes. Compare to the JSON which is far more obvious:
{ "fruits": [ { "name": "apple", "physical": { "color": "red", "shape": "round" }, "varieties": [ { "name": "red delicious" }, { "name": "granny smith" } ] }, { "name": "banana", "varieties": [ { "name": "plantain" } ] } ] }The fact that they have to explain the structure by showing you the corresponding JSON says a lot.
JSON5 is much better IMO. Unfortunately it isn’t as popular and doesn’t have as much ecosystem support.
JSON5
Nice. I mostly use Qt JSON and upon reading the spec, I see at least a few things I would want to have out of this, even when using it for machine-machine communication
You’re using a purposely convoluted example from the spec. And I think it shows exactly how TOML is better than JSON for creating config files.
The TOML file is a lot easier to scan than the hopelessly messy json file. The mix of indentation and symbols used in JSON really does not do well in bigger configuration files.
YAML is complex and has security concerns most people are not aware of.
YAML is racist to Norwegians.
If you have something like
country: NO(NO = Norway), YAML will turn that intocountry: False. Why? Implicit casting. There are a bunch of truthy strings that’ll be cast automagically.What in tarnation
That’s “country-ist”. Nothing to do with the genes of people living over there.
People bitch about YAML but for me it’s still the preferred one just because the others suck more.
TOML like said is fine for simple things but as soon as you get a bit more complex it’s messy and unwieldy. And JSON is fine to operate on but for a config? It’s a mess. It’s harder to type and read for something like a config file.
Heck, I’m not even sold on the S-expressions compared to yaml yet. But then, I deal with so much with all of these formats that I simply still prefer YAML for readability and ease of use (compared to the others.)
ISO 216 paper sizes work like this: https://www.printed.com/blog/paper-size-guide/
It’s so fucking neat and intuitive! How is it not used more???
Presumably you could just buy that paper size? They’re pretty similar sizes; printers all support both sizes. I’ve never had an issue printing a US Letter sized PDF (which I assume I have done).
Kind of weird that you guys stick to US Letter when switching would be zero effort. I guess to be fair there aren’t really any practical benefits either.
I’ve literally never even seen A paper in America. Probably would have to special order it from another country
Ah fair enough.
I mean I’d love to use it. Of course America is behind the times of civilized nations.
sorry to tell you this bud…
Clearly the rest of the world are communists! It’s not us, it’s you! I’m not crying you’re crying! 😭😭😭
It’s also worth noting that switching from ANSI to ISO 216 paper would not be a substantial physical undertaking, as the short-side of even-numbered ISO 216 paper (eg A2, A4, A6, etc) is narrower than for ANSI equivalents. And for the odd-numbered sizes, I’ve seen Tabloid-size printers in America which generously accommodate A3.
For comparison, the standard “Letter” paper size (aka ANSI A) is 8.5 inches by 11 inches. (note: I’m sticking with American units because I hope Americans read this). Whereas the similar A4 paper size is 8.3 inches by 11.7 inches. Unless you have the rare, oddball printer which takes paper long-edge first, this means all domestic and small-business printers could start printing A4 today.
In fact, for businesses with an excess stock of company-labeled #10 envelopes – a common size of envelope, measuring 4.125 inches by 9.5 inches – a sheet of A4 folded into thirds will still (just barely) fit. Although this would require precision folding, that’s no problem for automated letter mailing systems. Note that the common #9 envelope (3.875 inches by 8.875 inches) used for return envelopes will not fit an A4 sheet folded in thirds. It would be advisable to switch entirely to A series paper and C series envelopes at the same time.
Confusingly, North America has an A-series of envelopes, which bear no relation to the ISO 216 paper series. Fortunately, the overlap is only for the less-common A2, A6, and A7.
TL;DR: bring reams of A4 to the USA and we can use it. And Tabloid-size printers often accept A3.
My printer will print and scan any A side paper. But I can’t even buy A paper! Fucking America
Also, A4 simply has a better ratio than letter. Letter is too wide, making A4 better to hold and it fits more lines per page.
Most preschool kids know what an A4 sheet is. Not sure how it can be used more.
Org-mode is like md but has tables and more. Emacs will even run computation as a party of interpretation. GitHub accepts it in place of markdown.
Would you say it’s worth considering in place of markdown for a non-emacs user? (I am curious to try emacs but I may not get to it anytime soon)
I can’t say that it is, no.
Haha appreciate the honesty :)
I do recommend emacs though. It is not the greatest editor, but it is an amazing experience. It is such an amazing experiment, that has an extensive set of different ways of looking at content and code - it will change how you think about coding.
org-mode is awesome for many reasons, but the similarities/overlap with markdown are an incidental benefit. I wouldn’t learn org-mode for that reason, however there are many other good ones that make it worthwhile. I’ve been using it for years for my own project management, tasks tracking, notes and many other things - it’s one of those rare tools that can do many things incredibly well.
Alright, but seriously: IPv6.
