By Jeremy Hsu on September 24, 2024
Popular smart TV models made by Samsung and LG can take multiple snapshots of what you are watching every second – even when they are being used as external displays for your laptop or video game console.
Smart TV manufacturers use these frequent screenshots, as well as audio recordings, in their automatic content recognition systems, which track viewing habits in order to target people with specific advertising. But researchers showed this tracking by some of the world’s most popular smart TV brands – Samsung TVs can take screenshots every 500 milliseconds and LG TVs every 10 milliseconds – can occur when people least expect it.
“When a user connects their laptop via HDMI just to browse stuff on their laptop on a bigger screen by using the TV as a ‘dumb’ display, they are unsuspecting of their activity being screenshotted,” says Yash Vekaria at the University of California, Davis. Samsung and LG did not respond to a request for comment.
Vekaria and his colleagues connected smart TVs from Samsung and LG to their own computer server. Their server, which was equipped with software for analysing network traffic, acted as a middleman to see what visual snapshots or audio data the TVs were uploading.
They found the smart TVs did not appear to upload any screenshots or audio data when streaming from Netflix or other third-party apps, mirroring YouTube content streamed on a separate phone or laptop or when sitting idle. But the smart TVs did upload snapshots when showing broadcasts from the TV antenna or content from an HDMI-connected device.
The researchers also discovered country-specific differences when users streamed the free ad-supported TV channel provided by Samsung or LG platforms. Such user activities were uploaded when the TV was operating in the US but not in the UK.
By recording user activity even when it’s coming from connected laptops, smart TVs might capture sensitive data, says Vekaria. For example, it might record if people are browsing for baby products or other personal items.
Customers can opt out of such tracking for Samsung and LG TVs. But the process requires customers to either enable or disable between six and 11 different options in the TV settings.
“This is the sort of privacy-intrusive technology that should require people to opt into sharing their data with clear language explaining exactly what they’re agreeing to, not baked into initial setup agreements that people tend to speed through,” says Thorin Klosowski at the Electronic Frontier Foundation, a digital privacy non-profit based in California.
awful ethics aside what a disgusting waste of processing power. software already barely runs
now you know why
Screenshotting every 500ms is insane.
Even a 0.30$ ch32v003 could handle this tiny amount of data. It’s not a resource limit
I was curious enough to check and with 2KB SRAM that thing doesn’t have anywhere enough memory to process a 320x200 RGB image much less 1080p or 4K.
Further you definitelly don’t want to send 2 images per-second down to a server in uncompressed format (even 1080p RGB with an encoding that loses a bit of color fidelity to just use two bytes per pixel, adds up to 4MB uncompressed per image), so its either using something with hardware compression or its using processing cycles for that.
My expectation is that it’s not the snapshoting itself that would eat CPU cycles, it’s the compression.
That said, I think you make a good point, just with the wrong example - I would’ve gone with: a thing capable of handling video decoding at 50 fps - i.e. one frame per 20ms - (even if it’s actually using hardware video decoding) can probably handle compressing and sending over the network two frames per second, though performance might suffer if they’re using a chip without hardware compression support and are using complex compression methods like JPEG instead of something simpler like LZW or similar.
Why think of it as a compression problem? Isn’t the spy device already getting compressed video form some source? That makes it a filtering problem. You would set it to grab and ship key frames (or equivalent term) if you wanted a human to be able to see the intel. But for content matching, maybe count some interval of key frames and then grab the smallest difference frame between the next two key frames. Gives a nice, premade small data chunk. A few of those in sequence starts looking like a hash function (on a dark foggy night).
Would want some way to sync up the frames that the spy device grabs and the ones grabbed when building the db to match against. Maybe resetting the key frame interval counter when some set of simple frames come through would be enough. Like anything with a uniform color across the whole image or something similar.
Just spitballing here. I like your impulse to math this.
We’re talking about fingerprinting stuff coming in via HDMI, not stuff being played by the “smart” part of the TV itself from some source.
You would probably not need to actually sample images if it’s the TV’s processor that’s playing something from a source, because there are probably smarter approaches for most sources (for example, for a TV channel you probably just need to know the setting of the tuner, location and the local time and then get the data from available Program Guide info (called EPG, if I remember it correctly).
The problem is that anything might be coming over HDMI and it’s not compressed, so if they want to figure out what that is, it’s a much bigger problem.
Your approach does sound like it would work if the Smart TV was playing some compressed video file, though.
Mind you, I too am just “thinking out loud” rather that actually knowing what they do (or what I’m talking about ;))
I assumed HDMI had some form of encoding, thanks for the correction. Looks like v 2.1 does.
I think the syncing idea between the spy device and db is still useful. The video itself has stuff to use for reducing the search space by making sure they puck the same instants to fingerprint and exfiltrate.
I don’t think they will compress the screenshot and send them but run content in a tensorflow lite model or even just hash a few of the pixels to try for an ID match
Well that makes sense but might even be more processor intensive unless they’re using an SOC that includes an NFU or similar.
I doubt it’s a straight forward hash because a hash database for video which includes all manner of small clips and has to somehow be able to match something missing over 90% of frames (if indeed the thing is sampling it at 2 fps, then it only sees 2 frames out of every 25) would be huge.
A rough calculation for a system of hashes for groups of 13 frames in a row (so that at least one would be hit if sampling at 2 fps on a 25 fps system) storing just one block of 13 frame hashes per minute in a 5 byte value (so large enough to have 5 trillion distinctive values) would in 1GB store enough hashes for 136k 2h movies in hashes alone so it would be maybe feasible if the system had 2GB+ of main memory, though even then I’m not so sure the CPU speed would be enough to search it every 500ms (though if the hashes are ordered by value in a long array and there’s a matching array of clip IDs, it might be doable since there are some pretty good algorithms for that).
I would sample a few dozens equally space pixels out of the frame, then drop similar value frames, and send that with timestamp. In the cloud, you runs those few pixels in a content recognition model.
It doesn’t have to be especially accurate or know any niche content, the point is to make a psychomarketing profile of the customer like “car guy, watches tool reviews”.
TVs I’ve come across are such displeasure to use, it’s incredible
Installed pi-hole this week. Number one blocked domain with 1600 queries… Scribe.logs.roku.com.
Do not connect your Smart TVs to network people, seriously. Just a bad idea. Use a media center PC or some other device that allows you to stream content, and make sure the TV itself is just a big monitor, nothing more.
They’re eating the dogs. 😊
This is why our “smart” TV is not allowed to be connected to the internet.
There is such a thing called HDMI Ethernet. If you connect some sort of Android box to your TV it might establish an Ethernet connection with it and thus connect to the internet.
A smart TV is not allowed on my property.
But can you really be sure that it doesn’t connect to another network? i have to check again but if i recall correctly there are TVs that try to connect to other open networks or even look for other TVs from the same manufacturer and connect through those to the internet. I have to double check this again, so take this with a grain of salt
If that’s true - lan for your own content with network isolation and ripping out the WiFi antenna, I guess?? I hate this
I am a bit puzzled about the “even when your laptop is connected” part.
I have a small android box connected to it and am not using apps on the TV so it should have no chance of sending screenshot out even if it takes them.
The TV itself is not connected
what kind of Android box do you have? anything you recommend? (looking to have some sort of streaming client)
It’s a Chinese one that I used at first for retro gaming with emuelec. Now it is dual boot and I have kodi and newpipe on it too.
Nvidia Shield. The bigger one.
Yes, it’s a couple of years old at this point, but it’s still the best device of its kind.
Not to mention the remote is FANTASTIC.
the google tv with chromecast dongle is quite decent, good price/performance ratio
But if you connect your phone to the android box then the TV could use the phone to send the screenshots.
TV->android box->phone->internet.
Sorry for being paranoid but can the TV piggyback the connection used by the the streaming device/android box to send data back to the TV OEM?
Not yet but it is clear in the future most devices will be able to do that, we will have ubiquitous low grade internet access everywhere. Your neighbours devices will let your electronics snitch on you even if you seal up your own internet
Good point. We can have a honeypot wifi. Check my other comment in the thread.
Okay, so for the new folks with networking, how do you set up a honeypot wifi? Have a (second) router powered on with no connection? Or is it something you can set up with one router?
Many routers have a functionality to create a guest wifi. These usually run on a separate VLAN so that it can’t access other devices on the network. After creating this guest wifi, you have manually disable internet access on this but keep the wifi on.
Unfortunately the process varies between router to router.
The only connection the TV has is hdmi. I do not think that back and forth communication is possible there.
If the TV has wifi, it can do its thing but that would also be easy to disable.
I think for the TVs internal wifi, it’s better to create a honeypot Wi-Fi exclusively for it, or a VLAN. It will constantly try to send data and fail. If we don’t let it connect to anything, the TV might start sniffing for other open networks.
I’m happy to see this, my wife and I were about to buy a smart TV. Now I’ll just get the dumb variant.
Now I’ll just get the dumb variant.
These don’t really exist on a consumer level anymore. What you’re looking for is called a commercial display, which is what’s used in businesses and hospitals.
No, they still exist.
Ah, I stand corrected.
Luckily they exist in smaller sizes still. I’m just getting a small tv for occasional use.
Earlier this month I finally disconnected the wifi for my 7 year old Roku TV. I miss being able to turn it on w/ voice activation but I’ll trade that in for my privacy
So how do you all guys watch content on these “dumb TVs”?
If you connect e.g. android box, how is it any different than connecting the TV itself? Do you think producers of android boxes aren’t such pricks? This bugs my mind.
OTA antennae for sports.
For streaming, I usually watch it on my laptop so that I can have easier options to skip and replay.
My desktop is connected via HDMI so I have that as an option but I rarely take advantage of it.
I live by myself so I don’t have as much pushback as you likely would FYI
best way is a mini pc you can put an open source OS on
then you totally control it. they can be found cheap used and are usually upgradable
they are thrown out by schools and buisnesses all the time. it does not have to be very powerful by pc standards
it can also be your first home server if youre interested
But that is terrible to use. I can’t imagine my kids or wife to use this with TV…
no, not true you can put whatever you want on it. ours boots into a nice tv like ui and they open stremio with a remote and thats it
its up to you to make it nice and easy
the user experience is not radically different from a corporate experience except its faster and without ads or spying
You hear that? It’s a whisper… It’s a multinational multibillion dollar class action lawsuit coming after Samsung and LG. WTF!
I had to update my LG recently and it had to get approval for all sorts of weird shit. Oddly enough, it let me continue using just about everything even after I denied all the very invasive checkboxes. I guess even they can’t deny use of your own tv if you reject the agreement lol
For future reference, you can update LG TVs via USB so you can avoid connecting it to a network.
mine would be getting only choppy static more than anything. where i live there is only sattelite available and it costs more than cable
I hope they enjoy my 25 million screenshots of Buffy, Angel, and Stargate.
i genuinely do not understand how TVs are so corrupt and greedy. You just display pixels, that’s it! The entire purpose is to convert 1s and 0s to pretty color
Jokes on them tho, they lack common understanding.
I watch a video about someone modding a shitbox and they think i can afford this new spyker sports car or any other 80k e car.
Obviously that shit is a swing and a miss. You want to give me advertising that suits me? Start by advertising stickers about cars because that’s something i could afford…not something i would buy tho.
Hmmm
“how to diy replace stolen catalytic converter”
“96 Ford esprit strange smell and noise in roof”
Youtube: buy this $90 grand all terrain thingy on credit!
Btw, is there a firmware hacking/flashing scene for smart TVs?
my tv can barely play Netflix I doubt it’s taking pictures every second lol
