“It is a sign that people aren’t happy within the U.S. government, clearly. The shooting [of Good] was the last straw for many people," Dominick Skinner, ICE list founder, told The Beast.
Information included in the new leak includes around “1,800 on-the-ground agents and 150 supervisors. Early analysis by the organization suggests that around 80 per cent of the staff identified remain employed by DHS,” according to The Beast.
it’s like half up. here is some. the special:export is kinda working, but i’m very new at ripping wiki sites
https://archive.is/ipv5h
Any chance you know Python, or maybe you can have ChatGPT write a script that uses bs4 to parse through each of the name hyperlinks in the html? A simple loop should be able to visit each page, cache the results on disk, and finish up eventually.
The link doesn’t work where I am, still.
it’s very slow for me. i’m looking into python and all the other stuff. if you one specifically that will scrap the whole site, i’m down to run it. any OS except mac i can use. i can’t copy/paste each agent page, looking to scrap the whole site at once
edit!
i think i have it scraping it all to html files. it’s taking time, just trying to make sure it gets the agents and incidents at least.
I see the edit, sweet!
I wouldn’t be able to write the script myself because I can’t load the site at all. So I can’t analyze the html and determine how to loop through it an extract all the additional links for scraping. Unless you want to send me the html. Then I can work with that, but without testing on the live site.
If you have something working though, great! Let’s just start there.
Check by opening one of the html files with your browser, or VSCode or something, and see if it has the data you were expecting.
the files look correct, just html version of the pages, just not sure if it’s going to crawl and get every page. i don’t even are if it’s just a folder of html file sat this time, as long as all the agents are retrived. will keep testing.
I just set up a little bot in Jupyter Notebook to get them as well, and it’s working! Very slow, but working. I just fetched all the links to the agent detail pages. There are 1532 total. Once I fetch all the detail pages, I’ll move on to the incidents too.
Definitely keep pulling yourself, though. Who knows if I will be able to get them all?
Anything else worth grabbing while I can?
not sure, i’m looking through agents now. it’s HTML, and loads with broken CSS/styles, but the text is there
these people just straight up advertise they’re pieces of shit. wow. no shame.