My only reasoning for why such system doesn't exist yet is that they don't want to make easier for people to search stuff, probably to avoid making it easier for folks to dox others?
I think they implement some anti-scrappers measure. There was a small forum I wanted to download (it was like 30.000 pages or so), and the requisition you could do by second were waay limited.
I can't wait until I have an AI that I can order to make its own smaller internet archive. There will be 30TB harddrives latter this year curtesy of seagate. Just one of those things can hold a lot of sites
>30TB
BBC Science Focus Magazine estimates that the big four providers store at least 1,200 petabytes between them, which is 1.2 million terabytes.
https://www.sciencefocus.com/future-technology/how-much-data-is-on-the-internet
Just need 40,000x30TB drives and you could archive the entire internet
Even some of the IQfy archives refuse to let you search for specific posts due to the processing power involved, you really think that the Waybackmachine can handle it?
>866 billion pages archived
Because it'd take shitloads of processing power? Just a guess.
Nah, it's not that. There is something more nefarious other than just economics.
Probably for the same reason they don't allow certain websites to be archived.
https://www.secret-bases.co.uk/
>secret-bases
Well, they don't seem all that secret now, do they?
that site excluded itself with the following directive
<meta name="robots" content="noarchive, noodp">
https://web.archive.org/web/0/twitter.com/moot
he asked them to remove it
https://web.archive.org/web/*/twitter.com/IQfy
My only reasoning for why such system doesn't exist yet is that they don't want to make easier for people to search stuff, probably to avoid making it easier for folks to dox others?
the IA is just barely operational as it is. if they had to run a publicly-searchable index on damn near a quadrillion words, the building would melt.
Their removal policy sucks ass. They remove pretty much whatever people ask.
Imagine some buttholes going over ancient history and erasing it.
It's like preservation by obscuration.
>They remove pretty much whatever people ask.
Then what is the fricking point of an archive?
Ask a leftist about history
They don't remove it, just hide it. In 100 years they'll make it open to the public again.
>In 100 years they'll make it open to the public again.
Why are they pretending that they will be around in 100 years?
It would be one thing if it was just DMCa, but anything else has no reason to be removable.
All history, ancient or not, should be erased though.
has the Wayback Machine been slow for a few months for anyone else? and i mean REALLY slow.
I think they implement some anti-scrappers measure. There was a small forum I wanted to download (it was like 30.000 pages or so), and the requisition you could do by second were waay limited.
Also, when archiving saves, it's terribly slow.
site:archive.org used to work in the good old days, sundar needs a frickin boeing whistle blower kiss to the face
I can't wait until I have an AI that I can order to make its own smaller internet archive. There will be 30TB harddrives latter this year curtesy of seagate. Just one of those things can hold a lot of sites
>30TB
BBC Science Focus Magazine estimates that the big four providers store at least 1,200 petabytes between them, which is 1.2 million terabytes.
https://www.sciencefocus.com/future-technology/how-much-data-is-on-the-internet
Just need 40,000x30TB drives and you could archive the entire internet
I did say smaller
Even some of the IQfy archives refuse to let you search for specific posts due to the processing power involved, you really think that the Waybackmachine can handle it?
They can. They just don't want to for the reasons why
mentioned.
You'd think a widely used archival website in 2024 wouldn't be as slow as YouTube on a 56k modem. I hate how slow it is.