Personal youtube video archive stats: 19% gone
Local Link | Content link | Reddit Link
r/DataHoarderu/Frederik2002 (622)

Total videos downloaded: 131539 (approx since January 2021)

Still online: 106633

Unavailable (Deleted/Unlisted): 24906

Of those one huge channel with 6000 videos over the past 12 years was recently deleted. Still, a seemingly safe gaming channel has had about a dozen videos either privated/unlisted or deleted (some due to newly added age verification hits)

I do not wish to disclose the exact channels/videos, these videos will soon end up on archive_dot_org. Though it's a warning to you: add all your subbed and favorite channels to a daily youtube-dl (yt-dlp) download.

If you subbed to too many channels to create a list manually: create a Google Takeout for the youtube profile you're using, set the smallest archive size. The first archive will contain a CSV with subscriptions.

EDIT, Sep 27th: To be completely honest, I should add that I specifically archived some political channels that were at risk. In that sense, my high percentage of vanished videos is a good KPI of my archival choices.

Comments (154)

[deleted]

always save what you love, especially music i would have downloaded and put it on my phone/ipod

Here's a channel that might fill a similar niche.

https://www.youtube.com/channel/UCa3SDs8cPme7tD9S8BIVYrw/playlists

Has Skyrim music, Oblivion music and some playlists for WoW, Overwatch and The Witcher

Can confirm, there is this one mix on YouTube which I like to hear in the morning before taking to the skies, at some point it was gone. It was on and off for a few months, but I was lucky and managed to get a copy at some point.

I've been getting into synthwave remixes of modern pop music, thank god I've been downloading the ones I like because some have already disappeared.

Ya, it's interesting the things on the internet that disappear.

Going through my steam list there are quite a few games that no longer have storefronts. Even popular games are sometimes missing. I noticed the other day that Deus Ex: Human Revolution only had the directors cut available, and while I know they are mostly the same thing they aren't quite and now new people can only play by... other methods.

it's interesting the things on the internet that disappear.

Reminds me of how back in school adults and teachers would badger us about the things we put online being there FOREVER! Shows them right, hah. At this point it's easy to be paranoid that anything could vanish at any given moment.

Just like the whole "you won't have a calculator", the actual meaning of that could be something completely different.

In that case if you put something on the internet there's nothing stopping someone from taking it and re-posting it.

Just wish they'd say what they mean, because the truth is so much more meaningful.

Pretty sure for steam games they should remain available to download from your library even if the store page is gone.

I have no idea how long this lasts for, but I've downloaded unlisted games before.

Even assuming Steam is a best scenario awesome company that is willing to keep games on their servers that are no longer for sale, it would still end if the company every went under. So downloading all your games locally is still wise.

Of course realistically you'd have some amount of warning before the entire service just collapsed, but if you've got a large enough library that warning may not be enough.

Games that you have purchased on Steam should be available to download for as long as steam is extant, they are legally bound to so it doesn't matter if they are good or not, any attempt to remove a players access to their games they have purchased would lead to a massive, embarrassing, and expensive lawsuit.

Downloading all your games locally won't do jack shit because the game still needs to check with Steam to make sure you're authorized to play. Look at Bulletstorm, you could have a physical copy of that disk and it still won't work until you crack it because Microsoft shut down Games for Windows

Oh yes you can download them if you've already bought them.

The only real problem comes from if you haven't gotten a copy yet. I also wonder what happens if you want the DLC. Human revolution is available as another package so in that case it's "fine" if the answer is you're out of luck, but there are lot of other games that rent IP so can only be in a store for so long before disappearing(My wishlist too, was full of those gaps last time I checked).

I’ve downloaded all my steam games for this reason into a 4tb HD. Part of the trick for offline mode is to open each game all the way to game play to help the odds of it working once it’s removed or Internet is down etc

Won't work. Steam won't let it start if it's removed.

Wolfenstein was recently merged with its sequel on steam and one removed. But I can still play both.

You can’t download it again but you should still be able to play it if you’ve played it before

You will still be able to download it, you just need to hold a copy off it on Steam. This is assuming no bugs happen since removed games usually can’t be updated regardless if it’s for bug fixing, new content, or anything else. The only thing that may make it not playable is if it still goes through another launcher, but most games don’t do that.

Source: banned game dev from Steam

Some do. Depends if the dev ties their game to the steam authentication DRM or not.

The other problem is that, DRM or not a lot of games have prerequisites that won't be met on a new system, and without steam to fix that you'll be left figuring that our yourself(and heaven help you if it needs its registry entries or has something in another folder). It's why GOG was/is such a nice system, DRM free and installers included.

Youtube really gives you a sense of horror at how impermanent digital data is.

If a lunatic ran through a paper library with a flamethrower, at least there would be a chance to stop the fire and save the books. But a Youtube channel owner can click a button and disappear thousands of hours of history. It is permanent and instant.

But a Youtube channel owner can click a button and disappear thousands of hours of history. It is permanent and instant.

Doesn't have to be the owner, could be YouTube itself. And it's been proven people are much more willing to delete a file than throw away papers.

So it's more like the librarian is the one with the flamethrower.

Yeah, when I get enough money I want to make a setup to store all games and movies in existence locally.

How are you doing with downloading this data? I can't get more than a few minutes into archiving just subtitle data without being limited. Even with adding a 30 second delay, disabling ipv4 and all that

[deleted]

I would expect to run into that limit sooner as I am only grabbing the subtitles so downloads will be incredibly fast but I only get about 60-100 before I am stopped. Incredibly frustrating and entirely halts a project I am working on

Add a delay between each download? I don't know how fast each download is but maybe a few seconds could help.

Tried that, even tried up to 30 seconds but apparently that did nothing

use a VPN service (or proxy) where you can 'change' your IP

yup, tried that too

It would be worth exploring if they change request limits if you operate out of a gcloud instance

More than likely you are behind a huge ISP level NAT that's making a few hundred users appear as one IP online. They're all hitting YouTube enough for the rate limit to kick in.

Residential IPv4 (at home) + Cookies + sleep. Allow me to explain.

IPv6 uses no NAT so Google has stricter limits for IPv6. You could drive on both so the rate-limiter is spread over two IPs. Cookies is an absolute must. You must update them every once in a while, but this is the decisive factor for the rate-limiter. Sleep is important for video info pages.

--cookies "cookies.txt" --user-agent "$USER_AGENT" --sleep-subtitles 5

Looks like I don't even have a general sleep anymore. --sleep-requests is overkill if you fetch comments, looks like no rate-limit on them. Although I suggested IPv4, I drive half my downloads over IPv4, half IPv6 with same cookies. I guess I could run everything over one IP.

User-agent doesn't matter but I prefer it that way. If Google wanted, youtube-dl leaves a lot of traces to be detected anyway.

EDIT: I also only download few subtitle languages. I had to, else rate-limit.

EDIT2: You may want to try to use cookies for two different YT profiles (channels) with one account.

spread over two IPs

Depending on how big of a prefix you have and how Google applies the limits, you can potentially use way more than one IPv6 address. Though I'd assume they probably already thought of that and have a solution for detecting it.

If you use cookies then they know. If you don't use cookies the limits are A LOT lower

Never thought about using cookies with youtube-dl or yt-dlp, clever!

Did you tried yt-dlp? Works for me

Yup, tried that, as well as tar tube. Can't find anything that seems to download more than a handful of them (relatively, I need 6000 of them or so)

[deleted]

On jdownloader is it best practice to only have a few downloads and a slow rate? I've always wondered if cranking up the speed to max and having say 20 downloads at time would make the server go "huh? Let's throttle this IP of this balls to the wall downloading guy." I feel youtube used to slow the speed if it was being hammered but would like this confirmed.

Jdownloader. Literally copy the channel page or the video page and it should trawl it all for you. Then select what you want to download - video, audio - usually video in various qualities.

I would love to but it only gets the first 100 videos as far as I can find and all of the solutions require manually scrolling through the playlist to load it and manually copy the links. Not really doable with over 8000 videos to do

https://www.reddit.com/r/jdownloader/comments/jqtops/comment/i386rcy/?utm_source=share&utm_medium=web2x&context=3

https://www.reddit.com/r/jdownloader/comments/k11r3z/comment/gt33cc2/?utm_source=share&utm_medium=web2x&context=3

These responses from other threads might have the responses you need for it to work

I also download comments, which is a natural buffer to avoid the "too many requests" error.

Just a thought.

Youtube simply has too many videos to upload them all anywhere, including archive.org. And even if you upload them somewhere, they can potentially be deleted at any time. As a solution, I thought you could make a sub where everyone posts which channel or videos they have backed up. When someone searches for the channel or video, for example with Google, he finds the posts. Here it is important not to upload the videos, but just mention them, so they won't be deleted by Reddit. Then he can simply write to the owner of the files privately and ask for the YouTube videos. Or alternatively make all videos searchable with Soulseek.

Distributed YouTube Archive https://discord.gg/RaEfhKxV

I miss actual forums for this (and everything, really). Discord isn't really all that good for things like this.

"Oh golly, yet another thing for my way too long list of servers I joined for a single, minor purpose"

"Let me put this on mute so I don't get a zillion notifications, just what I want to check."

never checks it again

Very accurate. This is the kinda stuff where a regular, publicly accessible website is wonderful.

I can bookmark it. It takes barely any ressources when not in use (those few Kilobytes of storage oh noo) and when you need it, it's most likely easier found again.

If only someone could make some kind of... bulletin board-esque website where people could post up topics and comments.

Or maybe some kind of distributed bulletin board where all users get a copy of the content when they fetch it, so that archival is trivial and deletion practically impossible.

One could maybe make a distributed network of yt-dlp-aware home servers with some way to ask if someone could share their goldmine somehow

I also imagined that someone has this hoard, it is findable through a Google search or an application that makes all files searchable, like Soulseek and then you have to ask if you can have these videos, because some own a lot of Youtube videos but are either afraid to put them online because of copyright infringement or simply have no capacity (money, interest, lifetime) to upload them to hosters.

So the videos must be (easily) findable, must not be on servers (could be taken down at any time) and must not be available immediately, only with asking, because otherwise it could come to (automatic) copyright violations.

It gets even worse when a discord link is filled with references to other discords, creating a fractal pain in the ass.

Turns "Oh an interesting effort/project I may want to contribute to" into "fuck this shit, I'm out".

Better than granting some random corporation with no reason to trust a view into everything you do.

edit: Discord servers aren't servers, it's just an annoying misnomer. The source code and binaries haven't been leaked yet, so no one can have a personal Discord server. All are minimal abstractions within view of that untrustworthy corporation. Join actual independent servers and forums, not Discord's joke.

[deleted]

Quite so, yes. If it wasn't clear, I'm saying Discord is unworthy of trust.

You are de facto describing Discord. TMK, nothing is encrypted on their side, so they have full visibility and full access to everything everyone posts.

And, maybe it is just me, but I cannot for the life of me use it in any productive manner: search sucks, there's no way to properly split/stick to a topic, no proper message threading, and the usability drops to nothing if one joins more that 10-15 servers.

Why do we continually go in the wrong direction when it comes to user experience and usability of software?

You are de facto describing Discord. TMK, nothing is encrypted on their side, so they have full visibility and full access to everything everyone posts.

Indeed, I'm saying not to use it (I've just edited it so, as apparently there was some confusion). I'm saying that it is legitimately a better idea to join "yet another server for a single purpose", in the sense of actual independent servers like different forums, rather than the parody of the term that Discord uses.

And, maybe it is just me, but I cannot for the life of me use it in any productive manner: search sucks, there's no way to properly split/stick to a topic, no proper message threading, and the usability drops to nothing if one joins more that 10-15 servers.

There is that indeed, it is being shoehorned into roles it is most definitely not applicable to in any reasonable way. It actually does structured chatting worse than [Slack](https://en.wikipedia.org/wiki/Slack_(software)) (which itself should be avoided due to the aforementioned corporate risk, clones like self-hosted mattermost work) which is itself not a replacement for forums, email threads (client-side) or NNTP.

Why do we continually go in the wrong direction when it comes to user experience and usability of software?

I've been asking myself that same question for a while.

it's a plague. Discord is good for a few things, but mostly a product for zoomers with an attention span of 30 seconds. Thats why each any every shit service has notifications too, so that you won't miss a single fart from someone across the globe.

dear god bring me back to web 2.0 in its purest form.

chatrooms are just a pain to search and navigate.

To be honest I dont even like reddit, but I guess thats what people decide to use and there is a lot of great info here,

Having it on Discord misses the point of "being searchable on Google"

Plus, a single TOS violation and the Discord disappears into the abyss.

Do you believe in spez at first sight or should I walk by again? #Save3rdpartyapps

The point is to provide only the metadata on Reddit: link, id, channel, title, etc. and not the video itself, so that it is not taken down by a copyright takedown.

I tried to do exactly that with /r/YouTubeBackup, if I notice something deleted that I backed up I make a "curation" post about it, then update it with a link to video if I find one or reupload myself to someone's wish.

Seems like a torrent tracker or P2P solution of some kind would be best. YT could reward people with positive ratios by giving free YT premium

For this reason I've been looking for a "Jellyfin for YouTube"-type service. I've found ways to automatically download new videos, but without a polished way to view/browse from client machines it often feels like a waste of disk space.

[deleted]

I briefly looked at Tubearchivist, but it's made to download and view, not just view. I already have a download system that works and don't want to have to deal with trying to set up everything again

what is your method for auto downloading yt vids, is it a batch program with yt-dlp or something else?

Use ytdl-sub (disclaimer: I am the author)

I'll try this out, thank you!

[deleted]

Second yt-dlp plus Plex. I really like Funhaus which has had some scares, and my wife really likes Critical Role. For both channels I have scripts checking a couple of times a week for any videos posted in the last six weeks that aren't already present. It's peace of mind for Funhaus and my wife has liked the solution the couple of times that Critical Role has pulled an episode after the fact.

Plus, if the Internet goes out we both have a massive playlist of media we enjoy.

I do it all with bash scripts on a Ubuntu VM, including a script that keeps yt-dlp updated. I haven't touched it in over a year, it just runs.

There's a lot here about YouTube throttling downloads, and I ran into that some when I was downloading FunHaus quickly, fearing it would get deleted. Critical Role I just throttled the initial download way down, four or five episodes a day, and let it run. These days grabbing just the new stuff it's a total non-issue.

Important to note that you should support creators you enjoy. My wife has several types of subscriptions to Critical Role and I pay for Rooster Teeth FIRST membership despite not actually using it for anything.

[deleted]

I'm sorry to hear that. :(

Yeah, every channel is different. For a long time I fought with duplicate Funhaus videos because they would post a video, pull it down for a minor edit, and re-post it. I ended up telling it to not grab any video younger than 36 hours as a result, but I can see how it would be very different for a more volatile channel like you have.

[deleted]

It's such a great way to collect and curate your YouTube content.

For the "episode" problem, yt-dlp will let you use both more detailed timestamp (down to the second!) and the video ID. It doesn't hurt to do something like YYYYMMDD - [Video Title] - [Video ID]. Then you just categorize the library as "Other Video" to keep it from trying to match it to shows and seasons.

Maybe this small tool might also be of interest to you, I rarely see it being mentioned/known by people while I think it fills a niche job well. I feel like it’s not widely known yet:

https://github.com/Mikescher/youtube-dl-viewer

/u/TheGleanerBaldwin /u/FullStackOverflowed /u/Late-Night1499 /u/Hippocratic_dev /u/Ttylery /u/FrankMagecaster /u/Frederik2002 /u/Office_Clothes /u/Goolashe

I Think YoutubeDL-Material (docker link here) might be what you're looking for. The problem is it has quite a few issues that make it a struggle to actually use.

For example, I've not been able to actually run my Unraid docker instance of it because it currently just bootloops due to trying to load too much data on startup, seemingly from it trying to start several channel pulls at once (even though there is a limit set). There doesn't seem to really be a fix at the moment, either, let alone a response to acknowledge the problem. You also need to run a separate database service with it to get any real usability out of it once you have a lot of videos (they go over that in the readme/setup).

It's fantastic when it works, but it has to work to be fantastic :/

Tube sync + jellyfin works very well,

you can target channels or playlists and can customize everything like vid quality or check frequency from the web interface

VPN to home + SMB share of your youtube folder? The Android app ES File Explorer turned to shit, but it used to work well to play videos over network (SMB and FTP!) on my phone. I'm certain it's older versions suck less, except for the vulnerability it had.

[deleted]

Hard to say, but it's clearly in the exabyte range.

The average accepted number is that Google's datacenters manage 15 exabytes of data. Now whether that's 15 exabytes of data in each datacenter, or 15 exabytes spread across all datacenters, I have no idea.

I know that 15,000,000 terabytes is a pretty insane amount of data though.

Yet google cannot compete with the god of data hoarders - NSA.

Its in Yottabytes plus 25 years data retention if memory serves me. Salute to the nerds at NSA.

I find it difficult to believe the NSA stores multiple yottabytes, or even a single yottabyte.

A single zettabyte is 1000 exabytes and none of the major storage providers have claimed to be managing even one zettabyte.

https://seedscientific.com/how-much-data-is-created-every-day/

NSA didnt use providers, they build their own data center. Their data retention is 25 years, so what we discuss now is still readable in 2050 at minimum. As an agency that listen, view, record, log etc etc etc everything, its not hard to believe it. Never Say Anything is truly god of data hoarders, do not underestimate their nerds.

NSA didnt use providers, they build their own data center.

And?

You're thinking of cloud storage providers. Nevermind that most of the three-letter agencies have cloud contracts with all major providers (Microsoft, Amazon, Google).

Even if they build their own data center, they don't build their own hard drives. A single yottabyte is 50,000,000,000 hard drives, if every hard drive is 20 terabytes. That's 50 billion hard drives, with a capacity of 20 terabytes each.

There hasn't even been 50 billion (50,000,000,000) hard drives produced in the history of the world. The largest year for global shipments of hard drives was back in the 2010s... it might have been 2010 itself, and it was something like 630,000,000.

There's no way the NSA has close to even one yottabyte. We don't have the density, and we haven't produced enough drives yet.

EDIT: Just to really hammer this home...

Let's say the NSA bought all of Western Digital's Red Pro 22 TB capacity. Every single disk. Let's say they even got the government discount of 25% off the retail price, so $450 per hard disk... that'd still be $20,454,545,454,545.45, or $20.5 quadrillion dollars.

This dude estimates the Planet Earth to be worth $5 quadrillion dollars - https://www.mentalfloss.com/article/636789/how-much-is-earth-worth

So basically you need to sell four (4) Planet Earths to someone, and maybe one of Jupiter's moons, just to have enough money to buy all those drives from Western Digital. Now what are you gonna store them in...? Better get another dozen or so Planet Earths for the JBOD enclosures...

50b drives would also consume 140GW of electrical power while idling or 25GW while in Standby.

Why you limit to HDD only? Even some of us hoarders asking about alternative storage media - cds, tapes watever and somehow NSA stuck with HDD only? Their deal with providers/FB.google etc usually is like what the name say- provide data.

Hint: They listen to all internet traffic, and that alone is 20TB or so per day? dunno, but....with 25 years data rentention do you still think they keep it inside churning HDD?

As an exercise: They say we have 6 degrees of separation. NSA phone number tracking is 7 'circle' from you, and that just phone number, and dont forget, their IoT expansion, that capability alone is scary.

Hint: They listen to all internet traffic, and that alone is 20TB or so per day?

Sorry, you think the total amount of all internet traffic per day is only 20TB? I wouldn't be surprised if there were a few people on this sub who are responsible for moving that much data themselves in a single day.

Yeah it is not 20TB that is for sure.

If i remember right I think there are estimates it is in the 3-7 exabytes per day range of total global traffic.

I could see that if you think there are about 4 Billion internet users, that would be a something like 1-2 GB of internet usage per person per day.

No one has a yottabyte of capacity. Anywhere.

That's what I'm trying to get you to understand.

The cost factor alone makes it impossible, but once you start looking into the amount of storage media required, that also makes it impossible.

Not to mention the space and manpower to store, operate and manage 50 billion hard drives. This is clearly not even remotely feasible for many reasons.

Wow, thats surprising to me as a chia farmer. I didnt think chia netspace could be double what google manages.

What format are they stored in?

If you mean in terms of file system, Google File System.

https://en.wikipedia.org/wiki/Google_File_System

I mean codec, don't they transcode everything you upload to some weird format?

As far as I know, and apparently as far as this Wikipedia article knows, GoogleFS stores all data in 64 MB chunks.

All Google data everywhere in the world is stored like this, including YouTube. As far as what codec does YouTube use, it's a mixture depending on different factors - AV1, VP9, and H.264.

As a really wild guess, I think 15 EB might cost around $300 million today.

That would likely only be known by google and confidential data. They say stuff like "500 Hours of new video is uploaded per minute"

More than one of those big HDDs. honestky idk. 15-25TB.

Do you mind uploading those missing one into torrentsss? Happy seeder

The remaining bandwidth is too damn small!

One will go to torrents, others to archive org. I'm not sure if I will make a torrent but push each video to archiveorg individually. Sounds like a decent idea after all.

IA makes torrents automatically i think

It does a very bad job at creating a new torrent (obsolete padding files and other issues) AND downloading (incomplete/broken).

Damn, they really haven't cleaned that up by now?

Last and final time I tried was in 2020. Only a handful of my own torrent uploads made it completely through. So i've got to fix them now too

It’s economics too. A client of mine was earning $300k/yr two years ago, and then the algorithm changed and now his income is down to $500/month. It wasn’t worth it for him to maintain the channel anymore. Despite having 1.2M subs, he took down every video and got a job.

Why take it down? Its literally passive income

Gotta manage the comments sections, maintain the channel, etc. I asked the same question as you, fwiw.

Probably pissed that Youtube started being greedy and keeping more of the cash made from that content.

What do you do if you don't mind me asking

Financial Advisor. It’s a fascinating view of how money moves through different types of professions sometimes.

Yup, sounds about right. I believe I was at ~4 or 5% before the the unlisted older videos became private. If I had to bet my own archive is probably closer to 6 or 7% at this point...

My archive of ~750k videos has ~100k deleted, around 13%

Interesting thanks

That's a bit alarming. Sad to see such things happening.

It's inevitable. They've studied link rot, generally find that the half-life of a link is around 2 years.

Can you elaborate on link rot? Didn't even know yt would prune links

I think they are saying that, in general, any link on the web would be "dead" in an average of 2 years.

Nah half life means that half the links disappear every 2 years. The average will be more than 2 years.

The average life is the half-life divided by ln(2), so that would be 2/ln(2) = ~2.89 years, or about 2 years 10 months and 21 days.

Doesn't this include content that's still available but broke links in violation of "cool URLs don't change"? Obviously not good, but not as dire as the content itself disappearing permanently.

The content of this post has been removed by its author after reddit made bad choices in June 2023. I have since moved to kbin.social.

Download: yt-dlp with custom Bash scripts to make it manageable and updatable (one script to download a video to archive, one to download entire channel to archive). Nothing spectacular here but it makes the life easier. I mostly make use of "--download-archive" files, they contain the ID of each downloaded video.

Checker: I wrote a custom script to ask the official Youtube API "what's the video's status?" Then I fed these --download-archive files and let it run for half an hour. I was a little worried about the API quota (yes they have strict limits), but I didn't reach it.

If you can imagine an ad-hoc Linux machine scripting that's here and there, all over the place but somehow works - yes it's me at the moment. I started small too.

Will I make the scripts available? Absolutely, but I'm not ready yet.

[deleted]

Yeah I was worried about the quota, but it seems it's counted per request? I can fetch status for 50 videos at once per request and that's how I implemented it.

This tool is available if you want to start https://github.com/w0d4/yt-backup

This may be a dumb question to you folks, I just discovered odysee, would something like odysee not be a good choice since you would have to upload each video, and that's why a way to scroll through the "database" without uploading would be a better option?

I dont see a point reuploading dead stuff to Odysee. They need creators, not necessarily years old archives no one cares about.

Great point tbh

How would I go about uploading to archive.org?

I used to run a YT channel where I restored (video and audio) old documentaries, interviews, educational content. Almost all of it was not readily available anywhere on the net.

I still have the high quality files I rendered out. Like 1000+ videos I think.

How does archive.org work for that? If something gets hit with DMCA, does everything I uploaded get taken down? I don't want to do a ton of work and see it all get deleted.

They currently hide all youtube videos from search with noindex. I believe they don't delete content but would hide the item totally.

The URL structure for youtube videos there is "youtube-". Though you have the original videos, I'd say that doesn't make it a youtube reupload anymore. If I were to upload the videos, I'd reencode them to something sensible in size (whereas the recommendation for YT is the highest bitrate you can afford to upload) and upload videos individually and provide the original link in item description.

1000 videos is no joke, the only way to upload to archiveorg in bulk is their ia-cli tool. However it requires some serious preparation. One "easy" way to use it, is to create .csv spreadsheets that declare the entire upload and files. Their web uploader is awful, best suited for a few 100MB uploads.

Maybe your stuff is better suited for a p2p network where you can just share an entire folder at once. Like DC++

No, they delete too, first was books, now some YouTube videos, and some sites are excluded from storage there.

Hidden is not deleted. I agree that it's unusable to us as users but the controversial stuff is kept hidden.

If you save a youtube video link with the wayback machine it can save youtube videos for you, but sometimes it fails and you need to try again later. I have seen 3 hour videos saved so there is not a limit there. It just saves it at one quality though and you need to be able to remember or access the link of the video you want to find later.

Welp, now I’m worried. I had been putting off some archival for a while now and I fear things I love from a younger YouTube may now be gone.

Yeah, if anyone on this thread can hook me up with Terry Davis "greatest programmer of all time" and all his best of videos id be really grateful, youtube has deleted it for offensive content apparently

Here's a channel that has republished all of their vids from archive.org

Long live the archive community

Are you using or have you heard of a gui to browse your downloaded videos? Or do you just have them lying around in folders? I'm thinking of something like YouTube itself. Your words are inspiring which is why I'm gonna take a look into this topic to backup channels I like.

EDIT: it looks like youtube dl and tube archivist are the way to go. Can you confirm?

You should use the updated version "yt-dlp"

I took a break from my archiving activities for a long time because I couldn't figure out my low DL speeds, and thought it was a problem with my setup. Turns out that youtube-dl is out dated and downloads videos extremely slow.

I looked at tubearchivist today and it doesn't have even once the word "archive" or "archive.org" mentioned. Maybe it does keep videos in an easy folder format on the disk, but don't like it by the looks of it.

I have my subscriptions in 720p that I watch/transfer to phone and separately archived stuff I'm not interested in following all the time. Yes folders.

I've set up Tube Archivist today. I agree that it's not pretty looking but I do prefer it over just having a bunch of video files in a folder. Looks like I will need another HDD.

Now I'm leaving it for the night to do some work. Let's see how it looks tomorrow.

It does look good, I must say. It was enough to install Tube Archivist as it already contains yt dlp. But it's 3 dockers containers that have to be installed. One is for elastic search and the other one for Redis. Wasn't too complicated. I have also applied some custom CSS via a browser extension, to get rid of that green GUI and made it black/grey. Looks more appealing now. Wish there was a better way to add CSS to it.

Thanks for the suggestion to use Google Takeout. It sucks there's no easy way to get your Youtube subscriptions anymore. There used to be an "export subscriptions" button at the bottom of this page, one of the many features Youtube has helpfully removed for some reason.

I also made a script to extract it from the main page, but to get the data you'd need to go in browser devtools, get the request data etc. Either way it's cumbersome but Takeout is the easiest to explain.

I've done it with my YouTube channel and that is to sign up to Odysee. Its free and I linked it to my YouTube channel. So now when I upload it gets synced to Odysee. Odysee videos don't get removed like YouTube. What with Odysee being decentralised.

Its a great way to archive some channels as well like the eevblog as when you view on Odysee the videos download at full quality.

Its a good thing I horde files as had one video up about a shady carparking fine company here in UK. Their website is insecure so I pointed it out with a video. Was up for a year when they decided to claim it under copyright grounds and I got a strike. And thats where YouTube fails. The whole video was my content the parking company were doing it to just remove the video. So I uploaded it to Odysee instead where it remains. Also wrote a blog post about their shady practices and their still insecure website.

Something similar happened to me too, lost my whole YouTube account!

I was actually looking into how to download entire channels and had no idea, are you saying that YT-DLP has that feature built in?

Yes, yt-dlp can archive an entire channel. All you have to do is enter in the home page's link after you have your code worked out. i.e. if you wanted to DL Pewdiepie's channel it would be

https://www.youtube.com/user/PewDiePie

Again, that's after you got your code worked out though. I believe there's some help in the sidebar, but I created my own. IM me if you need any help.

Yup. Doesn't even require any tricks; just throw a channel URL at it, and it will figure the rest out.

Which 6000 vid channel got deleted?

social/historical/political/news where guests presented a certain topic. you can guess the recent reason

Even worse, some of my saved videos turned subscription only...

tubearhivist

So ur deleting the whole youtube ?

Doenloading*

No, I download what I care about.

Running this 2 times a week

i=0 ec=0 while i="$((i+1))" ./yt-dlp 'https://www.youtube.com/c/Gdconf' ec=$? [ $ec -ne 0 ] && [ $i -lt 3 ] do :; done echo "Download completed in $(( i )) tries --- exit code: $(( ec ))" exit $ec

Hey, how do you keep track of the videos that are still online and the ones that got deleted or unlisted?

https://old.reddit.com/r/DataHoarder/comments/xopxll/personal_youtube_video_archive_stats_19_gone/iq0zqlj/ - a custom script

Right! I always loved Ben Sinclair in the higher maintenance series back when it was a series of shorts and have been kicking myself in the ass since they all went missing.

I should have backed it up and your reminder is appreciated, as I consider what I might want to add to the archives asap