There's a bunch of heuristics and guesswork to try and handle typical cases. Yes, a lot of regexes.

It does retry for a while because sometimes the torrent gets renamed, or more info gets added to AniDB.

I've had the thought, but generally liked the idea of handling most of these cases in code, as it effectively improves the automated matching, whilst handing off the task means more manual work for someone.

There are backups, but I almost never use them. Restoring a backup often means you have to reprocess a bunch of stuff and deal with potential mismatches etc. It's basically a last resort.
Fortunately the design of this system has some redundancy (database is replicated from the updates server to the web servers), so I can often use that.

There's rate limiting here too; I'm probably more generous than Nyaa.si though, plus AT would get less traffic than Nyaa.
Otherwise I generally keep an eye on traffic patterns to see what looks problematic.

I've actually avoided doing that. Not having to co-ordinate has its perks, and I always wanted the option to walk away. Also harder to label the operation a 'conspiracy', which requires at least two people.

I'm thinking of putting them in a 7z (or maybe multiple). The smaller packages (like NZBs) will probably just be on the storage server.
The larger stuff will probably be a torrent.
I'm not sure if anyone's interested in the screenshot data, which is the largest data set.

I haven't really been keeping up with it, so I don't know.

Thank you for your service all these years.

This is a fantastic site. Thank you for over 10 years of amazing service. Hope everything goes well for you in life.

Thank you for all the years you maintained this site. It'll be missed! It was a great place and you put a lot of work into it. I appreciate that you're giving a few months of notice, because that increases the odds of someone being able to get a replacement up and running. I hope very much that someone will step in and do that, but even if they don't. the extra months of notice mean people have more time to look for inferior alternative options. I wish you the best of luck for the future.

Thanks for everything throughout the years. Always loved getting everything on AT because it was so organized, informative and user friendly. Felt like I found the holy grail of anime when I stumbled upon this site by accident a decade ago.

Best of luck on your future endeavors.

Life is fleeting, thanks for being a part of mine! (￣^￣)ゞ

The AniDB tagger does sound like a lot of manual work. You can probably look up the episode on anidb's database, but only after someone bothered adding metadata for it, which might take hours to days. So I'm guessing you have to have a lot of regular expressions to detect each group's naming scheme? Does this have to run regularly to re-classify episodes that are unclassified?
Recently I saw someone recommend "thexem.info" to some release group that posts on nyaa, but doesn't care to post on other sites with more stringent metadata requirements, in the context that they should post on other sites too.
Let's say, for example, CR decides to use absolute numbering for one anime, while for another it uses season numbers, other times this is mixed.
This seems like a pain to manage even for someone doing releases for a single group, as opposed to needing to manage this for every release out there.
Of course, someone making a clone of AT could just not classify it to save on time, then the burden is on the user to try/search all the names groups use until they get what they want.
It wouldn't be anywhere near as comfortable as AT, although I've encountered a number of times where an episode ends up in "Unsorted".
Maybe another option would be to offer users a way to contribute this metadata, something like anidb does, but even if that reduces the burden on you, now the burden is on moderators to resolve disputes or avoid abuse, and that starts complicating things again, not to mention it complicates backend implementation considerably.


> There's some degree of unpredictability that goes with managing services like these, e.g. a disk dies, DB corrupts itself, some AI bot goes crazy scraping things consuming lots of resources etc.
That's probably unavoidable, although at least you have backups? AT actually seems quite snappy, meanwhile TokyoTosho seems to be constantly dying probably due to high number of bot requests.
I often access this site through a proxy/VPN/Tor without any issues and unlike many sites you don't block them in an attempt to deal with bots. I've also seen some bots cause serious problems fo Apache, but nginx-like httpds do much better.
Any suggestions for how to deal with this? Nyaa tends to ratelimit, which sort of works, but I know many people tend to use your site's mirror of nyaa because they find their VPN is often ratelimited. TT doesn't, but is quite slow. Sites that use WAF's like cloudflare often break automation (RSS and others).

> There's also the me factor - as long as AT is around, I'd feel compelled to improve things. So I think I just need to walk away from it entirely.
Never found anyone to share some of the burden with you over the years? I suppose it is often hard to find people you can trust to handle things.

I think what's on everyone's mind right now is what will we transition to, if any of us should pick up the torch, how much work it would be.
Compared to the past, it seems fewer people are willing to do projects like these in the current year, but I hope to be proven wrong.

Not directly related, but how will the NZB and subs be released? A torrent? Maybe archive.org post? Posting it to usenet? A number of these? I'm assuming there's no point in scraping the NZBs/subs yourself as it would add a lot of load to the storage server, while an actual upload of all those tiny files would be a lot more efficient.
Do you know of anyone else that posts to usenet current releases? I'm aware of one site, but it seemed private and with questionable requirements like "no collectors", whatever that means, can't even make a large anime archive based on past postings like that?

Currently the average public anime pirate is in a tight spot: I'm aware of 2 large archives going back to 15 years of releases or so, and of course AT's nzbs.
One of those archives recently shut down (monthly costs were at least 15x yours, owner paid them out of his pocket too), the other is kind of incomplete, and with AT's nzbs going away, the only options left are private trackers for many, at least for older releases, not a pleasant spot to be in.

If this previous comment is yours, I don't understand your question, so you'll need to clarify.

We will miss you A.T~nyaan

This website is irreplaceable for many things

This is very depressing news. I love this site. I don't think there's anything like it and I worry there's never going to be again.  This community was great for NZB I know it feels pretty much irreplaceable.

I understand you need to rest and move on from this side but this is a dark day indeed.

Thanks for everything.

There are few other sources of subtitles, fewer well-curated catalogues.  Will be much missed.

thanks for everything ;-;

I found out about anime tosho just a year ago. And so far its the best place to download anime for me. I don't know if you will change your mind about shutting down or not but i wish this site to run a bit longer.

thank u admin for everything

If possible share you nyaa queries, please!!! It would be a huge help!

But thanks for all the fish and animus.

I guess I will be graduating from anime.

This was one of the places that still worked on XP.

sigh

That is good to hear.

You can already download partial database exports here. I'll make a full final dump once updates stop.
I'm looking to see what I can do with code releases. Some stuff is already available here.

I have personal income sources and not many responsibilities outside of AT.

In general, not a lot, and there's quite a number of days where I don't post at all. A lot of questions actually get answered by others here.

OHNO ;( I just found your site late last year :/ Hate to go back to Nyaa, which is a mess and you were the only quality indexer with primo AV1 releases.
I see the work you had to pour into this and I know for a fact it took a toll on you, even if it's part of your heart.

From the deepest of my heart - THANK YOU! <3

The cost of hosting this site is probably less than 2% of the admin salary, the average European salary. As the admin said, this website is very cheap, but the time spent is very long is this hobby.

It's amazing how dumb people really are.  15 years of running this site out of pocket... I believe... since I never really saw a donation link like a lot of other sites have. Do the admins have full time jobs, families, a life away from the computer screen? - (horror stories for most of the people who come here)  It's always me me me and not them. How much time does it take to answer people's questions and help with problems day after day? - (not counting the trolling, spamming, and flame wars)  As a downloader I thank the people who took the time to give us these uploads and thank Anime Tosho for making them available in an easy and organized way.   You guys will be missed.

Make your API, database and other resources publicly avaliable and open source.

Thanks a lot, will dig into it :)

I'll try to release all the backend scripts during the coming months, but for your question, the general gist is something like:

- run mkvmerge --identify --identification-format json [filename] - this will give you a list of everything with related info
- run mkvextract [func] [filename] where [func] is attachments/tracks/tags/chapters. For attachments/tracks you'll need to build the relevant command from the identify output above

There may be a few special cases to consider (like Vobsub being two files), but that's about it.

Hi Admin, thank you for your great service over all these years and good luck in the future adventures :)
Could you please show an example script on how to get all the subs, chapters, tags and attachments from a specific release in a neat "animetosho-7z" style? I assume you use mkvextract, mkvinfo and... what else? Can't figure out how to get subtitle file names in language-tagged "track3.eng.ass" format. Would be so great to continue stacking up attachments in a good-old-familiar way after The_Black_Day comes...

They need to make their data and API public.

And now the admin is killing the site and betraying the anime piracy community.

Anime Tosho's quality as a site is unmatched. Think of this as one stop shop where everything is available through one site. I mean how awesome is that. Still, I'm very sad to see this go but I'm really grateful because this site exists. Godspeed, admin.

As an encoder that made countless releases thanks to the subs AnimeTosho made more easily available to everyone, I'm very grateful for everything you did and the quality of the service for so many years. We will be missing you, many thanks again and hope your next adventure is a good one too!

Not managed by me, but here's a site where users share subtitles: https://kitsunekko.net/

I'm with you on not liking their API (or their general stance on aggressively protecting "their" data).
I don't intend to antagonise anyone though (and my releasing a scraped copy of AniDB might make them more aggressive with restrictions).

If someone wishes make some publicly accessible DB dump of AniDB, there's nothing really stopping them other than the effort required.

Thanks for your thoughts, the idea of shutting down some parts has crossed my mind, but I don't think it'd work out.

As far as backend processing goes, most of it actually runs itself. What probably takes the most effort maintaining would actually be the AniDB tagger (people come up with new naming schemes, AniDB lacks aliases people use, people debate on which season of Demon Slayer to label, fixing up mislabelled stuff etc). Stuff like retrying uploads has mostly been automated, so I only need to keep an eye on it.

There's some degree of unpredictability that goes with managing services like these, e.g. a disk dies, DB corrupts itself, some AI bot goes crazy scraping things consuming lots of resources etc. Average workload might be low, but these events can throw a wench in if you had planned your time otherwise. Other stuff includes keeping on top of relevant news/comments, maintaining opsec etc.
Unfortunately a lot of this can't really be eliminated by getting rid of parts of the system.

There's also the me factor - as long as AT is around, I'd feel compelled to improve things. So I think I just need to walk away from it entirely.

Thank you for everything you've done over the past few years!