I thought it would be funny to include NN-generated text about anime, the way “This Rental Does Not Exist” included char-RNN generated text about hotel rooms. But the accompanying GPT-2-117M snippet generation turned out to be a little more tricky than faces.

This leaves 70k clean anime-themed gibberish text samples which can be loaded by the same randomization process as the faces—hours of fun for the whole family.

The final generated GPT-2-117M text samples have a major flaw: it will frequently end its generation of anime text and switch to another topic, as denoted by the token <endoftext> . These other topics would be things like sports news or news articles about Donald Trump, and would ruin the mood if included on TWDNE. The problem there is that gpt-2-PyTorch does not monitor the model’s output, it just runs the model for an arbitrary number of steps regardless of whether <endoftext> has been reached or not. To remove these unwanted topic changes & leave only anime text, I run each text file through sed and it exits/stops processing when the token is reached, thereby deleting the token & everything afterwards:

While running the first pass, some GPT-2-117M instances will fail for reasons like running out of GPU VRAM (each instance takes almost 1GB of VRAM and my 1080tis only have ~10GB usable VRAM each). These can be fixed by looking for empty text files, extracting their ID/seed, and trying again:

# ife it was fun it made my eyes roll I felt that it was great, it fixed any that I"

# feel that I am not here to argue that "it wasn't even good but it was awesome and it changed my l \

# x2, x3. But for a few moments of enjoyment x3 was the ending and I \

# re it all gets a second time. This anime x2 is not only a really great game and it \

# is about (Katsuko) x2 and x2 was my first love story, x2 is one whe \

# hat I did not watch the whole series for my liking was "AoT, the End". The first episode \

# good, I have some I wouldn't name because I don't want to spoil them) 土安黄满收

# into all the other titles because I think there are too many titles you can download that might be \

# he episodes as well, but for the sake of brevity, here is an overview of the games I saw (I won't go \

# In my second review, I went by many titles, which you can find all here to download t \

# series) was shown as an early preview of the game which can also be downloaded fro \

# irst episode (after a lot of scenes with a lot of bad guys, which is the end of episode three of the \

# on the Tokyo Otaku Mode list (I'll tell about that soon, I promise) . The suitable ending for the f \

# me Funimation's Anime Festival (and it's a very popular anime) and was nominated as a top 100 anime \

is the latest hit. In the first episode of this anime, "

I couldn’t figure out gpt-2-PyTorch ’s minibatch functionality, so I settled for running it the simplest way possible, 1 text sample per invocation, and parallelizing it with parallel as usual.

I found that feeding in a long prompt with many anime-related phrases & words seemed to improve output quality, and I noticed it would easily generate MAL/Wikipedia-like “plot summaries” if I prompted it the right way, so I went with that. (I threw in a parody light novel title to see if GPT-2-117M would catch on, and as for “Raccoon Girl”, that was a little joke about The Rising of the Shield Hero which I’d seen so many anime memes about in January/February 2019.)

I used the gpt-2-PyTorch CLI package to run the downloaded model without messing with the original OA code myself. The repo explains how to download the GPT-2-117M model and pip install the Python dependencies. Hyperparameter-wise, I roughly approximated the OpenAI settings by using top_k=30 (OA used top_k=40 but I didn’t see much of a quality difference and it slowed down an already slow model); the temperature hyperparameter I did not change but gpt-2-PyTorch seems to use the equivalent of OA’s 0.7 temperature.

The full GPT-2-1.5b model which generates the best text has not been released by OpenAI, but they did release a much smaller model which generates decent but not great text samples, and that would just have to do.

“okay my brain literally cannot comprehend the fact that these aren’t just real drawings and were made by a computer. what” Ash

I needed to feed GPT-2-117M such a large prompt because it is a general language model, which has learned about anime as merely one of countless topics in its giant corpus. The primary goal of training such language models is to then use them on narrow task-specific corpuses to ‘finetune’ or ‘transfer learn’ or provide ‘informative priors’ for a new model, with the global knowledge serving to model all the generic language (if you have a small corpus of medical clinical reports, you don’t want to waste precious data learning something like capitalization) and getting the model up to speed for the new corpus, and tapping into hidden knowledge of the original.

By the same reasoning, training GPT-2-117M on an anime text corpus might lead to better or at least, funnier, generated text. nshepperd wrote some GPT-2 training code for finetuning and retrained GPT-2-117M for ~3 epochs on Canggih P Wibowo’s 2016 Anime News Network scrape, using the title & plot synopsis fields ( TITLE|PLOT SYNOPSIS ) I have also used the finetuning code for Project Gutenberg poetry and wrote a GPT-2-117M finetuning-training tutorial using the poetry as a running example.

GPT-2-117M appeared to start overfitting quickly and training was stopped, yielding a final trained checkpoint we call “GPT-2-anime” (441MB). To use it, one can simply unpack it into the GPT-2-117M model directory, overwriting the files, and then running the OA code as usual.

Sample unconditional output:

======================================== SAMPLE 1 ======================================== a boy who works in a magic shop. Her magic power is not despite the power these animals and the atmosphere grows stronger every night. She can't use magic at all so she can get her witch powers back. Shirobako: My Island (TV)|"Abe Eigaku is a rookie high school student who is about to have a battle dyed by the colors of samurai and a sordid fangirl who only wished to destroy the hell that befall him. One day, he runs into a samurai and strikes a deal. The deal: If he can get to, then distract the soaked Edo sword will be able to change his form. The deal was an attack, and the young boy is left with a injuries and he is left with bitter injuries in the hands of the Kumano family. From the experience that inherits the "gods' ability" in him, he comes into an outdoor fighting ring with the Gices. There he meets Setsu, a former show comor who has changed his life---like his father, himself as a ""demon"" of the Gices, and a member of the Gices. At the family's house, he makes his ultimate weapon, the three-headed demon Gekkou, Kabuto. Yukino comes to his aid, and their rivalry soon evolves as they head into more serious and skillful monsters. With his ability, Yukino faces others both fierce and charming. The series follows their everyday lives, together with their personal ones and each other, as they go through their daily lives and try to grow up as close to each other as possible to the one they love. Berserk: The Golden Age Arc (TV)|Guts is a young man who has been accepted into a powerful industrial city by a mysterious woman named Null Mother, and is then summoned to the city of Midland. The first he ever summoned to his world was the Ghost Ship, a brutal battle that has been held on for thousands of years. Now he is on the eve of the Bloody War, when the inhabitants of the little world try to destroy the automated sentry. However, Guts cannot stop them and blithelyave their fate as the inhabitants of Midland are devourant of the very world they reside in. Lupin III: The Legend of the Gold of Babylon (movie)|Deep beneath New York city are buried tablets that tell the tale of Babylon's gold that was lost during Babylon's destruction. Lupin is interested in finding this gold, but will have to deal with two mafia families and Zenigata during his quest in unsolving the mystery of the tablets. During Lupin's journey he encounters an old woman who has a connection with this treasure. Fortune Arterial: Akai Yakusoku (TV)|Kohei Hasekura has lived a live of transferring schools for boys all his life. He's been the site of a recently bankrupt well known as Moon River for years and has been looking forward to getting it back. At his new school, however, he sees the school's most wanted fight and gets a duel to choose the strongest in the class. The battle is about to be fought by a monster called Hildegarn either. Hasekura's friend Orca Buchizouji gets dragged into the fight and loses a majority of key to the duel. Hasekura fights alone against the bullies and punks from the previous season, and when the duel is over everyone comes to an end. Gunslinger Girl: Il Teatrino (TV)|When the Social Welfare Agency investigates the disappearance of a operative, their inquiry leads them right into the lair of their rival, the Five Republics. The assassin Triela infiltrates the hostile organization, but her search is cut short when she finds herself staring down at the barrel of a gun... Yuruyuri Nachu Yachumi! (TV)| Magical Sisters Yoyo & Nene (TV)|Yoyo and Nene are witches living in the Magical Kingdom who specialize in curse and decusing. They are negotiating with a woman who wants to find her sister, a witch that disappeared twelve years ago, when a monstrous tree appears in front of Yoyo and Nene´s house. Embedded within the tree are unfamiliar buildings which prompt Yoyo to explore these strange constructions. During her scout, Yoyo unintentionally ends up thrown into modern day Japan. He is taken away by a doll-cat who falls from a magical tree on his way back. He is taken back to Japan by Gogyou, a girl who is their own, who falls from a magical tree. He and his friend Nanami lands on a magical journey to recover the lost magical powers of the magical stone. Sakura Taisen: Ecole de Paris (TV)|"Anastasia was supposed to be invaded by various people. She is so damaged that she can't see anything. But her friendces

There are two immediately visible problems with the GPT-2-anime output.

Too Short: prompted/conditional output from GPT-2-anime looks… much the same. Apparently what happened was that during the finetuning, GPT-2-117M learned that the format was incorrigible and internalized that to such an extent that it would only generate title+short-synopsis-followed-by-new-pairs. If a prompt is provided, it might influence the first generated pair, but then GPT-2-anime knows that the prompt is guaranteed to be near-irrelevant to the next one (aside from perhaps franchise entries alphabetized together, like sequels) and can be ignored, and each title+plot synopsis is only a few sentences long, so GPT-2-anime will transition quickly to the next one, which will be effectively unconditional. This is annoying but perhaps not that big of a problem. One can surely fix it by using a dataset in which the title/synopsis is then followed by a much longer plot description (perhaps by merging this ANN scrape with Wikipedia articles). And for generating random text snippets for TWDNE, we don’t need the control since the unconditional samples are all on-topic about anime now (which was part of the point of the finetuning). Low Quality: some outputs are thoroughly unsatisfactory on their own merits. The first entry lacks a series title and any introduction of the premise. One entry has a title—but no plot. And while most are shorter than I would like, the last one is simply far too short even for a plot synopsis. This is a problem. The outputs are moderately interesting (eg “Shirobako: My Island”) and provide a good plot synopsis which is a nice starting point, but as-is, are unacceptably low-quality for TWDNE purposes, since they are so much shorter & less interesting on average than the long prompted GPT-2-117M examples were.

However, the 2 problems suggest their own solution: if the GPT-2-anime plot synopses are good premises but GPT-2-anime refuses to continue them with plot/dialogue, while GPT-2-117M generates good long plot/dialogue continuations but only if it’s given a good prompt to force it into anime mode with useful keywords, why not combine them? Generate a wacky plot synopsis with GPT-2-anime, and then feed it into GPT-2-117M—to get the best of both worlds!

As before, sed will take care of the <endoftext> markers (& if one doesn’t like the synopses’ “Source:” end-markers, they can be removed with sed -e 's/ (Source: .*//' ), the titles can be italicized, and we can post-process it further for quality: if either of the synopsis or plot summary are too short, drop that text snippet entirely. This should yield long consistent high-quality samples of anime-only text samples of the form title/synopsis/coherent plot-summary or dialogue or article about said title+synopsis.

After generating 12MB of GPT-2-anime plot synopses yielding ~35k synopses (assuming some percentage will be thrown out for being too short), I fed them into this script for GPT-2-117M similar to before, to generate text-snippets #70,000–100,000:

TARGET= "/media/gwern/Data/thiswaifudoesnotexist" I= 70001 fgrep '|' /home/gwern/src/gpt-2/samples | sed -e 's/^\(.*\)|/_\1_: /' | \ while IFS= read -r PROMPT ; do if [ ${#PROMPT} -gt 150 ] ; then PLOT= " $(CUDA_VISIBLE_DEVICES= 1 python main.py --seed 5 --top_k 40 \ --text "Japanese anime manga light novel. $PROMPT . Plot \ Summary. In the first episode of this anime " | \ sed -e '/<|endoftext|>/{s/<|endoftext|>.*//;q}' ) )" if [ ${#PLOT} -gt 250 ] ; then echo " $PROMPT In the first episode of this anime $PLOT " \ >> " $TARGET /snippet- $I .txt" echo " $TARGET /snippet- $I .txt" cat " $TARGET /snippet- $I .txt" I=$(( I+1 )) fi fi done

Sample combined output:

/media/gwern/Data/thiswaifudoesnotexist/snippet-71240.txt _Mayonaka wa Junketsu_: His older brother has been hospitalized for about 5 weeks for about 6 weeks. Because of his unaccidental death, Ryouta's parents send him to live with his uncle, Yuuichirou. To his surprise, Yuuichirou is actually the son of his father's lover! Will Ryouta get to know Yuuichirou? (from B-U) In the first episode of this anime 【The Second Episode】, the story takes place in Koegi-ji town. After Yuuichirou and his family leave to live with his uncle, they encounter "Ryouta and the three of them". They learn about their real relationship, but they're not going to tell this kind of story for fear of being banned from the show. The characters' father, Kazui, is the first name his family does not use. The story begins with the trio and Kazui being together, while their parents are fighting for the survival of their families, Yuuichirou and Ryouta in the middle of a battle. 【In the Next Chapter】 I also do a series of interviews with people from the anime. ---------------------------The story is about three girls in the house. -------------------From that moment on, the plot follows them as their story goes on. They are all in shock about what has happened, but they believe that Yuuichirou is the real Ryouta. From that moment on, they feel they have a chance to change their path for the better after the events at the hospital which led to Yuuichirou's death to come home and they try to gain friends by helping Yuuichirou. ---------------There are many other characters besides the three in the story, so what is the real one? The main character is called Yuuichirou Kazui ( 招仮猿 名子 ) 【First Character】 【Second Character】 【Third Character】 ---------------This anime tells a story very similar to the show, just with characters that make you smile. If you are in a certain situation and see the person that is your favorite, there is no problem. However, you need to be mindful of the following things to make sure you know who you are dealing with. The story is very short, but there are many things that need to be done in order to make you smile. There are many times that I would write a scene that just needs to end, the most important part, of the scene, or I would have to make the scene not end at all. As long as it doesn't end with a big scene that was in a certain moment, you wouldn't be able to use your eyes to make out that someone is laughing, or a child being played or something. Just be mindful of how you are dealing with those moments and keep yourself a little calm even if they are actually happening /media/gwern/Data/thiswaifudoesnotexist/snippet-71241.txt _Shijin no Koibito_: Collection of short stories: (1) Shibatte (2) Love Me Tell Me (3) I Love You (4) I've Fine (5) Meido (6) Take Of The Class (7) Liar (8) Take Of The Precious Parade In the first episode of this anime iku wa Hana kawakukushigai wa (Love me tell me a secret), a student named Ritoye has to learn to read all of the kanji correctly. While they are doing this, they'll notice something odd, they can't talk and the class will change. They'll also notice some other things that happened, and Ritoye will get annoyed with the school and will leave her class. Once they're through with her class, she'll explain the problems and the solutions. She'll begin by looking back through their textbooks, searching for the kanji that says 'I'm on the line' and 'I'm here to talk a little bit about the story'. At some point in the story Ritoye finds out that the class used to be 'on the line' with 'I'm just being a teacher', so if someone is on the line they can start a lesson by asking the name of that person you're teaching and giving a name of the class 'I-Mitsu' which you have to pick from a few names on the internet. While the topic gets a little heated, there'll be a brief period of humor where the class becomes even funnier. One character shows off some random drawings while Ritoye is sitting by her desk, so maybe it's not a complete mess of drawings, but I was sure it would have been better if it was. The manga was originally published in February 2014. I'll update this article if anything changes, then give up. ------------- RITOYE: A school principal ~ ~ ~ My parents will be around a week or two after school. During that time, the school's president will take care of the school and it will be my job to pick up students and take care of the textbooks. We don't even need to call her. Her name is Ritoye. She has all the classes they need, but she still says she doesn't like 'those guys... I love you everyone. I really do.' I'm here, to get this in front of class, as I want to make sure she knows I'm serious. That's what the head teacher says. I'll just take everything from you to make sure she knows that I'm serious. "You, you're the best teacher in class, aren't you?" It doesn't matter how difficult, we'll give you everything. We'll /media/gwern/Data/thiswaifudoesnotexist/snippet-71242.txt _Hitodenashi no Hirusagari_: A mysterious disease. A disease. The disease has drastically differ from the usual suspects of the type of disease, but it is basically living in a hospital as a guinea pig for a few weeks. A short story of a person's persistent determination to save a person's life, and the story of a person's struggles to find it in this "comedy" story. (Source: MU) In the first episode of this anime 『Hitodenashi no Hirusagari』 (1962) the protagonist, Hiroki, begins his life. In the next episode he has to learn that he cannot leave Kyoto. From this he starts to find out more about himself and the world, as well as a great deal about the life he has created for himself. He learns a great deal about himself, as well as about the world. It goes through the story like the story of a kid having a terrible experience...or the story of young people taking it upon themselves. What Hiroki learns, however, is quite difficult to grasp because of the many factors that are involved. This is the first light novel I read that started off as a light novel, in the spirit of the first book that came out in English. When I came through the last arc of this short anime, I was very disappointed. I was expecting much much more of a series of stories and anime but instead, I found that it felt more about the plot. As the first episode started, in a similar manner to the last arc, each arc is different from the previous arc. From what I can gather, many plot points were missed when read through and how often it was made a point of focus to tell a story instead of the original. At the end of a story, you get to make your own sense of it based on how you read it. One time there was two main things left out as the main plot that needed me to know a lot more. First, I couldn't really get that the stories you saw in the first arcs were actually the people that you could actually talk to. This left me feeling more like I had lost some of my sense of immersion in a story and just didn't really find it there. For example, I started to feel that the main plot was very different from what you would see in the first arcs. The two main themes we saw in the first arcs were how to live and how to do a life. I found the story of how the life of a person is different from one that you might see in the books. The story of a person's living and living alone is also different from the story of the world that you might see in a short play. And to answer the question that many people have asked me over the last six months, and these comments, but in order to /media/gwern/Data/thiswaifudoesnotexist/snippet-71243.txt _The King Of Debt_: Souta is a rich guy, who is unlucky in love, and punishes by the rich son of a rich Chinese mafia as he is tossed about debt. But is he really bad at such a guy? (from B-U) In the first episode of this anime 何月处, Souta is forced to marry a rich kid to buy him a piece of his ass. A guy who doesn't know how to make friends with strangers, only has a friend for the rich guy's sake. Souta does this by being the only person in the house that doesn't have a job but a car, and even a friend of hers can't really call her sister 'fae'. He also has an idol he can sing on stage, and he is forced to fight for her (a guy named Katsu) against Katsu himself in the episode entitled Souta Is Fae. In a final episode, he is forced to run the family business for $100,000. What's the point in doing it? The kids only get to run the business, but when they pay out the money they want in their inheritance, Souta starts to run the shop as well. It's very simple for Souta and the guy he's running his business for. After the episode has been rated H, the new episode has been rated H... favorite favorite favorite favorite ( 5 reviews ) Topics: Drama, TV, Romance, Romance Community Audio 91 91 Seiken Hamamoto's Hacronyms 2 1 of 1 View 164 of 190 Topics: Drama, TV, Fantasy, Fantasy Community Audio 88 88 The Man From The Outer Heaven 4 3 of 3 View 164 of 189 Topic: Drama, OVA Community Audio 87 87 Seiken Hamamoto's Hacronyms 9 8 of 8 View 164 of 190 Topic: Drama, OVA Community Audio 85 85 The Man From The Outer Heaven 3 2 of 2 View 164 of 185 Topic: Drama, OVA Community Audio 84 84 Seiken Hamamoto's Hacronyms 2 2 of 2 View 164 of 189 Topic: Drama, OVA Community Audio 83 83 Seiken Hamamoto's Hacronyms 3 3 of 3 View 164 of 190 Topic: OVA, DVD, Blu-ray, DVD-R Topics: Drama, TV Community Audio 82 82 Seiken Hamamoto's Hacronyms 3 3 of 3 View 164 of 190 Topic: Drama, OVA Community Audio 81 81 Seiken Hamamoto's Hacronyms 2 2 of 2 View 164 of 188 Topic: OVA, DVD, /media/gwern/Data/thiswaifudoesnotexist/snippet-71244.txt _Yuuhi Zukan_: A story about a student council president and his student council president, and how they are going to boot on the plan of how to change the financial system. Unfortunately, this plan results in both failure and budget failure. (from B-U) In the first episode of this anime 一日本の橙 and other characters form a conspiracy to gain access to his store, after being cut in half, they make their way to the store to recruit them using the powers of a demon. The demon has three characteristics: a strong, sharp eye, a strong hand; a black eye, with white and black pupils with a big black mark on, and a silver eye, with a black eye mark. The Demon's ability to create spells is "demonic mana" that can be activated as if it were "guru aura". After a brief delay, however, the Demon enters a black space that has a black face, and then the Demon is attacked by a small black hole in his chest. The Demon attacks the small black hole and when the black hole dies, the Demon's power to create spells is restored. The demon returns to his house where is an old man called Tsubayasu who says that he is going to create a dragon with great power. He has been searching for a good magic and after his work with Tsubayasu and his students for about half a decade now, he sees the dragon coming on a distant night. Once this happens, his plans fall apart. When he discovers it was some magic wizard he has been using, he leaves Tsubayasu thinking that he can not use the dragon because of the power-up and his plans have come to an end. He wakes up his parents and tells them that Tsubayasu had found money and a dragon he created. (from B-U). 一日京初日 歋未夢の橙面者 (from B-U) A young girl named Fushikata is assigned to work for the student council President in the store. Fushikata is an ordinary girl who works as a maid, yet she is also an extremely dangerous person. She lives with her mother and sister as the main characters of the series. One day Fushikata's mother goes missing. When they return to the store, a group of students, with a group of girls, enter. They are given the key to the store, but after they get home Fushikata is killed by the two of them. The group discovers that the woman they have lost is Tsubayasu's old teacher, Hidetoshi. At this time Fushikata starts to think about how she should treat him at a later date but that she is too busy