Loki in the Multiverse

Cartoons can be claymation, hand-drawn, pixelated, or cut out from paper, but the hero in them is usually the same: brave, charming, and a little absurd.

Project Concept — to explore what my cat, Loki, would look like if he were placed into various animated universes. To achieve this, I fine-tuned a neural network using his photographs and attempted to render him in eight different animation styles: from anime to a black-and-white cartoon. Core Project Question: Does a cat remain recognizable when the entire surrounding world—including its texture, color, and even the rules of drawing—is transformed?

To generate the images, I selected the Stable Diffusion XL neural network, which I fine-tuned using DreamBooth with LoRA. The entire model is not retrained; instead, a small adapter is added to it to learn the specific cat. For fine-tuning, I used 28 of my own photos of the cat and cropped them to a 1:1 aspect ratio.

The dataset includes close-ups of cat faces, the cat in various poses, and under different lighting conditions. The BLIP neural network generated a caption for each photo, and the phrase «a photo of zwx cat» was prepended to each one. The rare combination of the zwx model is specifically tied to my cat.

First, I checked whether the model remembered the cat. For this, I used the simplest possible prompts, similar to the captions in the dataset: «cat lying on a bed, ” „cat yawning, ” „cat portrait, ” „cat on the windowsill.“

The difference is immediately apparent. A standard model generates random cats—gray, striped, black and white. The trained model, having seen all the pictures, draws my cat: a ginger tabby with light eyes, a pink nose, and big ears. It even transferred details from the house, such as the brick wall and the colorful pet bed, from the photos.

The cat is recognizable even at 500 steps. Furthermore, the 500-step version generates livelier and more diverse frames, whereas the 1000-step version begins to memorize the dataset: the brick wall and the bed from my photos reappear, and the fur looks «scorched, ” as if the image was dried out by a filter.

The retraining became immediately evident when I tried sending the cat into cartoon universes using short prompts like «zwx cat in claymation style» or «zwx cat in anime style.» The model generated eight nearly identical close-up photos of the cat. The style only permeated the background: claymation blocks, pixel noise, and red brushstrokes. The model remembered the photographs so well that during full-scale training, it simply ignored the style request.

For the final series, detailed prompts were utilized, specifying animation technique, material, lighting, and narrative, alongside a negative prompt of «photograph» to prevent the model from reverting to photorealism.

Each universe was described through animation technique and era, without naming studios or cartoons. For each one, I generated three options and selected the best.

In other universes, style prevailed over the cat. In the clay world, the model sculpted gray-blue figures: the neural network’s conception of a «clay cat» overshadowed my own features. In a black-and-white cartoon from the 1930s, the cat transformed into a typical dancing character of that era—the cat’s main characteristic, its orange color, is simply impossible in black-and-white graphics. In the pixelated game, the cat shrank down to a few pixels, making it unrecognizable.