A neural network that learns to cut out shapes from paper: pencil sketches, kraft paper, newspaper scraps, and three colors—coral, turquoise, and black.
Project Idea
I am making paper collages: first, I sketch a figure with colored pencils on a gray sheet, then I cut out fragments on top—hair from craft paper, clothing from old newspapers, and accents from coral and turquoise paper—and add white glossy highlights, like lacquer. The figure remains unfinished; color only exists where the paper is placed.
The project’s idea is to teach Stable Diffusion XL to recognize this signature style and apply it to any subject—a cat, a cyclist, a boat. The goal is not to copy my work, but to test which techniques the neural network considers part of the «style»: the palette, the texture, the unfilled space, or the logic of the cut-out forms.
Source Images
Dataset — 5 pieces, resulting in 23 square images at 1024×1024: a full composition on a gray background and close-up fragments (face, hair, windows, cups, fox tail).
The portrait was created by me manually; the remaining images were generated in the Recraft style.
How the model was trained
Base model — SDXL 1.0, fine-tuned using DreamBooth + LoRA in Google Colab on a Tesla T4 GPU. Square frames were cropped from the originals; images with empty backgrounds were padded with gray fields to maintain a 1:1 composition with ample negative space.
resolution=768 · batch=1 · grad_accum=4 · lr=1e-4 · snr_gamma=5.0 8-bit Adam · fp16 · gradient checkpointing max_train_steps=1500 · checkpoints: 500 / 1000 / 1500 · seed=0 instance_prompt = «paper collage in AYCOLLAGE style»
Image Series
The style prefix is the same in all requests; only the subject matter changes. The seed is fixed (100 + frame number), so the differences between the images are the result of the plot, not randomness.
'a black cat sitting on a windowsill, kraft cardboard and newspaper fur, turquoise window''a cyclist riding through an old town, coral red coat, newspaper buildings'
A woman reading a book in an armchair, paper-cut hair, wearing glasses
A bouquet of tulips in a glass jar, newspaper background shapes
'a sailing boat on the sea, turquoise paper waves, newspaper sails''a man holding a coral red umbrella in the rain, black paper silhouette'
Results
Limitations: The dataset is small (5 works), so the model tends to repeat compositions featuring «object on the left and empty background.» Including more of your own work with different subjects and perspectives would improve diversity.
What was achieved: A gray background and ample negative space. Color palette: coral, turquoise, kraft, black. The newspaper texture reads like a stamp. Glossy white highlights and thin pencil lines around the forms.
What did not work: Meaningless «newspaper» text; gradients that are sometimes too smooth instead of flat paper layers.
How the shots differ: In some places, the model leans toward portraiture (hair-shavings), while in others—toward «architectural» pieces (straight, cut-out forms).
Connection to the concept: I managed to transform my personal approach into a universal language applicable to new subjects.
Description of generative model application
- Trained Model. Stable Diffusion XL 1.0 + LoRA, fine-tuned by me on the described dataset, for generating the final series.2. Dataset Expansion. Images of «city, ” „still life, ” „dancer, ” and „fox“ were generated in Recraft using prompts in the style of my original work to gather more subjects for training.3. Claude (Anthropic). claude.ai, the Claude Sonnet 5.5 model: assistance with drafting prompts for point 2, dataset preparation (cropping into squares, captioning), adaptation of the study notebook, and drafting the structure and text for this long-form article. The final texts and conclusions were edited by me.4. BLIP, Salesforce/blip-image-captioning-base, was left as an optional step in the notebook and was not used in the final training.
