I decided to train Stable Diffusion to generate images in my own artistic style by compiling a database of 30 drawings that share a common technique and mood.
The main objective was to train the model to replicate irregular, organic lines, slightly disproportionate geometry, and basic drawing patterns.
The neural network was tasked with generating new characters that adhered to my personal style. In the final series of images, the characters varied significantly in color palette, mood, and the geometry of their forms. This outcome is the result of my experiments with prompts and with LoRa strength. The characters in the series are imperfect, which aligns with the initial objective. However, the line sometimes becomes noticeably smoother and more geometrically precise than in my own work. In some images, the line becomes nearly chaotic and begins to resemble my own pieces more closely, but unfortunately, the lines are not distributed correctly, which results in unnatural shapes for the character’s mouth, eyes, and other body parts. Simultaneously, this is an advantage—one of the goals of the training was precisely to create unique and disproportionate forms.
The primary objectives were met: the generative model grasped the core essence of working with my style. It aimed to make the lines more dynamic and chaotic, avoid perfect facial proportions, and create unique contours. However, the result is far from ideal: the neural network understood my request verbally (it understood the description of every image in my archive), but it failed to grasp this visually, so most of the images are distant from my aesthetic.
For the training process, each image was assigned a brief description, along with a general description of recurring patterns. After the model was trained, we tested various LoRa capacities (nearly all final images were generated at 0.5–0.7) and fine-tuned the prompts for the most successful generations. In total, about 25 images were created, but 11 made it into the final series.
