RunwayML Experiment - Case Study with Ceramics
PART I
THOUGHTS ABOUT CERAMICS, MACHINE LEARNING, AND THE FUTURE OF NON-ESSENTIAL WORK.
For the last few weeks, I have been isolated at my apartment with my husband. We moved from Santiago, Chile to New York 3 years ago and we have never had clear plans about how long we are staying here. For that reason and financial security, I decided that my pottery business had to be based on a made-to-order model and to carry a small stock. I also decided to rent a space in a collaborative studio for ceramic artists and to don’t have my own studio. Today I realize that everything that seemed to suit my lifestyle has turned out to be sort of an obstacle. Nevertheless, I feel that pottery develops your resilience and I also believe that limitations can be good for creation, for re-thinking your work and for pushing your creativity to new territories.
I have been thinking about how the crisis we are facing highlights just how dependent we are on supply chains, materials, and workspaces. Along with these thoughts, I have been trying to focus on what I have available at this moment: a computer, my phone, internet and a lot of archives with pictures of my work. Can these assets be my new tools to work with?
Maybe the way we have been creating, producing, making, and shipping products and art will turn out to be antiquated. Today we are being forced to live more virtual lives and to move towards a more digital existence. Perhaps we are on the verge of a new creative revolution, and now it is the time to discover and explore the capabilities of digital tools. If we go back to the way things were, at least we will know that there is an alternative for staying creative.
Digital tools such as machine learning, 3D modeling, and printing, augmented reality, and virtual reality have had exciting progress in the last few years, especially for designers, artists, and creators. Until today, I have never paid attention to any of them, mainly because I am passionate about handmade crafts and because I don't have a background in computer science. At the same time, my partner has been developing software to make Machine Learning easy and accessible for the last year. In a coronavirus world, I've turned out to be curious about its functionality.
What is Machine Learning or ML (from a ceramicist’s point of view)?
ML is when a computer uses pattern recognition and algorithms to perform and create a specific task without using explicit instructions. It has the power to understand, and even predict and define, future trends or aesthetics. Among other features that I mention below, ML can create things similar to what it was trained on and find unexpected combinations that can be useful for you.
Using Machine Learning.
RunwayML is a software that is accessible for those of us who don’t know how to code and want to explore Machine Learning capabilities. The application has pre-trained machine learning models that can help you perform tasks such as colorizing and restoring old images, generating paragraphs of text from a prompt, stylizing images in the style of famous paintings and many more. The models are interesting to experiment with, but they are too limited if you are looking for customization.
Colorize and Restore Old Images Model. Source: www.runwayml.com > ML Lab
Generate paragraphs of text from prompt “NASA discovered a new type of clay on Mars”. Source: www.runwayml.com ML Labs
Style Transfer. A Mug + Matisse
Browsing RunwayML Models
Test 1.
I trained a custom model using images of my mugs and cups. The first step to train a custom model is to gather the dataset (a collection of similar images) and then pre-process them. In this case, the pre-process step comprised standardizing the background, removing undesired elements and cropping them into a square format. It's important to consider that for better training results, your dataset should be no less than 500 images.
Mugs & Cups dataset for my custom model and Test 1.
After I had the dataset ready, I uploaded the files to the software. I selected the generative model StyleGAN, set up the training steps to 3,000, and waited for 3 hours until the training process had finished. When it was ready, I could explore the model through the latent space interface. The latent space consists of 512 dimensions in which each one of them offers infinite possibilities of synthetic images.
Setting up the training experiment.
The results were not what I expected since the synthetic images were very similar to my work. This detail made me realize the limitations and biases of the hand-building technique I use and that the mugs & cups dataset wasn’t diverse enough, as you may see from the pictures below, all the mugs and cups have a similar cylinder-like appearance. So for the second test, I wanted to explore if ML could help me envision a different version of my work, one in which the objects look more organic. I wondered what might happen if I mixed the mugs & cups dataset with another dataset.
Navigating the latent space of Test 1. The result of this training process is a generative model. As it names says, a generative model is capable of generating images or outputs based on the dataset that was fed in the first step. In Test 1 my dataset was the mugs.
Test 2.
I browsed the Internet looking for “organic” shaped objects that could relate to what I was looking for, and I found a great collection of Open Access* vases from the MET Museum. The catalog had over 28,000 images, and I selected 400. This step was very similar to when you look for inspiration and create a mood board.
Vases dataset for my custom model and Test 2.
*In the digital space, there is still a copyrighted aspect unclear to me. Can a mood board be subject to copyright infringement? What is considered public domain? I think this is a whole topic that needs to be considered in the future.
Dataset preview of Mugs & Cups + Vases for Test 2.
I went through the same steps as in Test 1 and waited for the model to be ready. This time the outputs created in the latent space dimensions were unusual, unexpected, and the model could recognize the handles and play with them. The color palette and the overall aesthetic were also interesting.
Latent walk video.
I hand-picked 124 generative “synthetic ceramics” that I organized into collections. None of the objects above are real. They have all been generated via ML.
During the process of this experiment with ML, I felt I was a curator. I could select the dataset by manually picking the images I found interesting and while browsing the outputs generated by the model I was, again, the one who identified which of them were interesting and meaningful to me. After spending a few hours navigating the latent space of the custom model, I hand-picked 124 generative images and organized them into collections you can check below.
Thoughts and Next Step.
My vision for ML is not one where technology comes and takes over everything. Instead, is one where ML can become your creative partner and assistant. A new kind of collaborator that can surprise you, provide unseen details of your work, tune with your aesthetic quests, and create new opportunities for creative collaborations. Now that I have explored some capabilities of this technology, I am wondering how I can translate the outcomes into final products or experiences. The paths I am thinking of are:
Human-built by modeling with clay.
Machine-built by printing clay with 3D.
Virtual built by mapping and creating a 3D virtual version of the objects. Can you imagine a world in which you only own “essential” products, and everything that is considered “non-essential” is available in digital format? Is this going to be the real sustainable way of creating in a post-COVID19 world facing the climate change threat?
Handles.
When I found these synthetic creations I was fascinated. The combinations and playfulness of the handles makes me wonder if ML can be creative.
Color Palettes.
Mugs created by ML. Color palette is soft and neutral.
Mugs created by ML. Color palette is soft and neutral.
Shapes.
My Final Selection.
UPDATE: Here is a basic demo of how to use RunwayML.
PART II
FROM IDEA TO “REALITY”
It is interesting to see some of the questions I had last year.
Can you imagine a world in which you only own “essential” products, and everything that is considered “non-essential” is available in digital format?
At one moment I really thought it was the end of tangible objects and struggled to imagine how I could create digital versions of them and sell them as collectibles. I read some articles that talked about how the fashion industry was starting to create digital clothes but couldn’t really be passionate about the idea.
The scene is quite different in 2021 specifically because of the evolution and mainstream adoption of NFTs. But I don’t like how NFTs are used to make art. Maybe the use case for real state and legal documents makes sense. But for art, I still can’t find real value and meaning.
From ML to hand-built, to 3D modeling, to 3D Printing.
The option of hand-building one of the vases was the most attractive but I had to consider that I wouldn’t have a ceramic kiln available so I used a box of Air Dry and Self-Hardening Clay. I also realized that my tools were at the studio so I had to use a kitchen knife and cut a piece of cloth from a bedsheet. After building the vase I let it dry for a day and it was ready for display. It is important to note that this kind of material is not waterproof so it is not functional and is intended for decoration.
Later I took pictures of the piece from different angles and then used DaVinci to create a render.
Once I had the render ready I googled for a 3D printing service that could make a piece with a soft and smooth surface and came across Kwambio. I reached out to them at the beginning of the pandemic so the process was slow and it took around 1 year, and 73 emails, to finally receive the piece. Although it was slow, the service they offer is promising.
To start working they sent a guideline with instructions and explaining what is possible to 3D print, the thickness minimums and maximum requirements, specifications for firing and glazing, and production time. There was also a .pdf catalog of materials and prices. I had to adjust one of the handles because the thickness minimum was 5mm. My pieces are usually 2mm thick so it is a big difference in feeling and overall look. Anyways, I still consider the 3D printing option as a great alternative for remote creation and production.
Unfortunately, the result didn't look as I expected and it arrived broken. But it was worth the try. What do you think?
