Credit: RadomanDurkovic / Shutterstock
Watch a home improvement show long enough and it might seem like the renovations blend together, with wall colors, light fixtures, textures, and finishes all starting to look the same.
The term for this is design fixation, and it’s not unique to updating houses. Faced with a task, designers, illustrators, and other creative workers often draw on familiar patterns. While this can shorten and simplify the design process, it may lead to solutions that are repetitive or not quite right.
In a new paper, researchers from MIT, Harvard, and the Toyota Research Institute found that generative AI can help designers come up with new ideas by quickly creating images based on text prompts. Specifically, designers with access to a generative AI tool the researchers developed called DesignAID found the tool more useful than a traditional image search.
“For each of a number of diverse verbal descriptions, DesignAID can almost instantly generate an image — and it’s often far better developed and polished than what an artist or designer could do with a quick sketch,” said paper co-author a professor of information technology at MIT Sloan and the director of the MIT Center for Collective Intelligence.
Designers find value in AI-generated images
The early stages of design are the best times to come up with diverse, sometimes unconventional ideas. But this can be difficult. It usually takes a large team to generate lots of ideas, and it takes time to create visual representations of them, such as sketches. The process is also expensive: In the automotive industry, it can cost $1 billion to design a new car and $3 billion to redesign an existing car. Given those factors, even the most seasoned designers might opt for what’s familiar.
For the study, researchers had 87 crowdsourced designers use the DesignAID tool to create a mood board, or a set of images that helps capture the artistic direction of a project. DesignAID produces images that are based on a written description of the design problem, as well as automatically generated variations of this description. To generate diverse variations of the description, the tool uses a technique called semantic embedding, which combines a word’s contextual meaning and statistical similarities among words to come up with additional variations.
Participants found the automatic generation of images from words “more inspirational, enjoyable, and useful” than simply searching for images on a website such as Pinterest. However, the designers said they found the tool more valuable for image generation than for generating diverse ideas.
Malone and his co-authors, who received the Best Paper Award at this year’s ACM Collective Intelligence Conference, noted that this type of tool could accelerate the design process. Of course, not every image was a perfect fit. Some were silly, irrational, or physically impossible. That would be a bigger problem if the model were being used to produce outputs by itself instead of with human supervision.
“The things that make AI undesirable for certain use cases are positives in a combined human-computer creative process,” Malone said. “When you start to get some distance from the original meaning of the words, that’s what starts to trigger the most creative ideas. Then, in the worst case, designers can just throw away the dozens of bad ideas and keep only the good ones.”
Different content-generation modes for different design goals
Malone and his co-authors created two modes for DesignAID. The low-diversity Direct Input mode generated images based on input phrases, allowing for rapid iteration for refining ideas. The high-diversity New Idea mode generated “related but semantically different” ideas and then created images for the new phrases. This allowed the designers to explore the design landscape. “The new ideas and their associated images are intended to spark the creativity needed to break out of any design fixation,” the researchers write.
Two mood boards created using DesignAID generative AI system for creative ideation (left) and a Pinterest search system (right) in response to the prompt "chair for children." Credit: Courtesy of the researchers
The two modes had distinct purposes. When designers had a clear vision in mind, they preferred Direct Input. When designers didn’t have a clear vision, or if they were less experienced, they preferred New Idea, finding that mode more inspirational.
Related Articles
Discovering that study participants had differing preferences for using DesignAID didn’t come as a surprise, Malone said. After all, some designers need help coming up with ideas, while others need to fine-tune ideas they already have.
The results could further support the idea that AI complements knowledge workers but doesn’t replace them. “If you try to build a computer to come up with the best visual representation of a single phrase, that’s hard to do automatically. You need humans to judge what’s good,” Malone said. “The human can help steer the computer in various ways, and the computer can steer the human in various ways. It’s a useful synergy.”
The research paper was co-authored by Malone; Alice Cai of Harvard University; Steven Rick and Jennifer Heyman of the MIT Center for Collective Intelligence; and Yanxia Zhang, Alexandre Filipowicz, Matthew Hong, and Matt Klenk of the Toyota Research Institute.
Read the paper: “DesignAID: Using Generative AI and Semantic Diversity for Design Inspiration
