Ben Blumenrose has noticed that the floor for AI-generated website design is rising fast. As co-founder and managing partner at the venture capital firm Designer Fund, and a practicing designer for two decades, Blumenrose has been tracking the rapid improvement of AI-generated visuals. He can still spot the difference between a site designed by a professional human and one produced by a model (some tells: the proliferation of “pills,” floating dashboards, and gradients, gradients, gradients), but the AI results are “five times better than what the same kind of tools did a year ago.” He anticipates the trend will continue. “Probably in six months to a year it’s going to be very hard even for me to tell,” Blumenrose says.
One of Designer Fund’s investments is Framer, a no-code website design platform. [Image: Designer Fund]
The AI industry has been working hard to improve creative output from models through crisp evaluation metrics, rubrics, and automated tests and tools. A few years ago, the challenge was to reduce the volume of slop in the world by ensuring that AI-generated human hands had the correct number of fingers, or AI-generated essays didn’t mix metaphors. But now that “correct” has mostly become table stakes, the path to creative work that convinces, or even impresses, an expert like Blumenrose is less clear-cut.
Taste, or at least an approximation of it, has become one of the next frontiers in model development. A host of startups are promising that they can encode taste into AI. Meanwhile, the major frontier labs are thinking about how to edge their models into subjective domains like writing and design. Anthropic’s Claude Design, which launched in April, has already encoded a particular aesthetic that’s replicating across the internet.
But as every human knows, even within a narrow profession, opinions on what’s considered “good” can vary based on each person’s alchemical mix of social, cultural, and work experiences. Developing taste in any creative field is both a technical challenge and a personal project: You need to learn enough information about what works from experts and from history, but then you need to develop an understanding of what you like and why. The same is true when teaching a model taste.
The first step in that process, information gathering and evaluating, is similar to how tech companies have operated for decades, and how frontier labs have been developing model intelligence in the last few years. The critical next step—deciding what feels good and unique, and then making a choice based on discernment—is more foreign to the processes and systems for building technology.
Still, technologists across the industry are attempting to tackle that qualitative quandary. At large and small scales, they’re teaching models what “good” looks like for specific creative fields, aiming to hone AI taste to a fine-enough point. But as it turns out, those methods rely on the taste of a whole lot of human professionals.
Expert data in, expert data out
The first hurdle toward developing AI taste is getting the right data sets. Most general-purpose model providers like OpenAI and Anthropic have built their model knowledge bases by scraping the internet and gathering publicly available information (or proprietary information, for which some AI companies have been sued and, in some cases, held accountable).
For a long time, the big labs have worked with “post-training,” or the process of refining an existing model to perform well at certain tasks. With the goal of developing a generalized model intelligence, that training has been directed toward a best average. In the visual domain, this has resulted in reliable, if not terribly creative, output. But domain-specific expert-level taste requires domain-specific expert-level data. And now, a crop of established companies and new startups are looking to move past a generalist knowledge base to solve this problem in the creative space.
Take Figma, which uses foundation models in-house for productivity and production tools, but notes that they often have shortcomings. “Off-the-shelf models often lack the ability to judge design quality in a reliable way, and the amount of reasoning and resulting latency doesn’t always correlate with the complexity of the task,” says Figma’s AI Research Lead, Sumithra Bhakthavatsalam.
In other words, frontier models can work too hard and for too long to produce less-than-tasteful results. That’s why the company post-trains for faster outputs and less overthinking during iterative design work. It also tunes the model for overall quality, variety, and a deeper understanding of Figma and Figma’s customers’ design systems. To power that post-training, Bhakthavatsalam’s team collects curated “best-in-class” examples of “great” and “good” from Figma’s designers.
At Krea, co-founder Diego Rodriguez knows that general inputs won’t yield results that his designer, filmmaker, and architect customers can use for their professional work. Krea offers a suite of tools for creative professionals including image and video generators, and a model Krea users can help train, which means the output quality must be high. For Krea’s visual generators, Rodriguez’s team has gathered millions of references of film angles and styles, printing techniques and materials, architectural materials and elements for post-training to ensure that the models they build with have the visual vocabulary for professional creative work.
A recent partnership with architectural firm Henning Larsen has helped Krea improve their architectural-related data, and as a result has refined Krea’s offerings to the architectural industry. It’s a mutually beneficial arrangement: Henning Larsen gives Krea expert feedback on things like the quality and accuracy of their generated visuals, the usefulness of the platform structure, and Krea’s offerings for professional architects; and Krea provides the firm with domain-specific tooling, like an interface tailored to their workflow and an image generator trained on their field.
Rodriguez says now Henning Larsen can spin up far more accurate and expressive client proposals in less time.This is Krea’s first partnership like this with a specific company, but Rodriguez plans for more. “I want to partner with the best across [creative] industries—the best in film, the best in architecture,” says Rodriguez. “They can tell me things I hadn’t even thought about.”
AI taste is made of human decisions
After teaching AI models what domain-specific information even looks like, the challenge becomes guiding them toward discernment. Edwin Chen, founder and CEO of Surge, which supports frontier AI labs with post-training, thinks many in the industry are wrong in the belief that if they make a model smart enough through rubrics and popular benchmarks, that model will be able to reason its way to tasteful outputs.
“I think people underestimate that this isn’t purely a technical problem,” says Chen. “It’s actually a humanistic and values-based problem, too. You’re not just teaching [the models] a lot of more facts, you’re teaching them what they care about.”
According to Nick Heiner, Surge’s head of RL environments, the key post-training method for taste is reinforcement learning with human feedback (RLHF), a process where a panel of experienced humans provide feedback on AI’s outputs that are then used to push the model in a better (or more tasteful) direction.
To understand how it works, imagine 10 poets reading two side-by-side examples of AI-generated poetry on a screen. They rate and comment on each one based on their professional opinion, submit their answers, and move on to the next set of examples. (Surge does in fact work with poets and other published writers.) Now picture hundreds of poets looking at hundreds of poems. Now, thousands—or even hundreds of thousands. Now imagine there is also a collection of designers looking at side-by-side layouts for a homepage. And architects clicking on the material that would be the best match for an east-facing exterior wall. And filmmakers selecting the right lens for a key scene. All of those people are reinforcing what “good” looks like so the model can fine-tune its output. The trick to replicating human taste is getting lots of humans to tell computers what they like and why.
Surge taps into human expertise to help models develop a better sense of taste. [Image: Surge]
But even within that process, there are many decisions that Surge and other companies running post-training can make that further mold which definition—or definitions—of taste gets synthesized into a model. “Some of them involve how you’re arranging the humans,” says Heiner. “Maybe you have one set of experts doing the original work, and then you have another set of experts reviewing it… Maybe you have some other mechanism where you’re reviewing the reviewers, and then if someone’s not doing good reviews, they get demoted back down.”
You can adjust the way you write the instructions for the humans, how you set the priorities for their feedback, how you direct the experts’ attention, whose choices are given more or less weight in the results. You can change the criteria for the experts you select in the first place. You can change the number of tasks that go into a data set, or their type—maybe the layouts are in a carousel instead of side-by-side on a screen. All of these choices affect the training outcome. All of these choices are subjective.
“At the end of the day,” says Heiner, “what it comes down to is the taste of the humans—not just the taste of the experts, but also the taste of the data production lead, and the taste of the people leading your post-training experts.”
And once Surge has completed their work, there’s a whole new team of humans on the client side making decisions about what “good” means. The process turns many individual human aesthetic choices into data sets that teach individual models to make “good” choices—or maybe more accurately, human ones.
Surge is not the only company directly collaborating with the frontier AI labs and other model developers to hammer on this problem. Taste Labs recently came out of stealth mode with $18.5 million in funding to train models on taste, building infrastructure and tools to collect expert annotations and evaluations—starting with the field of design (the company did not respond to a request for an interview).
This year, the creative hiring platform Contra launched Contra Labs, “a independent human data & creative evaluation lab” that connects its network of more than 1.7 million creative professionals with labs and developers looking for expert feedback on model output. Contra co-founder and CEO Ben Huffman says that designers, writers, video editors, and other creatives at the high-end of their fields can make from around $50 to $250 an hour working on RLHF and other types of post-training. Huffman says he believes this kind of creative expert-level evaluation work is going to become a huge job category.
Post-training toward taste is happening at smaller scales too. Pickford is a startup creating films that respond in real time to audience sentiment through AI. Bernie Su, an Emmyaward–winning writer and director, writes the original screenplays and character profiles for the films; AI redirects the story based on audience response within prescribed guardrails. To make sure that the AI has the right level of taste to match the audience’s expectation, Pickford does extensive post-training on a matrix of foundation models, and has a team that reviews video of each screening afterwards to give feedback for the model. (Their film Whispers has been selected as a finalist for the Outstanding Innovation in Emerging Media Emmy.) At this stage in the industry, humans are guiding AI taste from start to finish.
A human’s work is never done
It takes a lot of people to build an automated system today, and many experts believe that will be the case for a while—if not forever. In fact, Contra’s Huffman says he doesn’t believe it’s possible to give a model taste. What Contra Labs offers is feedback from working creatives about which output they think is better from their professional experience and personal taste.
“LLMs can improve their outputs based on that preference data,” says Huffman, “but they’re never going to be able to create something truly tasteful and competitive that’s net new and resonant, because they just don’t have a pulse.” According to Huffman, for any creative work with multiple definitions of good, humans need to continue to be in the loop.
Krea’s Rodriguez compares his company’s offerings to the sword, not the samurai. He wants to create tools, images, and platforms at a high-enough fidelity for creatives to do their jobs better—not to encode a closed definition of taste. Krea has even trained their own foundation model, and recently made it available as open weights, or free to use with a custom license, for creatives to tweak to their own personal and professional taste.
And Figma’s Bhakthavatsalam says the company doesn’t see AI as a stand-in for taste: “Designers can use AI to expand creative possibilities, but the discernment and empathy behind each prompt and action remain uniquely human.”
On the other hand, Surge’s Chen is bullish on the industry’s ability to encode taste into AI. “It is inevitable that we will get models with taste very soon,” Chen says. “We teach the models all these aspects of taste, all these things that we thought were very intrinsically human, like creativity and serendipity. Then every few months, we see the models get better at all of them. One by one, the things that we were sure were uniquely human just turn out to be patterns the models can learn.”
Whether this proves to be true across the board, it’s true that models are getting better at complex work, or at least humans think so. A recent beloved best-selling novel Daggermouth was allegedly written with AI; nearly one in five young Americans uses chatbots for mental health advice, up from one in eight just late last year.
But even if Chen’s prediction is right, Heiner says Surge plans to keep experts in the process. “I think you will continue to need humans to inject what [taste] looks like, because if you think that AI is built to serve humans, we do need to speak up and say what the thing is that we’re looking for it to be,” he says.
Models may be able to learn and reproduce existing patterns well enough for most professional purposes, but taste isn’t static—and tastemakers will likely keep pushing past any encoded definition.
Blumenrose predicts that we’ll start to see more obsessive, elaborate websites that are personal expressions of craft and creativity. At Designer Fund, they often toss around the phrase “AI would never,” to describe the kind of choice that only a human designer could make—out-there, counterintuitive, or even “wrong.”
“Because we’re humans, I think we’ll start doing things that the computers can’t do or won’t do,” he says. And then maybe someone—or thousands of someones—will teach that to an AI, too.