Using AI Without Exploiting Artists

Mike Taylor is the former co-founder of Ladder, and is now writing a book on Prompt Engineering for O’Reilly Media

The art world is being disrupted by generative AI, and artists aren’t happy. Generative AI models like DALL-E, Midjourney, and Stable Diffusion were trained by scraping millions of images from the internet, without permission. Artists claim this violates copyright, while the companies training these models argue this falls under fair use.

Contrary to popular belief, AI models are not just copying artists images – the model is many times smaller than its training data, so it can’t possibly contain a copy of every image it trained on. However, copyright law didn’t really foresee the existence of an advanced AI that can flawlessly replicate any artist’s style in an instant. The good news is there’s a practical way to avoid this debate all together, and move on from lazily adding in the style of greg rutkowski to your prompts.

The surprising thing is that by using AI in a more ethical way that doesn’t exploit artists, you actually get better, more creative results. DALL-E is already blocking the use of any artist in the last 100 years, Stability allows artists to opt out of the training data, and Adobe trained FireFly only on licensed content – this trend is only going to continue as legal and social pressure rises. Start practicing ethical AI use now, before you’re forced to adopt these techniques later.

1. Unbundling

The first technique for avoiding using an artist’s name in your prompt, is to unbundle the ‘memes’ from their art style and use those instead. Memes aren’t just funny viral images, but any repeating pattern in popular culture, like design elements, compositions, or artistic concepts.

I learned unbundling from Bakz T. Future, who demonstrated how you could use a language model to describe an art style, then use that description in your image model prompt. If you can reverse-engineer the various characteristics and components of an artist’s style, then you can use them to create original images that are still subconsciously familiar.

Rather than prompting  a dragon in the style of greg rutkowski, instead upload an image of his work to ChatGPT, where with a Plus subscription it can see the image and describe it.

‍

The artwork is likely inspired by traditional fantasy art but with a modern twist, as seen in the level of detail and the cinematic quality of the scene, reminiscent of concept art for video games or movies in the fantasy genre.

​

I didn’t go to art school and I had never heard of the chiaroscuro technique for emphasizing the contrast between light and shadows. Without knowing the right word to describe this meme, I’d never know what to include in my prompt to replicate it. You don’t even need to write the prompt, you can get ChatGPT to do it, and use it to generate a novel image (I used Midjourney as its better at fantasy aesthetics).

2. Remixing

Once you’ve deconstructed the individual memes of an image (or multiple images), you can start to mix and match styles until you find something you like. This is actually how all creativity works, as covered in Everything is a Remix. As Steve Jobs said, “Great artists steal”, but as the original quote he stole it from reads, they “make it into something better, or at least something different”.

After you’ve broken down the component parts of an art style, you can remove elements you don’t like, and add in elements you do like from other domains. Even just a few small tweaks can make a huge difference to the originality of your artwork, which can make a big difference in terms of differentiating your creative to suit the brand you’re working for.

When I was working on artwork for my book on memetics, I stumbled across a remix on reddit that I loved.  Someone had photoshopped Star Wars into old paintings of ancient Rome, and called it “a tale of two empires”. This was fitting because George Lucas actually borrowed a lot of the story of the rise of the empire in Star Wars from the history of the fall of Rome.

(IMAGE)

I had found a meme that I liked, and wanted to emulate this art style (minus the Star Wars IP because I didn’t want to get sued). I loved the concept of juxtaposing modern or futuristic technology with ancient ruins, and it suited one of the core themes of my book, that our prehistoric brains aren’t evolved to deal with modern society. I tracked the style down to Thomas Cole, and the Course of Empire paintings, a famous example of the Hudson River style.

‍

I hadn’t been exposed to any of these memes before, but I now had an art style I wanted to remix.  I also knew exactly what I wanted to remix it with: the cover of Neal Stephenson’s Snow Crash. I had read this sci-fi novel as research for my book on memetics, because it popularized the term “mind virus”, and it quickly became one of my favorite books of all time. The cover is iconic, showing a man clutching a samurai sword, standing in an ancient stone arch overlooking a futuristic city.

‍

‍

‍

I remixed this with the Hudson River School style, and with some iteration ended up with the prompt I used for the book. To iterate on the prompt just meant further remixing, until I found that including the memes utopian and ancient worked best, including specifying the shot should be From behind. Wide angle. I also found Midjourney v4 gave better results than DALL-E 2.

‍

‍

Through this lens, prompt engineering is just finding whatever combination of memes gets you to the right part of latent space (the multi-dimensional universe of all possible images the model can generate). Although I started my journey with a random image on Reddit, I ended somewhere more unique and creative, without ripping off that artist’s work.

To further aid my creative exploration, I took my prompt template and used permutation prompting to see how including different types of ancient soldier arriving in different futuristic cities would turn out. In Midjourney you can do this by putting different words in curly brackets (i.e. ...of an ancient {samurai, roman, persian, ...} soldier..., and it’ll generate images for each combination of soldier and city.

‍

Once you have a map of the creative space you’re operating in, you can choose to go in a direction that differentiates your work from the original source that inspired it. It’s often hard to predict what you’ll like until you see it, and it’s far simpler to communicate with the rest of the team what you want, when you have examples you can show them.

‍

3. Prototyping

I’ve had plenty of objections from clients who are nervous about using AI in their ads, due to concerns about copyright infringement. What I have never heard pushback on is using AI in the idea stage, for mocking up what an ad creative might look like. If you never use the AI generated image for the final production, the risk drops to near zero. All you have to do is hand the AI prototype over to a real human, and have them make the finished product. Someone I know said “Midjourney’s place is in the middle of the journey”, and I think that’s right.

Using AI only for prototyping has the advantage of not violating any artist’s rights, and it doesn’t put anybody out of a job. In fact, it makes everybody’s jobs easier. Clients aren’t trained artists, and get frustrated they don’t know the right words to communicate to the creative team. AI solves that, because it’s often easier to show examples of what you want, than it is to explain why it is you like it. AI lets the client, as a non-artist, to reverse-engineer their vision.

Equally if you’re the artist, this cuts down on the back and forth and second guessing that comes with working with business people. A mood board of AI generated images communicates a lot more than the typical client brief, while still leaving you space to elevate the creative by mixing in your own memes. These AI models also make your work more efficient, because you can generate many permutations of your prompt template, in order to explore the creative space faster than you could if you had to draw everything manually.

When working with a client in the childcare space, I had the creative insight that parents feel guilty for using a babysitter, but kids often really enjoy it, and are excited by what their parents are doing. I thought we could communicate this through the crayon drawings the kids made with the babysitter. This was a complex idea to communicate, so I turned to DALL-E, and in 10 minutes had generated a number of scenarios that I could share with the client.

‍

The client didn’t end up going with this idea, but this only took 10 minutes and it was immediately obvious whether it would work. I didn’t have to pull a designer off one of their projects, or commission a freelancer. We didn’t have to go through the emotional turmoil of pitching a fully baked idea to a client, only for them to reject it. Instead I could turn my half-baked idea into something tangible, and proactively add value to the client.

The Future of Creative Work

People worry designers and artists are going to lose jobs due to AI, and in some pockets of the economy that’s probably going to be true. There’s early evidence that it’s already impacting demand for freelancers, who are already seeing a 5% decrease in earnings. However, in my own work with AI, I’m finding it is giving me greater respect for artists, and making me think more deeply about what like (or don’t like) about artwork I see. I never studied art, and it was even part of my identity to be proudly uneducated on high culture. Now with AI I’m making art every day.

AI is increasing my demand for (human) creative work, because now it’s easier, I’m adding good design and artwork to everything I do. As an example, I am including a custom AI generated illustration on every one of my blog posts now, when previously I’d use a crappy stock photo. On a couple of projects I started hiring human illustrators to take my AI art and make it better. AI art is a gateway drug to hiring human artists, because it increases demand for art among people who wouldn’t normally think about using art in the first place.

This increased demand won’t always be filled by artists, that much they are right to be worried about. As they say, AI won’t take your job, but someone using AI might. I don’t know if they’ll call themselves a ‘prompt engineer’, but a new type of creative director role will emerge that crafts prompts instead of artwork. I have already worked on a few projects in this vein, like in the sci-fi short story “In Screens We Trust”. The deliverable wasn’t finished artwork, but a prompt template the author could fill in to generate a consistent set of images for the story.

‍

‍

As more people include AI in their workflows, they’ll do more work and at a higher quality than before. Eventually, as less discerning people start flooding the market with AI generated content, the brands that stand out (as always) will be those that manage to differentiate themselves from the masses. The winners from the AI boom will be those that aren’t simply copying existing art styles, but unbundling the memes of those styles and remixing and rapidly prototyping through prompting, to make something new and original.