Closer #002

Welcome to Closer,

a weekly newsletter where I take one of my photos, ask AI to describe it, then see what kind of image it can generate from its own description—an ongoing experiment in the capabilities of AI, and what that means for humans. This is edition #002.

I want to start with a shower thought I had this week. If we're continuously asking AI to make art, to see how 'smart' it currently is or to gauge how great the danger of it taking over our creative professions is, aren't we actually training AI to be an artist as well? I mean, we want it to do our dishes, clean our homes, do our chores, right? But if we have it practicing art instead, won't it end up becoming more like us: artists forced to do chores, longing for a creative life and unwilling to do the unpleasant work?

Anyway, for this second edition, we're looking at an image of my two cats. Because you can't have cat images soon enough in a project like this. We start with the input: a prompt and my image. Followed by AI's description of that image, a prompt to generate the image, and the final output. Prompts are in the grey callout boxes, the AI description is cursive.

LLM models used

GPT-4-turbo for image interpretation. DALL·E 3 for image generation.

The input

The output

The image is a cozy, warmly lit photograph featuring two orange tabby cats relaxing on a couch. The focus is on the cat in the background, which is lying on its side with eyes closed, appearing completely content and at peace. Its pink nose and soft fur are clearly visible, adding to the overall feeling of calm and comfort. The cat’s ears are slightly tilted back in a relaxed position, and the fur pattern includes classic orange tabby stripes with white around the nose and mouth area.

In the foreground, slightly out of focus, is the second orange tabby cat, positioned with its back to the camera, partially obscuring the scene but enhancing the sense of intimacy. The cats are nestled closely together, likely lying on a dark couch or chair covered with a maroon blanket. A corner of a darker leather or faux-leather cushion and a loosely draped red scarf or cloth is also visible in the upper left, adding layers of texture to the composition.

The overall atmosphere of the image is one of warmth, comfort, and quiet companionship.

Impressions

What's striking about this description is, with the original image as input in mind, it's completely accurate. It has taken some liberties with the elements in the image, the maroon blanket it speaks of is the corner of a pillow in real life, and words like 'appearing' and 'likely' give away that this is not a human description for we will use much more absolute terms. Yes, the cat 'appears' to be content but we also know that's most likely true and will almost always state it as such. In a way, you could argue the AI description is more truthful than the one a typical human would give.

The second thing you'll notice is, despite the accurate description, the resulting image based on it is still completely different. The general elements are present but that's about it. It seems there are some slight nuances required to deliver an image where the subjects are actually in the same positions and the lighting reflects that of the original image. What we got was another image that, funnily enough, just as perfectly depicts the description.

Those are my main takeaways this week. Is there something I missed that you would like to highlight? Do I need to make adjustments to my prompting? Feel free to send a reply. This is just as much a journey for me as it is for you.

See you next week.

Mitch