YouTube creator DoodleChaos made a full-length music video using AI. The video comprises varying camera angles, styles, and themes; all made using AI. DoodleChaos had to guide the AI tool to produce something that made sense and aligned with his conceptualization of the song.
DoodleChaos is very pleased with the output and looks like a graphic designer who spent days working on it. However, it was all done by one person and an AI tool just using the song’s lyrics.
How did DoodleChaos create a music video using AI?
DALL-E 2 is among the most powerful image generation tools on the planet. MKBHD recently made a video that demonstrated the tool’s power to produce images based on natural language input. In simpler terms, you need to input text and describe the image you want it to produce.
DoodleChaos doesn’t have access to DALL-E 2 and wanted to try it out. But it isn’t available to the general public yet, so he resorted to other available tools.
DoodleChaos first stumbled upon VQGAN + CLIP, which can generate AI videos but eventually settled for Disco Diffusion V5.2 Turbo. The result is a gorgeous music video he made by commanding AI using natural language input.
Was it easy?
Disco Diffusion V5.2 Turbo, like DALL-E 2, can produce images based on your wildest imaginations. But you cannot expect it to create a video by merely inputting the lyrics and letting it come up with something. DoodleChaos inputted the lyrics and even added manual inputs to help it understand the requirements better.
DoodleChaos explained in his video description, “I added keyframes for camera motion throughout the generated world. These keyframes were manually synchronized to the beat by me. I also specified changes to the art style at different moments of the song. Since many of the lyrics are quite non-specific, even a human illustrator would have a hard time making visual representations. To make the lyrics more digestible by the AI, I sometimes modified the phrase to be more coherent, such as specifying a setting or atmosphere.“
It was a collaborative effort between DoodleChaos and the AI tool which led to the production of a full-fledged music video. However, it still relies on crisp language input to produce the images and clips you want.
Graphic designers must not fret about their jobs as AI tools could have many important pre-visualization applications. These tools will still require someone to guide them through the whole content creation process.
