VO: We will go for a ride to the beach

video

Making some quick animations for an older track

Author

Matt Crump

Published

February 20, 2024

Code

from diffusers import DiffusionPipeline
from transformers import set_seed
from PIL import Image
import torch
import random
import ssl
import os
ssl._create_default_https_context = ssl._create_unverified_context

#locate library
#model_id = "./stable-diffusion-v1-5"
model_id = "dreamshaper-xl-turbo"

pipeline = DiffusionPipeline.from_pretrained(
    pretrained_model_name_or_path = "../../../../bigFiles/huggingface/dreamshaper-xl-turbo/"
)

pipeline = pipeline.to("mps")

# Recommended if your computer has < 64 GB of RAM
pipeline.enable_attention_slicing("max")

prompt = "synthesizer beach. beach view. Sunset. looking out onto the beach. 80s cartoon. linocut. synthesizer music."

for s in range(30):
  for n in [5,10]:
    seed = s+21
    num_steps = n+1
    set_seed(seed)
    
    image = pipeline(prompt,height = 1920,width = 1080,num_images_per_prompt = 1,num_inference_steps=num_steps)
    
    image_name = "images/synth_{}_{}.jpeg"
    
    image_save = image.images[0].save(image_name.format(seed,num_steps))

synthesizer beach. beach view. Sunset. looking out onto the beach. 80s cartoon. linocut. synthesizer music. - dreamshaper-xl-turbo

I didn’t realize that all of the tracks from my albums are selectable on instagram as music background for instagram reels.

I’ve occasionally posted some reels, mostly with strange video fx stuff that I was messing about with. I mostly never post reels.

But, I do like making strange little video clips. And, I now I could do that for each of the tracks on the older albums. That could be fun. Will it though? I need to find out.

I chose the song “we will go for a ride to the beach” from VO. This was made in when I was deeply cloistered in my basement, dreaming of going outside once again someday in the future, to the beach.

Here’s the song if you’re curious. Let’s try a youtube embed this time

I’m not sure how to change the video for these youtube tracks that are generated by distrokid. I do have control over this youtube channel, but these tracks are all auto-populated and I can’t seem to change them. Anyhow.

Basic animation idea

I used the python code at the top of this post to generate some images from Dreamshaper-xl-turbo. I have a local copy of dreamshaper and I sometimes use it to make pictures with synthesizers in it.

The prompt for these was: “synthesizer beach. beach view. Sunset. looking out onto the beach. 80s cartoon. linocut. synthesizer music”. The file name for each has info about the random seed used. I only made 8 of them, but it would go on forever making variations of things that look like this.

These are just OK, good enough for a quick animation project. Not sure what I’m going to do one/all of them yet, but it will involve something, probably in Adobe Fresco where I can cut them up, and do a quick animation.

Some very light animation was had to get the waves moving a bit. Then some colorization with after effects.

Tried to get the length to correspond to a musical snippet from this song, but that wasn’t super easy. I nailed it in after effects, but I’m not totally sure how instagram reels works for getting the size of a loop, so it got a bit clunky at the end.

View this post on Instagram

A post shared by Matt Crump (@mattcrumplab)