Well this year I wanted one of those 6' singing pumpkins on the house. I spent a bunch of time researching how to do this, and I really didn't find everything I was looking for in one place.
Now you can find it here, because I'm going to save you the pain.
First, you need to build a scaffolding for the pumpkin. I made mine out of five 10' pieces of 1/2" PVC piping, eight 1/2" T connectors, four 1/2" + connectors, and four 1/2" elbows. I also purchased (at Lowes), a roll of plastic hardware cloth (24" x 25' x 1/2" squares). I also used garden twist ties on a spool (for mounting the cloth on the frame, but zip ties would work). Then I purchased some spray paint to darken it all up.
Cut all of the PVC pipe to 23" (you will have one extra 23" piece). Build a tic-tac-toe shape with a frame around it. It should be 6' square. Unroll the hardware cloth in three strips and mount it to the frame with the ties, enough to support the weight of the light strands you will be putting onto the frame for the face.
Paint the frame a suitably dark flat color to hide it. so it doesn't reflect when the lights come on.
Mount either rope light, or mini lights to the mesh cloth to make the pumpkin face. I made triangular eyes. The lower part of the eyes are one channel. The upper two edges of the triangles for the eyes are on a second channel. This allows the pumpkin to "blink" approximately every 4-5 seconds for 300 ms.
The "mouth" has several shapes, and a channel for each. An upper lip, bottom lower lip, lower lip (just slightly above the bottom), a middle lip, an O mouth, and a smaller o mouth with some corners showing (this is the U mouth described later).
The pumpkin can be controlled with eight channels. So how to animate it? Here's what I found that was interesting. The various facial poses shown on this site were my guide --
www.garycmartin.com/mouth_shapes.html
Using the various lip shapes, I mimic the facial features shown by Gary. I started off by taking my lyrics and picking a letter like "O" and marking all of them, then using Vixen to animate the "O" mouth. Big mistake, and really slow progress....
So I read a very interesting web page:
www.keithlango.com/tutorials/old/lipSync.htm
which explains the principles of lip sync animation. Less is more. Ok, but I still am a geeky guy, and I wanted something to take text, and convert it to phonemes (the mouth shapes). I love open source, so I wanted something free.
What I found was a cool program that takes .wav files of audio, and typed in lyrics, and helps you layout the phonemes. It's called Papagayo, and it's available here:
www.lostmarble.com/papagayo/index.shtml
I took my .mp3 files and used audacity to chop them down to verses, and convert them to .wav files. Each .wav verse is loaded into Papagayo (trust me it's easier to work with smaller sections at a time!) and animated.
Before using Papagayo, I used a Vixen add-in called Tap-Tempo to determine the timing of the song I was sequencing. For example, I chose "Thriller", and I used a timing of 64, giving 16 events per second. The 16 number is used to set the FPS rate in Papagayo.
Now, loading the first verse into Papagayo, I immediately set the frames per second (FPS) to 16, then paste in the lyrics of the first verse. Use the tutorials to learn how to manipulate the sentences, then the words, then the phonemes. Use the software to get the mouth animations to look just right. After you've done that, you are ready to "Export Voice" or just use the grid display in Papagayo to map your phonemes into Vixen.
The mouth shapes in Papagayo are; AI, E, etc, FV, L, MBP, O, rest, U, WQ. In Vixen, on my pumpkin face, I mapped this to; AI = Upper Lip + Bottom Lower Lip, E = Upper Lip + Lower Lip, etc = Upper Lip + Middle Lip, FV = Middle Lip, L = Upper Lip + Bottom Lower Lip (same as AI), MBP = Upper Lip, O = O mouth, rest = nothing or Middle Lip -- depending on how it looked, U = U mouth, and WQ = U mouth.
Now, pick the first word (phoneme) and find the corresponding audio portion in vixen. In my case, Papagayo time index 18 = 0:58.752 in Vixen. At this location in Vixen, I used the AI phoneme to turn on the upper and bottom lips (that's my mapping). Let's say that the next Papagayo transition phoneme is at time index 22, and it's "etc". This means that the AI phoneme runs from time index 18 to time index 21, or 4 frames. In Vixen, the upper and bottom lower channels are on from 0:58.752 through 0:58.944.
Rinse, repeat for the entire verse, then the entire song. Errors in timing will creep in, so you may need to re-align the locations of the transitions in the sequence, but the mouth shapes should be correct for the lyrics.
What we really need is an add-in that takes the export from Papagayo, asks for the channel mappings from the mouth shapes to the appropriate Vixen channels, and the time index in Vixen for time index 1 in Papagayo. Presto, chango, a tool for quickly (relatively speaking
doing lip-sync.
I hope this helps someone else, and I welcome any comments, or even better -- a faster way of doing this.
Andy
Now you can find it here, because I'm going to save you the pain.
First, you need to build a scaffolding for the pumpkin. I made mine out of five 10' pieces of 1/2" PVC piping, eight 1/2" T connectors, four 1/2" + connectors, and four 1/2" elbows. I also purchased (at Lowes), a roll of plastic hardware cloth (24" x 25' x 1/2" squares). I also used garden twist ties on a spool (for mounting the cloth on the frame, but zip ties would work). Then I purchased some spray paint to darken it all up.
Cut all of the PVC pipe to 23" (you will have one extra 23" piece). Build a tic-tac-toe shape with a frame around it. It should be 6' square. Unroll the hardware cloth in three strips and mount it to the frame with the ties, enough to support the weight of the light strands you will be putting onto the frame for the face.
Paint the frame a suitably dark flat color to hide it. so it doesn't reflect when the lights come on.
Mount either rope light, or mini lights to the mesh cloth to make the pumpkin face. I made triangular eyes. The lower part of the eyes are one channel. The upper two edges of the triangles for the eyes are on a second channel. This allows the pumpkin to "blink" approximately every 4-5 seconds for 300 ms.
The "mouth" has several shapes, and a channel for each. An upper lip, bottom lower lip, lower lip (just slightly above the bottom), a middle lip, an O mouth, and a smaller o mouth with some corners showing (this is the U mouth described later).
The pumpkin can be controlled with eight channels. So how to animate it? Here's what I found that was interesting. The various facial poses shown on this site were my guide --
www.garycmartin.com/mouth_shapes.html
Using the various lip shapes, I mimic the facial features shown by Gary. I started off by taking my lyrics and picking a letter like "O" and marking all of them, then using Vixen to animate the "O" mouth. Big mistake, and really slow progress....
So I read a very interesting web page:
www.keithlango.com/tutorials/old/lipSync.htm
which explains the principles of lip sync animation. Less is more. Ok, but I still am a geeky guy, and I wanted something to take text, and convert it to phonemes (the mouth shapes). I love open source, so I wanted something free.
What I found was a cool program that takes .wav files of audio, and typed in lyrics, and helps you layout the phonemes. It's called Papagayo, and it's available here:
www.lostmarble.com/papagayo/index.shtml
I took my .mp3 files and used audacity to chop them down to verses, and convert them to .wav files. Each .wav verse is loaded into Papagayo (trust me it's easier to work with smaller sections at a time!) and animated.
Before using Papagayo, I used a Vixen add-in called Tap-Tempo to determine the timing of the song I was sequencing. For example, I chose "Thriller", and I used a timing of 64, giving 16 events per second. The 16 number is used to set the FPS rate in Papagayo.
Now, loading the first verse into Papagayo, I immediately set the frames per second (FPS) to 16, then paste in the lyrics of the first verse. Use the tutorials to learn how to manipulate the sentences, then the words, then the phonemes. Use the software to get the mouth animations to look just right. After you've done that, you are ready to "Export Voice" or just use the grid display in Papagayo to map your phonemes into Vixen.
The mouth shapes in Papagayo are; AI, E, etc, FV, L, MBP, O, rest, U, WQ. In Vixen, on my pumpkin face, I mapped this to; AI = Upper Lip + Bottom Lower Lip, E = Upper Lip + Lower Lip, etc = Upper Lip + Middle Lip, FV = Middle Lip, L = Upper Lip + Bottom Lower Lip (same as AI), MBP = Upper Lip, O = O mouth, rest = nothing or Middle Lip -- depending on how it looked, U = U mouth, and WQ = U mouth.
Now, pick the first word (phoneme) and find the corresponding audio portion in vixen. In my case, Papagayo time index 18 = 0:58.752 in Vixen. At this location in Vixen, I used the AI phoneme to turn on the upper and bottom lips (that's my mapping). Let's say that the next Papagayo transition phoneme is at time index 22, and it's "etc". This means that the AI phoneme runs from time index 18 to time index 21, or 4 frames. In Vixen, the upper and bottom lower channels are on from 0:58.752 through 0:58.944.
Rinse, repeat for the entire verse, then the entire song. Errors in timing will creep in, so you may need to re-align the locations of the transitions in the sequence, but the mouth shapes should be correct for the lyrics.
What we really need is an add-in that takes the export from Papagayo, asks for the channel mappings from the mouth shapes to the appropriate Vixen channels, and the time index in Vixen for time index 1 in Papagayo. Presto, chango, a tool for quickly (relatively speaking
doing lip-sync.I hope this helps someone else, and I welcome any comments, or even better -- a faster way of doing this.
Andy
