Final Cut - Updated Strategy

Well, using a program to process my sound turned out to be a lost cause at this stage in the game. So I decided to shorten up the clip to a reasonable length to where I can manually do the sound. I took a base sound, reversed it and adjusted it's playback speed and used that for the majority of the background sound. Then I added an intro and exit of actual car sounds (about 5 seconds on each end). Once the first car starts to go in reverse I created a wha wha affect that signals we just entered into something strange. I played around at spots where the video is doing something interesting and then added another sound for where we actually have a cut. It's fairly simplistic but I think it's effective for the piece.

Rough Cut Sketch #3

This is my first attempt to sync sound but unfortunately it quickly loses the sync due to a rounding problem in my program. See, it takes the next button press time and subtracts it from the first button press time (in milliseconds) and divides by 1000 and then multiples by 30 to get the number of frames. If it's inbetween frames it rounds up.. So we get extra frames of video.

The only solution at this point is keep track of where I add frames and adjust the sound lengths accordingly. It's not very practical at this point either. When I have more time I'll devise another way around this issue.

Rough Cut Sketch #2 - Random Beating of the drums

I just went to town on the drum kit and here is the result

It has a few interesting parts but for the most part is garbage. I think if I'm going to get anything I consider decent I'm going to have to layer my results.

Basically take what I like out of each clip and put it aside and then process another one. Sadly this almost defeats the purpose of everything I've done so far.

Rough Cut Sketch #1 - Happy Accidents

When I went to render a clip I was playing with I got a significant number of warning messages saying could not retrieve a frame yet it completed rendering the file anyways. The result is this very interesting corrupt video file. I'm assuming that despite not correctly retrieving a frame it instead pulled random data where it thought that frame was stored and used it creating the strange glitches.

This occurred when I was using footage from my 2nd digital camera that for whatever reason says the frame rate is ~15.000001 instead of 15 on the head. I assume it's requesting a frame that exists but due to working with reals/integers it couldn't exactly find it.

Unfortunately when I tried this with other clips the result wasn't really as stunning and either crashed the program, or only had minor video glitches. I'm guessing because the original clip contained pretty much still images is why it's so pronounced. When I have more time I'll investigate this further as I really like the result and it would be great to be able to reproduce it on demand.
Here is the correctly rendered original

Production Strategy

For my drift 3 I intend walk from my home (~60th street) due west to 76th street and then head due south on 76th in a straight line. 76th/Cleveland starts as a residential area for a few blocks and then turns into commercial for a few blocks and then repeats. For my last two drifts I was pretty much exclusively in residential areas. For this drift I'm looking to find more "action" in terms of the footage I collect. My goal is to collect footage (exclusively video clips) that can be used in loops using the process I describe below. As far as sound goes I am going to attempt to use sync sound which I will also modify in the process below.


I would like to use Cinelerra's XML based EDL file format to create a improvisation based method of visual rhythm by way of Rock Band's Drum Kit. Which is a USB device designed for the Xbox360 gaming system. When you plug it in it detects a USB device but doesn't know what to do with it... Well in Windows it does but I'm using Linux so I have to find an alternative.

First, I needed to find software to access an Xbox360 controller on my computer.
I was able to locate a driver written to work with the Xbox360 controller but it didn't support Rock Band's devices. Lucky for me the device is pretty much an Xbox360 controller just not detected as one. I modified the driver to detect the device and treat it as a normal Xbox360 controller and it works!

Xbox360 Controller Source code;
xpad.h
xpad.c

My updated to xpad.c to work with Rock Band's drums

Then I wrote a program to simply log the input from the device into a text file
Button: X Time: ######

Where X is the button number and ##### is the time in milliseconds it was pressed at

Source code: joystick.cpp

Now that I had a log of input I needed to figure out what I wanted to do with it.
I wrote another program that took the log file and created a valid EDL data I could use with Cinelerra. The drum pad itself is 4 pads and 1 kick pedal. I decided to use the kick pedal as a “switch” so I could have 8 distinct actions for my input.

1 Go back 1 frame and create a freeze frame of that image until another button is detected
2: Go forward 1 frame and create a freeze frame of that image until another button is detected
3: Play video forward at normal speed of 30FPS
4: Play video in reverse at normal speed of 30FPS
5: Toggle flipping the video horizontally
6: Toggle flipping the video vertically
7: Jump to the last frame in the video clip (also doing a +1frame jumps to the first frame since it loops)
8: Jump to the middle frame of the clip

Source code: convert.cpp

The EDL format is pretty complex and I haven't found documentation on what does what so I had to basically run trail and error tests to figure out what did what. I was able to locate how to create simple layouts using the following format in the section of the file. Each follows the following format

<edit startsource="X" channel="0" length="Y"><file src="FILENAME"></file> </edit>
FILENAME is the video/image sequence to use
X is the frame to start at
Y is the length of frames to display

I was able to determine how to use filters/effects but it's not very easy to incorporate into my program as it's a complete different section of the EDL file and would require a lot more work to incorporate. Which is why for my flip methods I simply use a 3 additional prerendered clips to perform that task.

I'm attempting to create sync sound based on when I press a button but due to the design of my application it is just not feasible at this time to accurately sync sound.

Here is my "Base Case" Cinelerra project file to give you an idea of what one looks like.

Drift Questions

1. How long can I walk without running into another person? Would it be possible to do an entire drift without any human interaction?
2. What kinds of changes would I have experienced if I was to repeat the previous two drift paths at night?
3. What if I based my algorithmic walk on directions I received from random people? For example, Every time I cross paths with somebody on the street stop and ask them where to go next/what to record/take pictures of. Instead of avoiding people constantly stop anyone I could and ask them questions pertaining to the drift.
4. What if I only recorded sound/images based on suggestions of others I met during the drift?
5. Can I stumble across another woodpecker?
6. My drifts took place mostly in residential areas. What I focused more on areas that were more populated/busy/active?
7. Working with stills is very time consuming. Especially when you want to change the frame rate of a sequence throughout the sequence. Can I come out with an easier way to adjust the frame frame of a sequence/create rhythm?
8. I've recently discovered Cinelerra (my FOSS based editing software) works with an EDL file format that is XML based. Meaning I can directly edit the project file without using the GUI. Can I come up with a system to algorithmically edit my drift 3 video? Sound as well perhaps? Something similar to Animusic (see youtube) or the performance Glenn and Seth gave in the Film115 class...
9. Can I limit my sounds/video to only animals? Can I find non domestic animals besides birds?
10. All my images/videos in drift 2 were taken from the ground level. Can I locate spots where I can take shots from a much higher angle?

Drift Assement #4 - Nostalgia

During drift 1 I walked by my old Elementary School. Now, I drive by it from time to time but I haven't stepped foot on the school grounds in at least 15 years. It was a very odd feeling when I recorded the sounds on the playground as general layout was the same but all the equipment was new. A sense of deja vu but at the same time I knew I have never really seen these things before. After leaving the school the remainder of my drift I felt very distant as I couldn't help but be flooded with random memories of times at that school.