First the conversion from .mod to .dv

Then I open the .dv file in Kino and edit it, and export to raw dv, making sure to uncheck the options for splitting into separate files and for resampling audio

Finally, I convert the dv to flv to put on my blog:

You can see the result on the fedibblety blog

Once again I found myself needing to use imagemagick to do something, and was overwhelmed by the many options. After much fiddling around, I found out some options that worked for me.

In this case, I wanted to extract iptc captions from images, so that I could then insert the caption in a webpage with php. I use Picasa to edit photos and add captions. Picasa adds in the captions in the iptc information, which is the right place to add them. To extract the caption from the image above, do the following

The result should be output to standard out:

8BIM#1028=”IPTC”
2#120#Caption=”Rob and Spencer with zebras”

If you want to output to a file, simply do something like:

Now it easy to extract the caption using perl, python, php or whatever you like. For perl, we could simply pipe it:

And in case you are interested, I needed to do this for the postie wordpress plugin which allows you to post to your blog via e-mail. In version 1.1.5 iptc captions will be read and displayed (if they are in the image you attach).

I upgraded to Picasa 3 a couple weeks ago. I didn’t upgrade right away, because it took awhile for the linux version to come out. I can happily say that Picasa 3 works great under linux from my experience. It was easy to install (they had an rpm available), and it runs very nicely. It seems like it even runs a bit faster than Picasa 2.6. But best of all, it has some really cool new features. One such feature is the ability to make collages. There was some rudimentary collage-making features in Picasa 2, but Picasa 3 adds lots of functionality. I decided to try it out recently with some pictures I took at Thanksgiving. I would say that my only complaint so far is that captions only seem to show up when you format the pictures as “polaroids”. It seems to be that if you include a white border, the caption should show up there. This was particularly frustrating because I had “show captions” box checked, but no captions were showing up.

I’ll also admit that part of the reason I wanted to write this post was because I just learned about mediarss, which is a way to include images, sound clips, and video clips in rss feeds. A quick google search for a wordpress plugin resulted in the media rss wordpress plugin, which seems to be working. Now the only other question is whether or not it correctly shows up in my friendfeed.

Today I got an e-mail from a student asking why the following did not seem to be working:

The student reported the following output: “Uabcd”. (The expected output is “ABCD”, which is what I get on Linux)

I tried it, and it worked fine for me. Then I thought: maybe this is a Mac/Linux problem. Sure enough, when I look at the man page for my Fedora 7 box, it tells me that my version of sed is GNU 4.1.5, from June 2006. Mac Leopard (10.5) is using BSD sed from July 2004. Leopard came out in 2007, as did Fedora 7. Why is it 2 years behind? Why is it still using python 2.4? Why doesn’t it come with useful utilities like dos2unix? Mac has done a great job of making a nice GUI, with some pretty cool applications like iLife. It is falling behind when it comes to the command line utilities though.

You know what they say: “knowing is half the battle.”

I have been working on a new project at work to come up with some lists for testing speech recognition. We decided to use the TIMIT database, which contains recordings of many different sentences from many different speakers all around America. I first wrote a perl script to generate some basic stats on the sentences, like how many words were in each sentence, and what the word frequency for those words is. Then I wrote a perl script to randomly select some of the sentences, and create several different lists of sentences. Finally, I wrote an R script which took the original .wav files, and mixed in signal-dependent noise in one channel, so that we can vary the signal to noise ratio during presentation of the stimuli by adjusting the balance on our sound system.

Along the way, I ran into a couple problems with the original sound files. It turns out that 446 of the 6300 sound files were clipped, and highly distorted. I noticed this on my own in listening to a few of the files I had generated with R. I could have gone through all 6300 files manually, and removed the distorted ones, but that would have taken a long time. Instead, I used the program sox, which is a low-level, powerful audio processing program. I first used the find command to find all .wav files in the directory I was interested in (including sub-directories), then I passed each file to sox, and told sox not to play the output , but instead just give me some stats (-n stat). After some testing with a few clipped, and non-clipped files, I realized that for clipped files, the output from sox ended with a line that said either “Try: blah blah”, or “Can’t determine type”. I then later discovered that there might still be clipped files, and these would have a maximum amplitude of 1 or minimum or -1. So I knew that any clipped file would produce this output. So I passed the results from sox to grep (notice I had to redirect STDERR to STDOUT 2>&1), and then if the output contained a line starting with “Try:” or “Can’t”, then I moved that file the $file.clipped.

After doing this, I simply amended my perl script which randomly generated lists to make sure that the wav file actually existed. Clipped files now ended in .clipped, instead of .wav.

There was an additional problem I had previously discovered with these sound files. They seemed to have some non-standard headers in them, which meant that the R script I was using to add noise to them couldn’t read the files. However, passing the files through sox made the files readable by R. (Windows Media Player on a Windows box couldn’t read the files either.) I only wanted to process the files I was actually going to add noise to, so I used another handy little bash one-liner. This one cuts a column of the file which contains all the sentences I am going to use, and then for each filename, processes the file through sox, and outputs it to the destination directory of my choosing.

Note that I have expanded the code into one more line, but pretty much they are one-liners. I think technically a one-liner doesn’t involve successive commands, which the first example does, but the first command is just an echo, to make sure I know what it is doing.

This is the problem: if you want to sort a file that is tab-delimited (and some of the filelds contain spaces), then you must explicitly tell sort to use TABS as the field separator, otherwise it will use any whitespace character. For functions such as cut and paste, this can be done like so:

where -f specifies the field number and -d specifies the field seperator.
The sort command uses the -t flag instead. So one would think that this would work:

where -k specifies the field number and -t specifies the field separator
Unfortunately this does not work, because sort won’t accept ‘\t’, since it treats it as a multi-byte character. The solution is to place a $ before it, like so:

The dollar sign tells bash to use ANSI-C quoting
From: http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_03.html

3.3.5. ANSI-C quoting

Words in the form “$’STRING’” are treated in a special way. The word expands to a string, with backslash-escaped characters replaced as specified by the ANSI-C standard. Backslash escape sequences can be found in the Bash documentation.

So now I have the answer for myself the next time the problem arises. I hope someone else benefits as well.

Now I am very happy about my working fonts!

And now the code explained:
-density 600×600 says treat the pdf as 600×600 dpi resolution
-quality 90 says use the highest compression level for png (9) and no filtering (0)
-resize 800×560 gives the dimensions in pixels of the resulting png file

Happy ImageMagicking!

On *nix systems, R uses the readline library to give a command history. So I can simply type the up arrow, and find the last command I used. I just discovered that it can also search the command history in the same way as in my BASH shell. Since I like vi, I use vi key bindings in bash. I was pleasantly surprised when R automatically used these as well. I am not sure if it gets them from my .bashrc file or my .inputrc file, since I have vi keymaps set in both.
.bashrc:

.inputrc

So when I am using R, I can search the history for “foo” by typing:
<esc> /foo

and then use ‘n’ and ‘N’ to go to the next or previous match.

What fun!!