Deep Thoughts by Robert Felty

I am happy to announce the release of my fourth collapsing X plugin for Wordpress. This one is Collapsing Links, which was requested by several users of my other plugins. With the development of this plugin, I finally added the ability to have multiple instances of the widget. This took some careful study of some of the default wordpress plugins, and forced me to completely re-design how the plugin options are handled. In addition to the ability to have multiple instances of the plugin, the re-design also reduced the number of database queries, since now all the options for the plugin are stored in one row in the database. For more details, look at the Collapsing Links page.

Now that I have figured out these details, I plan on re-working the remaining plugins to allow multiple instances of the widgets as well. Expect to see these updates over the next few weeks.

There is one downside to having multiple instances of the widgets. In the current versions of Collapsing Archives, Categories, and Pages, you can use the plugin either as a widget or standalone. In future versions of the plugin, it will only work as a widget. When I originally wrote these plugins, widgets were new, and not standard for wordpress. Now they seem pretty ubiquitous, so hopefully this shouldn’t pose too much of a problem.

I hope that other people find the plugin useful. I must say that I enjoy developing wordpress plugins, since it a great way to utilize many different languages all at once — xhtml, css, javascript, php, and sql.

About a year ago I wrote a post about my frustration with the lack of a good LaTeX to html converter. Recently I found one. It is called plasTeX, so named because it is written in python. Finally a converter which works well with most any LaTeX package or macro you write, and produces sane, relatively standards-compliant html. Best of all, it comes with a built-in if \ifplastex, so you can specify that some things should be in the pdf version, while others should be in the html version. It also support some other formats like docbook, but I am not too interested in that, so I haven’t tested it all.

Though I discovered plasTeX a few months ago, I finally decided to play around with it some more this week, when I wasn’t feeling like working too much. So I decided to convert my CV from html to latex, so I can have both pretty pdf and html versions. So now I will update my CV only in latex, which is much nicer to write than html anyways.

To get both a pdf and an html version, this is what I do:

It took me awhile to figure out the syntax of the plasTeX config file. It is possible to specify these things on the command line, but I got tired of having too many things on the command line. The trick is that the plasTeX options are divided up into different sections. The sec-num-depth option, which tells plasTeX how many section levels deep to number, is in the document option section, while the split-level option is in the files section. The split-level option tells plasTeX how many different html pages to make. For my CV, I only wanted one. My plastex config file looks like:

Though plasTeX is pretty nice, there are 3 complaints I still have. One is that it uses <h1> tags for \section, which I can understand, but there is a long tradition in the web of having only one <h1> tag per page, which usually holds the title of the page. PlasTeX does have the ability to use different themes, so there might be a way to change this with a theme, but I haven’t figured that out yet. For the time being, I simply used \subsection tags instead.

My second complaint about plasTeX is that it doesn’t do any nice formatting of the html like indenting or line-wrapping. That is only a minor complaint though, since I can simply run it through htmltidy (as in the code above).

My third complaint is that plasTeX puts everything inside a <td> or <li> inside a <p>, which seems very strange to me, and creates some ugly formatting, so I used some CSS tricks to basically take away the standard effects of <p> tags inside these:

I am not completely finished yet, but you can check out the html and pdf results in: my new CV.

I started using the Fancy Categories plugin by Andrew Rader about 6 or 8 months ago. I have been slowly modifying it for some time now, and it seems appropriate to release it as a new plugin. (Note that I thought that Andrew Rader disappeared as well, but I now have found him at his new home at void*. The functionality is best described by simply looking at my categories list on this blog. The default wordpress categories list is a simple unordered list. This plugin gives it some dynamic capabilities, similar to the default on Blogger. I have decided to keep increasing the version number from what Andrew was using, so I am calling this version 0.2. Here are the main highlights:

• Changed name from Fancy Archives to Collapsing Archives
• Changed author from Andrew Rader to Robert Felty
• Added triangles which mark the collapsing and expanding features That is, clicking on the triangle collapses or expands, while clicking on a month or year links to the archives for the said category. This uses html entities (dings) instead of images, for a variety of reasons
• Lists the titles of posts, instead of just listing subcategories
• Removed the rel=’hide’ and rel=’show’ tags, because they are not xhtml 1.0 compliant. Now uses the CSS classes instead
• MOST IMPORTANTLY — it is compatible with both the pre 2.3 database which uses categories, and the 2.3+ database structure which uses the tag taxonomy

You can download it from the Wordpress plugin repository.

I recently started the FedDibblety house blog, documenting our home improvements to our new home in Indiana (which Clare’s parents built). I set up accounts for my wife Clare, her parents, Dave and Ellen, and my parents, Harold and Fran. Ellen, being a technology nut like myself, immediately jumped in writing posts about the house, and also had some feature requests. One of those features was to have the blog read in chronological order. Most blogs read in reverse chronological order, but since this one was kind of diary-esque, it seemed more appropriate to have it in normal order. I am also dating posts corresponding to the improvements made, not the actual post date. I started searching around for forum posts and such about how to do this, and I came across two relevant ones. I came across one post on wordpress.org which had a very simple solution, of simply adding one line to the index.php file in the root directory, like so:

This did exactly as one would expect. However, this also caused some problems. For those avid blog readers, now they would have to scroll way down (or go the archives) to see new posts, which doesn’t really make sense. In addition, the RSS feed was also chronologically ordered, so that RSS users would find it very difficult to see if the blog had been updated recently. While searching I also found another post on janegalt.net, in which the blogger asked the same question, and a reply by Jeff Licquia suggested a very simple solution:

To set up archives to use a different template, go into the management page, then go to Options, then Permalinks.

If you can’t, or don’t understand, the RewriteRules stuff, look at the note about not being able to use mod_rewrite. Where it says:

/index.php/archives/%year%/%monthnum%/%day%/%postname%/

Change “index.php” to something else, like “archives.php”, and put that in for the virtual site structure. Copy index.php to archives.php in your document root, then add the special flag to archives.php. You should be all set at that point.

(I’m a newbie to WordPress, but not to Web servers or Web stuff in general, so the above makes sense to me but isn’t tested. I’ll try to help if you have problems.)

This idea worked very well. I made the change in the Wordpress options page, then copied index.php to archives.php, using the $order=ASC flag, and removed that from index.php. Then I had to change the .htaccess file to direct queries to archives.php. I did not want to do this for feeds though, so I had to put in two rules, like so:

Then there was one last thing. In that blog (and in this one), I am using the fancy-archives plugin (as I write this post that site seems to be down, so I am not providing a link), which I have tweaked a bit. To get the archives in the sidebar listed in chronological order, I just had to change the SQL queries in the fancy-archives-list.php file to ORDER BY post_date ASC.

That was it!

I have recently been trying to transition from Mac to Linux, with much success for the most part, but a few hiccups as well, as is to be expected. One of the important uses of the computer for me is photo editing and sharing, especially since we got our Canon Rebel XT last year, which takes absolutely beautiful pictures. I had developed a fairly nice workflow on my Mac for photo editing and sharing, consisting of:

1. Import pictures from camera to iPhoto
2. Delete bad pictures
3. Create new albums
4. Edit some photos for lighting, cropping etc.
5. Add titles and comments to photos
6. Use caption buddy to write iPhoto comments to IPTC tags
7. Use an applescript to rename all the images in an album with a meaningful name followed by an automatically increasing number
8. Export the images from iPhoto
9. Use rsync to upload the pictures to my webserver
10. The webserver has a cron script which looks for new pictures, and then runs JAlbum to create new web albums for my pictures

It sounds like a lot of steps, but actually it was going quite quickly for me, and without many hitches. The biggest issue was with the IPTC tags. Using caption buddy is kind of a hack to get around that. It would be nice if iPhoto just wrote tags to the files by default. Oh well.

To get the same results with Linux, there are a number of changes. Obviously there is no iPhoto for Linux. The next best thing (actually better in some regards) is Picasa. Picasa is a Windows program put out by Google (they acquired it several years ago). As of about a year ago, there is now a Linux version as well. The Linux version requires WINE, which allows one to run Windows programs on other platforms (mostly Linux, but there is also a Mac version). Especially considering that Picasa is not a native Linux application, it runs remarkably fast and is very stable. It has some nice editing features, including a “I’m Feeling Lucky” button, which seems to do much better than iPhotos “enhance” button. It also includes a unique scrollbar, which changes scrolling speed, depending on how much you move it. Also, unlike iPhoto, it writes captions as IPTC captions, which are embedded in the file, which is handy, especially since that is how JAlbum knows about them. JAlbum will extract the IPTC tags, along with EXIF tags which include information about the settings of the camera for each picture. It will then make some nicely compressed and small images suitable for web-viewing. JAlbum is also highly customizable, which I like very much.

All that being said, both Picasa and JAlbum seem to have a few bugs.

Bug #1 — Null bytes

When uploading some pictures I had processed with Picasa recently, I noticed a strange character at the end of each caption. Viewing the album with Firefox on a Mac, this showed up as a question mark, making it look as if I were unsure about all of my captions. After some reading on the JAlbum forum, I saw a post claiming that this was a null byte character. That was helpful. I tried looking at the image files in my favorite text editor, vim, and saw a bunch of gobblety gook, along with some captions that I could recognize. There did seem to be some extra characters after the caption, but I couldn’t figure out which one was the null byte character. After some more searching about vim and control characters, I found out that the null byte character shows up as ^@ in vim, and that if I want to type one, I have to type Ctrl-V Ctrl-J. That meant I could remove the null byte character in vim! Well, I tried this with the image files, and that corrupted them. Bummer. Then I started looking at the html that JAlbum was generating, and indeed, the null byte character was still there. I tried doing a search and replace with vim, and that worked wonderfully. Unfortunately though, using vim to hand-edit a bunch of files was not acceptable to me, so I looked for a perl solution. After yet more searching, I discovered I could so a search and replace with perl, and that perl represents the null byte character as \0. Finally I had a solution. I simply added the following line to the shell script that runs JAlbum (after it is done processing with JAlbum)

Bug #2 incorrectly ordered metadata

This seemed to be working fine, until I noticed that JAlbum was not able to process several of my pictures. Instead of getting a picture with a caption, I only got a caption, and JAlbum would return an error that it failed to process several pictures.

JFIF APP0 must be first marker after SOI

This had actually been happening for awhile with 2 of the 1000 or so images I have. Since it was only 2, I did not worry about it. But now this was happening with many of the new images I had just uploaded. It quickly became apparent that images that I had tweaked (color, cropping, etc.) with Picasa were the ones that were not getting processed correctly. After searching a bunch more, I have come to the conclusion that it has something to do with the ordering of metadata in images, and how java processes that metadata. It seems that java expects the metadata to be in a very particular order — a different order than what Picasa outputs. That being said, it seems that many other programs can read in the files that Picasa produces. So it is not really clear to me what program is at fault. I just want a solution.

Solution #1 — Reprocess with JAlbum

After quite a bit of searching around the interweb, I found a post on the JAlbum forum that mentioned this bug. The solution: “Turn off the EXIF info”. Sure enough, this did the trick. However, that means that I was losing valuable information, and that was not acceptable for me. However, I noticed that if I processed the images with the EXIF info off, then turned it on, both my images and my captions would show up, in spite of the error message the second time around. So I decided to process all of the photos twice. This was not ideal, but it seemed like a solution.

Solution #2 — Reprocess with ImageMagick

I then started thinking some more, and I recalled a LaTeX problem I had back when I was just learning. I was learning how to import images, and I was having a problem getting the right bounding box on the .eps file I was trying to import. A more experienced TeXnician told me to use eps2eps on the image, and that that often corrected bounding boxes, when the program that produced the image in the first place had screwed it up. I found it very odd that there was a program to convert eps to eps, but that is what that program does. Sure enough, it worked. So I started thinking if I could use a similar technique here. I had also read on the JAlbum forum that someone tried simply opening the image with Photoshop and resaving it. That sounded like a good idea, but I needed a solution which could be automated. So I tried using convert from the ImageMagick suite, and that did the trick. Convert produces a new file though, which I did not want. So instead, I tried mogrify, which changes the original file. That worked! To re-iterate the conundrum again, JAlbum does not like the ordering of metadata in files generated with Picasa, but ImageMagick inputs them just fine, and outputs a format that JAlbum likes. Strange.

UPDATE: I just discovered that ImageMagick version 6.3.2-6.3.3 has a bug with IPTC captions, which simply deletes them altogether. Make sure your ImageMagick version is either newer or older than this range.

Okay then. Now to present my new photo editing and sharing workflow:

1. Import pictures from camera to Desktop/originals (Picasa automatically adds them to its database
2. Delete bad pictures
3. Create new albums
4. Edit some photos for lighting, cropping etc.
5. Add captions to photos (Picasa only has one IPTC tag available for editing)
6. Export the images from Picasa to Desktop/modified
7. for file in *.jpg; do mogrify $file; done
8. Run shell script to rename pictures to meaningful names with auto-incrementing numbers (see below script)
9. Use rsync to upload the pictures to my webserver
10. The webserver has a cron script which looks for new pictures, and then runs JAlbum to create new web albums for my pictures