Hand-made random seldomly blog-like thoughts
which might be worth entering Google's index
atom

Publishing Git repositories on a shared server

So you are the happy admin or user of a shared Linux server. You know that Git is very cool, and you want to install what's necessary on the shared server to host user repositories there. You're like me: you've read some Git documentation, but you just haven't found some sort of executive summary comprehensively listing how to do that. This place may help!

Fun with Google Translate

Here are a series of translations given by Google Translate:

Sadly, from German to French, Madame Dupont isn't returned :/.

Help your Windows friends and increase your karma

By default, Microsoft Outlook doesn't do proper quoting of replies - e.g. each line of reply is not prefixed with "> "; however, it should, to improve readability and allow for faster typing of multiple responses "inline" the replied message: if the replied message is left as is, the person has to manually "prefix" each part of his response with some special text, or manually set a style or coloring; however, when it's already prefixed, it's as easy as moving inside the replied message and type your text on a new line.

It's possible to configure Outlook to perform proper quoting of replied messages.. if you know where the option is buried. I bet that most of our Windows friends would be happy to use more standard and more efficient emailing.

• In the Tools menu, choose Options. On the Preferences tab, click E-mail Options.
• Under When replying to a message, choose Prefix each line of the original message.
• Click OK twice.

Eclipse 3.3.0 and broken Show Tooltip Description key binding

I personally try to use the keyboard as much as possible, opposedly to the mouse, when editing source code: I think it's much faster. When using Eclipse to edit Java, Java errors and warnings are displayed "inline" (as spelling mistakes in a word processor: a curly underline below the offending words); details about each problem can be obtained as a tooltip by hovering the mouse, but also with the Show Tooltip Description key binding. Strangely enough, when upgrading from Eclipse 3.2 to 3.3.0, the key binding stopped working. After a lot of head scratching, with no information on the net about this, I finally found that opening once the Problems window after starting Eclipse workarounds the problem. Quite strange.

Ruby equivalent to Java's synchronized {} block

In Ruby, the base synchronization tool is the Mutex class. You can use its synchronize method and pass it a block which will be guaranteed to not be interrupted by another Thread also synchronizing on this mutex. Unfortunately, the synchronize method doesn't like to be called from a Thread already owning the lock. If you look for something easier to use, as Java's synchronized {} blocks are (in my personal experience, calling/recursing into another method which also wants to synchronize on the same resource is a common use-case), the solution is to use Ruby's Monitor class. It's even possible to mix it in any other class and then synchronize on the object representing the resource which should be synchronized, as we can do with every Object in Java.

    require 'monitor'

    whateverobject.extend(MonitorMixin)
    whateverobject.synchronize { ... }

There *is* an official list of MIME types

My daily work involves dealing with MIME types quite a lot, as I maintain a web application and as I work with MMS data. I've been unhappily questing for a bright light each time I struggle with finding the official/standard MIME type for any content; is there any official/standard list for this pile of ***?; and finally the great Odie found the grail: this actually is the official list of registered MIME types, provided by the IANA. It is so kind to provide RFC links when applicable.

Zattoo on Mandriva

The Zattoo Linux installation procedure is manual. Here are the steps which worked for me:

• download their player, untar it in /tmp (it goes to /tmp/dist)
• download their xulrunner, untar it in /tmp/dist (it goes to /tmp/dist/xulrunner)
• rename all xulrunner shared libs: in /tmp/dist/xulrunner, do for i in *.so; do mv $i ${i}.0d; done
• urpmi libgtkglext-1.0_0
• have data files accessible to the player: sudo ln -s /tmp/dist/usr/share/zattoo_player /usr/share
• start it with (in one line):

    # LD_PRELOAD=/tmp/dist/usr/lib/zattoo/libavcodec.so.51:
                 /tmp/dist/usr/lib/zattoo/libavformat.so.50:
                 /tmp/dist/usr/lib/zattoo/libavutil.so.49:
                 /tmp/dist/usr/lib/zattoo/libfaad.so.0.0.0
      LD_LIBRARY_PATH=/tmp/dist/xulrunner /tmp/dist/usr/bin/zattoo_player

identify sucks for reading EXIF orientation of a JPEG file

My web-album software booh is great, but somehow kinda slow. Most of the slowness comes from XML parsing (since it's done with REXML, a pure Ruby XML parser, which is not too fast) and image resizing (since it's done with convert of ImageMagick, which is not too fast either). Though, there's also the fact that I use identify from ImageMagick to read the EXIF orientation tag of photos; and actually, on my Pentium-4 2.8 GHz, it takes exactly one second to extract the orientation of a 3 MB digital photo! that sucks! I've made a comparison with commandline's exif tool on a directory with 428 photos, and identify absolutely cannot stand:

    # time identify -format "%[EXIF:orientation]" *.jpg >/dev/null
    07:34.42 elapsed
    # time exif --tag Orientation *.jpg >/dev/null
    00:05.73 elapsed

tcpdump reminder

I always forget the needed parameter for a useful probe with tcpdump. So here it is:

    # tcpdump -s65535 -w/tmp/tcpdump.out host blabla.bla and port 22

Then of course, launching wireshark and clicking on "Follow TCP Stream" are a must.

kill -QUIT's brother

It's possible to show interesting GC related information from a running java program with the jmap utility.

    [gc@meuh ~] jmap -heap 12377
    Attaching to process ID 12377, please wait...
    Debugger attached successfully.
    Client compiler detected.
    JVM version is 1.5.0_09-b03

    using thread-local object allocation.
    Mark Sweep Compact GC

    Heap Configuration:
    [...]

Notice that in this information, it's possible to view if you're close to reaching an out of memory because of a too small permanent generation.

Verifying a S/MIME signed email

Openssl's smime utility will do.

    [gc@meuh ~] openssl smime -verify -in mailfile -signer /tmp/signer.pem >/dev/null
    Verification successful
    [gc@meuh ~] openssl x509 -noout -text -in /tmp/signer.pem
    [...]

What threads are running inside my Java application?

kill -QUIT triggers an stdout-based full thread dump. Including a backtrace of all threads, with precise locations where they are currently locked if applicable!

Ruby presentation at Linux Days

This morning, I gave a Ruby presentation (in french) in the LinuxDays event, Geneva. Was kinda cool, room was pretty nice, in a building close to the Palais des Nations of Geneva. About 20 to 30 attendees. The french presentation, made with OpenOffice.org, is available.

Linux + Laptop + Radeon + Beamer = 640x480

Well, at least on the "ATI Radeon 9250 and earlier" of the HP pavilion ze4300 (PCI 1002/4337), when plugging a beamer, Xorg (X Window System Version 1.3.0, Release date 19 April 2007, X protocol version 11, revision 9, release 1.3) says:

    (WW) RADEON(0): Failed to detect secondary monitor DDC, default HSync and VRefresh used

And then nearly all modes are removed because of default low frequencies, and both laptop and beamer start up at 640x480 (laptop screen is 1400x1050). This is not lethal for OpenOffice.org/Impress which will scale slideshows, but for the rest of your applications, you will probably end up pretty much screwed (however, this is ideal for Frozen-Bubble, which is probably the most important).

The solution I found on a forum is to force timings. I am not sure how large you can set, and if there is a possibility to break your beamer, so don't use too large values, but the results of my experiments, with a 4 years old beamer, are (within the Device section of the X configuration file) that the following setting allow 1024x768 output.

    Option "CRT2HSync" "28-50"

When using 60 instead of 50, the device chooses 1280x1024, and when using 70, 1400x1050 (but that's unreadable on the beamer output, of course).

PostgreSQL: VACUUM and VACUUM FULL

Due to PostgreSQL approach to MVCC, the databases need to be regularly VACUUMed and ANALYZEd, else the disk usage and performance will decrease to a crawl. Unfortunately, if databases have not been VACUUMed regularly enough in the past, a new run of VACUUM will not completely recover performance, due to a large number of unused space within actual data. I went to the extend of submitting a documentation patch to make it more clear in PostgreSQL documentation; fortunately, after the sterile troll that occurred in the thread, the patch was accepted upstream by a more sensible guy than the futile trollers who answered in the ML.

Notice that unfortunately, there is no easy way to know if there is a lot to gain from VACUUM FULL prior to trying it - and as it fully locks tables (and the DB connection which is performing the request, even if used for a totally different statement), you badly need to know for large tables. You may:

• on the production database, you can use contrib/pgstattuple. If the free_percent reported for interesting tables is large, and free_space is large compared to 8K, then VACUUM FULL should be very helpful.


• another way is to dump your production database, restore it somewhere, issue VACUUM VERBOSE on a given table on both databases (in production, and on the restore) and compare the reported number of pages needed. The difference is the potential benefit of running VACUUM FULL.

  production_db=# VACUUM VERBOSE table;
      [...]
  INFO:  "table": found 408 removable, 64994 nonremovable row versions in 4395 pages
  
  restored_db=# VACUUM VERBOSE table;
      [...]
  INFO:  "table": found 0 removable, 64977 nonremovable row versions in 628 pages
  

  In the above example, 6/7 of the data is unused. Scanning the table will be 7 times faster (or more - lower data amount means higher cache hit rate) after a VACUUM FULL!

Generating a DVD to play on a DVD device

It's somehow not so simple to generate a DVD to play on a regular DVD device (the one below your TV). There are details to make right, or the device will refuse to play it, jerk sound or things like that. It turned out that I didn't manage to properly generate a DVD with mencoder or transcode, but I could do it with ffmpeg. Here it is:

    # ffmpeg -i source.avi -target dvd -bf 2 target.mpeg

Don't forget to deinterlace interlaced material such as DV with -deinterlace. You can also add -threads 2 if you have a hyperthreading or dual core CPU. Personally, I dislike the little black borders left and right of the image when coming from a DV, so I also use -aspect 4:3.

Of course, I only showed the .mpeg-generation phase. Now you need to author the DVD. Under Linux, most tools suck bigtime. I use qdvdauthor, which looks promising, but which seems to still be unable to easily generate a menu with a video background, and to properly save project parameters between runs.

Java out-of-memory error with plenty of memory left

Happened on me that my web application running inside tomcat received the out-of-memory exception, although it was obvious it didn't run out of memory (java heap space). When looking closer at how the memory is handled, it turns out the GC of the JVM has several locations where objects are put, and classes and methods are actually loaded inside a specific one, the permanent generation. In my case, it turned out that I had a lot of classes loaded, and after some time the default size for the permanent generation (which is 64 MB, and is not altered by setting the heap space with -Xmx) was not enough to hold them all. Why the JVM (1.4.2) is not able to properly hint in the out-of-memory exception in that case is a pity, but life's sometimes full of unexpected things :). The solution was to enlarge the permanent generation with -XX:MaxPermSize.

What are the currently running SQL statements?

In an application using PostgreSQL, it might not be obvious to list the currently running SQL statements, together with the date they started. It is particularly useful if you see in the processes that you have running statements (ps auxw | grep postgres: | grep -v idle), and your machine is loaded, but you don't know where the load is coming from. Set:

    stats_command_string = on

in postgresql.conf, then you may see the running statements with:

    db=# SELECT current_query, query_start FROM pg_stat_activity WHERE current_query <> '<IDLE>';

strncat sucks

Contrary to popular belief, using strncpy and strncat in C programs is not the panacea. While strncpy would probably be still acceptable, strncat is truly braindead. As I could not find a GPL implementation of OpenBSD's strlcat, I wrote my own (for inclusion in the Frozen-Bubble server). So, this is GPLv2:

    size_t strconcat(char *dst, const char *src, size_t size)
    {
        char *ptr = dst + strlen(dst);
        while (ptr - dst < size - 1) {
            *ptr = *src;
            ptr++;
            src++;
            if (!*src)
                break;
        }
        *ptr = '\0';
        return ptr - dst;
    }

Notice it doesn't check src and dst are not NULL.

UTF-8 Byte Order Mark?

Some sucking Windows software add a BOM in front of UTF-8 files, even if it makes no sense. The BOM in UTF-8 is 0xEFBBBF. As a consequence, you might end up needing to remove such byte sequence in front of UTF-8 text documents you receive from the wild - especially when you feed it to an XML parser, because anything in front of the XML declaration will trigger a parse error.

Guillaume Cottenceau
atom