الْحَمْدُ لِلَّه, Fedora 16 has been released!  You can read the full release notes or just go straight to the torrents for downloading.  See the release announcement after the fold…

The Fedora Project is pleased to announce the release of Fedora 16 (“Verne”). For detailed information, see the release notes:

http://docs.fedoraproject.org/en-US/Fedora/16/html/Release_Notes/

Fedora is a leading-edge, free and open source operating system that continues to deliver innovative features to many users, with a new release about every six months. Fedora 16 brings exciting new features for desktop users, for system administrators, and for developers. Highlights of these features can be found below. If the mere mention of Fedora 16 is all you need, download options can be found at:

http://fedoraproject.org/get-fedora

Dedication

During the preparation of Fedora 16, the computing world lost one of its great contributors: Dennis Ritchie. Ritchie co-invented Unix and the C language. He also co-authored The C Programming Language, a book that taught many programmers just at the time personal computing was exploding. Without Ritchie computing would be nothing like it is today.

A humble man, not well-known outside his field, Dennis will always be remembered by those of us who practice the craft. Thank you, Dennis.

What’s New in Fedora 16?

For desktop users

A journey to the center of the desktop:

• GNOME 3.2, the latest update to the most popular desktop environment, brings new features, including:
  • System Settings gains an “Online Accounts” panel, which provides a central point for managing online accounts like Google, Facebook, etc.
  • A new contact management application is integrated with Empathy, Evolution and the new “Online Accounts” settings panel
  • A new document management application provides a simpler alternative to traditional file management for both local and “in-cloud” documents.
• KDE 4.7, the most recent version of this feature-rich desktop environment, brings substantial updates, including:
  • DigiKam 2.0 adds face detection and recognition, geotagging and more.
  • An updated Plasma Workspaces window manager (KWin) makes KDE better suited for mobile devices, and also adds improvements for desktop users.
  • A new shutdown dialog allows users with multiple operating systems to select the OS to boot next.

For developers

Twenty thousand lines in C:

• Perl 5.14, a new version of Perl, brings many enhancements, including Unicode 6.0 support.
• GCC Python Plugins extend GCC with Python 2 and 3, without dealing with the C internals of GCC.
• D2: Fedora 16 is the first Linux distribution to include the newest version of D, a systems programming language combining the power and high performance of C and C++ with the programmer productivity of modern languages such as Ruby and Python.

For system administrators

Around the world in eighty columns:

• GRUB2: This leap forward allows better configuration options and better support for non-x86 architectures.
• New system account ID numbering: Fedora 16 starts user IDs at 1000, providing more room for system accounts and making it easier for administrators not have services run as root. This improves interoperability with other Linux distributions that start user IDs at 1000.
• Chrony: Chrony provides network time protocol (NTP) client and server pieces which are more tolerant of unstable clocks and Internet connections which are not always on.
• ext4 driver mounts ext2 and ext3: Fedora 16 uses the ext4 driver to mount ext2 and ext3 file systems, reducing the size of the kernel code.
• Improved virtualization tools:
  • Fedora 16 provides improvements to virtual networking, making large deployments easier.
  • Virt-manager guest inspection is a unique-to-Fedora tool allowing read-only access to guest file systems, applications, and Windows registry.
  • Fedora 16 provides locking of virtual disks to prevent a disk being used by multiple virtual machines simultaneously.

This is only a taste of what is included in Fedora 16. A more detailed list can be found at:

https://fedoraproject.org/wiki/Releases/16/FeatureList

Moodle 2.0 brings an incredibly wide range of new features and capabilities that further extend what is possible.  I personally have worked on some of the new features in a minor capacity – both in submitting bug reports as well as helping to implement some milestones in line with what we needed at work.

Moodle is used on countless sites to provide education and share knowledge in every corner of the world.  So, congratulations to the Moodle team and to the world for this great step forward!

A few days ago, Matt Lee of the Free Software Foundation had asked in the #theora IRC channel for help in making a DVD of the Patent Absurdity documentary.  Several people volunteered to help, and I was one of them.  I discovered later that this was related to the “Who should see Patent Absurdity?” campaign of the End Software Patents project (an initiative started by Brad Feld, I just discovered!).

Incidentally, we all ran into different kinds of problems, doing what should have been a rather simple task.  The sources of the problems ranged from simple bugs to rather weird video formats.  In this post I will discussed the various issues that I personally encountered and how, alhamdulillaah,  was able to overcome them.

The most straightforward way to make the video was to take an already decent-quality clip, the 1080p-resolution Ogg Theora clip from the Patent Absurdity website, and just run that through ffmpeg with the ‘-target ntsc-dvd’ option to get a DVD-compatible MPEG-2 video stream.  While this did actually produce a compliant video stream that I could burn to a DVD, the resulting video actually had some audio synchronization issues as well as some other problems related to ffmpeg’s handling of Theora’s drop frames (segments of video that are basically empty frames because the data does not change at all).  So, the resulting video was unacceptable.

My next attempt was to actually go from the original source material that Matt had provided some folks over at Xiph.  What’s interesting is that the format was a very odd, 1280×1080 resolution, 4:2:2 chroma-sampled DV-encoded video with a display aspect ratio of 16×9 – that was 25GB in size..  The only application that actually supports this format is ffmpeg, which means, also, it’s more widely supported than one might think.  It’s still an oddball format, though.  So, I ran it through ffmpeg with the same target settings as above.  However, ffmpeg’s ntsc-dvd target does not change the chroma sampling format of incoming video, so it produced MPEG-2 video that had 4:2:2 chroma.  I’m not sure if that’s even valid for MPEG-2, but it’s surely not valid for a DVD.  So, I manually specified the proper chroma sampling using ‘-pix_fmt yuv420p’ (this means planar 4:2:0 chroma sampling).

Sure enough, this produced a DVD-compliant MPEG-2 video stream, but with one problem.  The aspect ratio was not preserved.  This resulted in video that was squashed horizontally.  And, try as I might, nothing I could do with ffmpeg with the MPEG-2 video stream from this point on could correct the aspect ratio, and using a utility, such as mpgtx, to fix it, would make it incompatible with DeVeDe, the application I’m using to master the DVD Video itself.  So, I had to do an intermediate step.

I deduced that the problem was rooted in too many transforms being required at the same time.  Part of the problem may very well have been my own inexperience with ffmpeg as well, as perhaps someone may be willing to clarify via comments.  Regardless, I decided to tackle the problem by doing the steps separately – rescale the video & resample chroma as one step, and then do the MPEG-2 encoding via DeVeDe, which will know how best to make DVD video it finds agreeable.  So, starting with that mutant DV video source clip, I extracted the audio out into a FLAC file (the source was 48k/16b/s PCM, so very high quality).  Then I worked on the video – I scaled it down to 720×480 with a 4:2:0 chroma sampling but with an aspect ratio of 16×9, and I encoded it with the lossless FFV1 codec developed by the ffmpeg developers themselves.  This gave me a 7GB AVI file – not small, but far more manageable than the original 25GB source file.  I downloaded that locally, and then I attempted to remux the video & audio back into the same file.  And this is where the next issue occurred.

I first noticed the problem on my server, where I attempted this technique first – that is, remuxing the FFV1 AVI with the FLAC audio into another AVI file, this time with combined video & audio.  I noticed my server started to lag tremendously, and even my rsync download of the video to my PC at home had halted (at more than 90%, no less).  I managed to kill the ffmpeg process, and upon inspection of the load of my server, it had apparently jumped up to 30 or so during that time.  I was confused, but I didn’t piece it together yet.

After the FFV1 AVI file had finished downloading to my PC (I had to restart the rsync, but thankfully, I had [and usually always] used the –partial-dir parameter to save partial downloads for just such an occasion), I attempted the same remuxing operating on my PC.  Less than a minute later, I noticed that my PC had frozen.  I knew it was still working because the hard drive was thrashing, so I attempted to SSH in to my PC from my laptop, which I happened to have handily next to me.  After wait for about 1/2 an hour for the connection to establish and actually be able to get the system to respond, I discovered that ffmpeg’s memory usage had jumped up to 1.3 GB.  And, as my PC has only 2GB of RAM and I had numerous bloaty apps open already (Thunderbird, Firefox, Konversation, Pidgen, Gwibber, and, of course, Gnome itself), this mean the system had run out of memory and was thrashing the swap file.  I managed to killall -9 ffmpeg, and that brought the system back to the land of the living.  I inquired on #ffmpeg why this might have happened, and the response I got was that ffmpeg doesn’t like FLAC in AVI.  So I tried other container formats – Matroska & NUT, notable.  They both exhibited the same problem, but I was able to kill ffmpeg in a similar fashion much more swiftly.  I decided to just suck-it-up and converted the FLAC to WAV and ffmpeg muxed that without any difficult whatsoever.  My PC didn’t even break a sweat.

As a side note, following some additional advice from #ffmpeg, I used mkvmerge to mux the FFV1 AVI file with the FLAC audio file, and it worked, but it lost the aspect ratio.  I tried to manually specify the aspect ratio, and it gave very odd numbers that were, perhaps approximations of 16×9, but I wasn’t satisfied, so I stuck with FFV1 & raw PCM audio in AVI.

Finally, I can jump into DeVeDe and attempt to make the DVD.  As I am quite pedantic about quality (in case you haven’t gathered that by now), I chose the best quality options for encoding the DVD video that DeVeDe gave me – highest rates for audio & video and two-pass video encoding.  Matt had specifically said he didn’t want any menu for the DVD – he just wanted it to play, so I removed all menu options.  The whole mastering & encoding process took abut an hour and a half, according to what DeVeDe reported at the end, but I just slept through that (I had stayed-up all night, partially because of this, but not solely so).  I burnt the ISO that it generated onto a DVD-RW, tested it on different systems (Fedora & Windows), including a consumer DVD player, and, alhamdulillaah, it worked beautifully everywhere I played it.

So, there you have it – the trek to make a DVD about the problems caused by software patents completely with free software.  It was a fun experience, and I’m glad that I was able to help with getting this important message out.  In shaaʾ Allaah this will have a positive impact on patent reform and strengthen the arguments against the enforcement and even existence of software patents.

Alḥamdulillāh, I’ve decided that, after upgrading WordPress, I should do a little housekeeping.  As usual, I find a list best presents the ideas I wish to express:

• I upgraded to WordPress 2.8.2 (mentioned here).
• I fixed some posts that had some typos (I usually am good about this, but these had slipped by).
• I deleted some links that really weren’t relevant to me anymore (i.e., Ruby on Rails) and I added some new ones.
• I changed to a new theme (comments welcome!).

I’d like to elaborate a bit more on the last point above, since it merits some discussion and the action is rooted in something I care about deeply.

Previously, for quite some time, I’ve been using the theme “Vidiyal“.  I’ve actually stuck with it for quite some time because of it’s simplicity, elegance, & beauty (in my eyes, at least).  However, a post by the author of the theme really made up my mind for me.  To put it shortly, despite the obligation to release derivative works of WordPress, such as a theme, under the GPL (as that is what WordPress is licensed under), he insists he does not have to.  I’m not so concerned with the legal issue, to be honest, but the insistence not to release the code as free software is what unsettled me.

To be honest, I didn’t really check before using the theme, and I guess I didn’t really care before this.  But now that the issue has been brought to my attention, I have to stick with what I believe is right.  So, I’ve stopped using all non-free (as in freedom) themes, and will only use themes that I know are free software.  I feel this is the better way to go.

I do hope that Sadish and others see that free software, both practically and, more importantly, fundamentally, are important concepts for protecting our freedoms as human beings.  Software freedom is something we should stand up for, and not tolerate being abused or disregarded.

So, with that, I close my rant and resume pretending to be productive.

Alhamdulillaah, I’ve just managed to produce my first HDR (high dynamic range) image the results of which I’m pleased with.  Once again, I have CHDK to thank for enabling this feature easily.

Note:  I begin this post with a mildly detailed explanation of the human visual system, HDR & LDR images, and the process to create HDR & tone-mapped images.  The pictures themselves are near the bottom of this post.

First, a little background about HDR.  The way our bodies capture and process imagery is, to put it lightly, amazing.  Starting from when light first hits the cornea through to when we perceive the visual object (not to mention how our brain’s store such visual images, which is far from just keeping a “bunch of bits”), the Words of Allāh resound with truth regarding His Creation:

And it is He who spread the earth and placed therein firmly set mountains and rivers; and from all of the fruits He made therein two mates; He causes the night to cover the day. Indeed in that are signs for a people who give thought. (Alqurʾān 13:3)

Thus, in the way that they are created, our eyes can capture an amazing amount of detail and range in their own way.  The range of regular (so-called “LDR” – low dynamic range) images is quite limited in comparison.  And this is, actually, the vast majority of image-related technologies, be they cameras, display devices, or even image formats such as JPEG.

HDR images, however, are those which do or have the ability to capture image details, specifically color & brightness information, at a much higher level than normal technologies.  Since normal technologies, such as our digital cameras and screens, can not capture nor display HDR images, we have to take a different route to get the information.  One technique is exposure bracketing where one takes a picture of the same scene at different exposure levels, one after another.  The reason for doing this is that, while the normal dynamic range of a camera may be unable to capture the bright, medium, and dark details of an image all at once, it does have the ability to capture these details separately with different settings.

Alhamdulillaah, there is also software that exists that allows you to take these separate LDR images and combine them into one now-HDR image that contains the details from all the separate exposures.  Doing so is not trivial, and it may take one multiple attempts to get something that looks good.  More often than not, the result is eery, because of a technique called tone-mapping.  Tone-mapping is a technique whereby the colors and range of an HDR image are brought down into a simulation of what our eyes would perceive in the scene.  In reality, that’s all but impossible, but it yields some interesting, beautiful, and sometimes striking results.

The specific software I use is the unpronounceable Qtpfsgui (I just say it one letter at a time).  Qtpfsgui is a front end for pfstools, which is an outstanding image toolkit for these kinds of images.  It takes care of aligning, merging, and finally tone-mapping the images one step at a time.  I short-circuited the first two steps (because Qtpfsgui can be fussy sometimes)  by using the command-line tool align_image_stack to align the different exposures (since I moved a bit between exposures) as well as creating the HDR image from the three separate images.  Then, I used Qtpfsgui just for the tone-mapping stage.

The first row of pictures are the original images, taken at -4, 0, & +4 EV settings.  Looking closely, you’ll notice that each range brings out different details: the darkest shows details in the clouds, the middle shows details in the sea, and the brighest shows details in the sand.  These are the separate LDR images at different exposures that I talked-about above.

The second “row”, containing only a single picture, is the result of the several steps I mentioned above to produce an HDR image.  I make no claims into how “real” this image looks, but it does show the details that I was looking for – from the clouds to the sand – in one image.  I also think it looks cool. 

This is the first HDR image I’ve created that I’ve wanted to share, and so I took this opportunity to explain the process too.  If you have any questions, then by all means, please post a comment and then subscribe to the comments so you’ll know when I’ve replied.