Comfortably Coding Blog of Mathew Rodley

I'm currently writing a neat little application using Rails 3.0. It serves up web requests and provides a nice RESTful end point for me to rig up some fancy calls to via a JavaScript UI.

To compliment this web application, I have written a primitive NodeJS substrate to manage communications with mobile devices. Though this layer, mobile devices can invoke actions on the web application. This works fairly well, except when request_from_forgery gets in my way. I needed to get around this restriction, but I did not want to disable CSRF completely on the web application. In a nutshell, I wanted to check the CSRF tokens if the user is coming from any non-trusted source and ignore the tokens when coming from a trusted source.

Since the NodeJS is in my sphere of trust and has already authenticated any users that it is communicating with it, I created a non-standard header to communicate a signed identity to my web application. The web application verifies this identity and then invokes the method being called on behalf of that identity. However, since I supply no CSRF token the request failed. Ideally, if the identity sent in the header is valid (and signed by the NodeJS application) I want to also ignore the CSRF token. However, I want the CSRF token to be verified for any other request. Rails provides a mechanism to ignore the verification token per controller or method, but I want a much deeper level of control (per request).

Here's what I did. If the authentication token is valid, I modify the request to place it in the forgery whitelisted category. This means it will not verify any tokens and the request will proceed as if it does not require CSRF validation. To do this I redefined the forgery_whitelist? method on the request object. It more or less looked like this:

if custom_header.is_valid?
  def request.forgery_whitelisted?; true; end
end

A couple of days ago, I upgraded my MythTV frontend to use an SSD drive. It's been great, it's very quiet and prodeces a low amount of heat. I also used this as an opportunity to refresh the system by rebuilding it anew. I backed up /etc/ and ~'s and went about rebuilding the system.

Unfortunatly, once I finished and tried the new system, things were not quite smooth and I was hitting a really annoying bug with my infrared remote used to control the system. The remote would only work in MythTV 2 minutes after startup. Wow, how annoying... A few quick commands showed me that this was nothing wrong with the drivers (which I had to patch myself), or lirc as irw reported the button presses coming through just fine. So my conclusion was that MythFrontend was doing something odd.

Beginning the Investigation

I was running mythtv--0-22-fixes, so I fired up SVN and checked out the source for that release

svn co http://svn.mythtv.org/svn/branches/release-0-22-fixes/ mythtv-0-22

Armed with the fantastic power of x-ray-googles, I began looking at how MythFrontend picks up LIRC events. A lot of this magic is in the sources in mythtv/libs/libmythui/*.
In mythmainwindow.cpp, Myth initialises a thread which reads from LIRCD on whichever UNIX socket or tcp/ip socket you have it setup from (this is in MythMainWindow::StartLIRC())

This begins a thread which has a its runloop defined as LIRC::run() in lirc.cpp. However, before this is called, LIRC is initialised by LIRC::Init(). Needless to say, all of this was performing successfully and Myth, for all intents and purposes was reading my keypresses from /dev/lircd which were picked up from my remote by LIRCD just fine (irw prooved this).

At this point I moved further down the stack. When a code is received from LIRCD, the LIRC thread calls LIRC::Process(data), passing it that keypress information. This method will do some logic and then send a KeyPress and a KeyRelease to the MythFrontend main window

            QApplication::postEvent(
                m_mainWindow, new LircKeycodeEvent(
                    QEvent::KeyPress, keycode, mod, text, lirctext));
            QApplication::postEvent(
                m_mainWindow, new LircKeycodeEvent(
                    QEvent::KeyPress, keycode, mod, text, lirctext));

and later

        for (int i = (int)keyReleases.size() - 1; i>=0; i--) {
            QApplication::postEvent(m_mainWindow, keyReleases[i]);
        }

Some quick working with a debugger told me that these functions were being called. At this point I began to wonder if the mainWindow was receiving these events. I tracked the callback handler for these events to libs/libmythui/mythmainwindow.cpp, to the customEvent method.

void MythMainWindow::customEvent(QEvent *ce)

Now, this is where it gets interesting. This customEvent method was being called for all Keypresses, HOWEVER it was ignoring some! - Aha, my problem was here somewhere (or so I thought…).

Inside this customEvent method, there is a large conditional to deal with a number of cases, one of which deals with LIRC keycode events:

    else if (ce->type() == 
             (QEvent::Type) LircKeycodeEvent::kLIRCKeycodeEventType && 
             !d->ignore_lirc_keys) 
    {

It turns out that this was not being called, which was why I was experiencing my problem (but not the root cause). Now, the smart readers here will notice the !d->ignore_lirc_keys condition. This was causing the problem. For some reason ignoe_lirc_keys was true for the first 2 minutes after starting up… What… why would this be? Well, to find out I had to figure out how ignore_lirc_keys could be set to true.

Setting ignore_lirc_keys to true

Some plodding around in the source code revealed that there is an LircEventLock::Lock() method in lircevent.cpp. This was being called by someone, causing mythfrontend to ignore LIRC keypresses. To get a simple backtrace at this point, I modified the code to include

#include <execinfo.h>

so that I can call backtrace() and backtrace_symbols(). This allowed me to get a stack trace at the point the Lock was set, and see who might be calling Lock, and annoying the hell out of me. Here is the result of that stack trace:

Obtained 10 stack frames.
/home/user/sandbox/my-myth-build/lib/libmythui-0.22.so.0(_Z11print_tracev+0x26) [0x5f7796]
/home/user/sandbox/my-myth-build/lib/libmythui-0.22.so.0(_ZN13LircEventLock4lockEv+0x1f3) [0x5f7a03]
/home/user/sandbox/my-myth-build/lib/libmythui-0.22.so.0(_ZN13LircEventLockC1Eb+0x25) [0x5f7bf5]
/home/user/sandbox/my-myth-build/lib/libmythui-0.22.so.0(_Z11myth_systemRK7QStringi+0x48) [0x5a9198]
/home/user/sandbox/my-myth-build/lib/libmyth-0.22.so.0(_ZN15MythMediaDevice15performMountCmdEb+0x58e) [0x669807e]
/home/user/sandbox/my-myth-build/lib/libmyth-0.22.so.0(_ZN7MythHDD10checkMediaEv+0x98) [0x67623e8]
/home/user/sandbox/my-myth-build/lib/libmyth-0.22.so.0(_ZN12MediaMonitor12CheckDevicesEv+0x5e) [0x669a66e]
/home/user/sandbox/my-myth-build/lib/libmyth-0.22.so.0(_ZN13MonitorThread3runEv+0x38) [0x669a708]
/usr/lib/libQtCore.so.4 [0x59cce32]
/lib/tls/i686/cmov/libpthread.so.0 [0x64e80e]

As you can see, Lock() is called by myth_system, which is called by MythMediaDevice::performMountCmd, called by MythHDD::checkMedia etc…

BINGO

Armed with this knowledge, I knew exactly what was happening. The myth_system() is used to execute commands, however when these commands execute, sometimes we want mythfrontend to ignore LIRC commands sent to it. For example, if myth_system executes mplayer, which is now in-front of mythfrontend, we should ignore LIRC keypresses on Myth. However, the mount command also caused mythfrontend to Ignore Keypresses while mount was being run.

But why two minutes? Because I have a cardreader which registers 4 SCSI devices which cannot be mounted without cards in them. if you try and mount them, the mount command waits an amount of time (sometimes up to 30 seconds, it seems for me) then gives up. The sdb, sdc, sdd and sde devices in /dev

user@kmedia2:~/sandbox/my-myth-build$ ls /dev | grep -e "^sd"
sda
sda1
sda2
sda5
sdb
sdc
sdd
sde
user@kmedia2:~/sandbox/my-myth-build$ 

dmesg

[    7.064559] usb-storage: device scan complete
[    7.101340] scsi 8:0:0:0: Direct-Access     Generic  STORAGE DEVICE   9602 PQ: 0 ANSI: 0
[    7.107230] scsi 8:0:0:1: Direct-Access     Generic  STORAGE DEVICE   9602 PQ: 0 ANSI: 0
[    7.120038] scsi 8:0:0:2: Direct-Access     Generic  STORAGE DEVICE   9602 PQ: 0 ANSI: 0
[    7.148029] scsi 8:0:0:3: Direct-Access     Generic  STORAGE DEVICE   9602 PQ: 0 ANSI: 0
[    7.148391] sd 8:0:0:0: Attached scsi generic sg2 type 0
[    7.148466] sd 8:0:0:1: Attached scsi generic sg3 type 0
[    7.148544] sd 8:0:0:2: Attached scsi generic sg4 type 0
[    7.148619] sd 8:0:0:3: Attached scsi generic sg5 type 0
[    7.447250] sd 8:0:0:1: [sdc] Attached SCSI removable disk
[    7.458246] sd 8:0:0:3: [sde] Attached SCSI removable disk
[    7.469466] sd 8:0:0:0: [sdb] Attached SCSI removable disk
[    7.491250] sd 8:0:0:2: [sdd] Attached SCSI removable disk

the mount command

root@kmedia2:/mnt# time mount /dev/sdb trash
mount: /dev/sdb: unknown device

real    0m18.158s
user    0m0.004s
sys     0m0.000s

By the way, if I rmmod -f usb-storage, which removes these devices in /dev, mythtv starts fine.

The Simple Fix

So, armed with the known cause of the issue, and the fact that myth_system accepts a flag called MYTH_SYSTEM_DONT_BLOCK_LIRC, which causes the system() command not to block LIRC keypresses on the front end, I added a small patch.

I changed line 124 of libs/libmyth/mythmedia.cpp from

        if (0 == myth_system(MountCommand))

to

        if (0 == myth_system(MountCommand, MYTH_SYSTEM_DONT_BLOCK_LIRC))

I compiled the libmyth library, and everything was OK… the bug was gone!

Well this seems pretty cool, http://codility.com

Although, it seems to just quiz you about basic asymptotic growth and does not seem to look at things like programming style etc. Still, I'm please my Ruby answer for the demo test got 100%, which, as the histogram shows... Not many people get. I suspect that the reason is, many people probably submit an O(n^2) answer, which fails for very large data sets. Here is what I submitted

And here is what I submitted:

def equi(arr)
  lhs = 0
  tmp = 0
  rhs = arr.inject(0) { |a,b| a + b }
  arr.each_index do |i|
    lhs += tmp
    rhs -= arr[i]
    tmp = arr[i] 
    return i if lhs == rhs
  end
  return -1
end

And Here's the analysis it gave back to me:

test	time	result
example Test from the task description	0.012 s.	OK
extreme_empty Empty array	0.020 s.	OK
extreme_first	0.008 s.	OK
extreme_large_numbers Sequence with extremly large numbers testing arithmetic overflow.	0.020 s.	OK
extreme_last	0.012 s.	OK
extreme_single_zero	0.020 s.	OK
extreme_sum_0 sequence with sum=0	0.020 s.	OK
simple	0.016 s.	OK
single_non_zero	0.012 s.	OK
combinations_of_two multiple runs, all combinations of {-1,0,1}^2	0.008 s.	OK
combinations_of_three multiple runs, all combinations of {-1,0,1}^3	0.012 s.	OK
small_pyramid	0.072 s.	OK
large_long_sequence_of_ones	13.349 s.	OK
large_long_sequence_of_minus_ones	10.781 s.	OK
medium_pyramid	4.336 s.	OK
large_pyramid Large performance test, O(n^2) solutions should fail.	21.369 s.	OK

Todays post is about creating a mirrored effect on images appearing on a website. Looking around, there are already a few people who have attempted to do this, however, I would like to tackle the problem myself and provide an in depth post about how to actually do it. At the end of this post we should have a fully functioning jQuery plugin which allows us to mirror images. We can approach this two ways, using Canvas or SVG. I have opted to use Canvas, however I did create a neat SVG file which is able to mirror arbitrary images, passed in by a query parameter. The first part if this post will address creating a reflection of an image. Once we have the ability to do this, we can roll it into a jQuery plugin to provide reflections for images which we specify. Read the rest of this entry »

In the past I have played around with JRuby and accessing some of the Java framework from Ruby. Its fantastic and I have raved about it infront of my friends in the past. Microsoft as well now have a product called IronRuby which runs Ruby inside Microsofts Dynamic Language Runtime (DLR). IronRuby is a Ruby interpereter which runs in the DLR and as a byproduct gives Ruby access to the .NET framework (yay, power to the programmer!). Read the rest of this entry »