status update

I've been working my day job, getting settled in. After a couple of months I've now started to pry the lid off the can of whoop-ass; that just means I'm feeling rather comfortable in the whole workplace situation. My desk is if not my sanctuary then at least my desk. Spent the summer coding Java (my official position is function tester) and generally mucking about the tool-chain at work. Lots of fun and late nights. Eclipse is to Java what intercourse is to life, seriously! Hot stuff.

So the open-source activities ground to a halt due to the above. I haven't left for good, though. Don't be silly. I recently built a new computer (how does a 25x improvement in debug Firefox build speed rub you?) and once I get the dev environment set up just right, I intend to attack the Prism bugs I left hanging. Then, well, I hope I'll be more involved with js-ctypes and those bits again. I kind of miss it.

So, that's it. For the 0 readers of this blog — enjoy!

my computer hates you

dudemeister$ history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head
143 lame
65 yet
23 omg
12 most
4 meme
3 the
1 suck-it

Never underestimate your own ability to do dumb things. I've been setting up unit testing for js-ctypes and they didn't work (called functions returned booleans instead of integers). When you reach the point where you're convinced your own code is up to snuff and there are essential parts of the XPTCall core that are buggy, just assume you're doing something wrong. Turns out I had declared library functions with (name, return type, call type, ...) when it clearly should be (name, call type, return type, ...). This from the get-go, even during my initial experimentation. It worked because the values I used meant default conversions and other parameters stayed within limits that made everything work but with unexpected results. Subtle, eh.

Also, I love the Linux integration but this sucks. I'm pretty sure it's GTK2 that draws some widgets on a solid background since the same Fx build on OpenSUSE 10.2 showed similar junk around the <select> whereas, as is evident, Ubuntu 7.10 draws it perfectly. An eye-sore is what it is.

whatever

The buffered output stream-related bug is solved! Turned out it was a bad patch in bug 363986 that caused it. Thanks Valgrind, if you weren't an inanimate blob of bits I would kiss you! (I may print out the code and roll around naked on the pages as a substitute.) And I have my first main tree patch; 4 lines, but it's still a Big Deal™ to me personally.

In other news, I managed to hose my OpenSUSE 10.2 install by upgrading GTK+ and some other bits and bobs, so I installed Ubuntu 7.10 yesterday. The debug builds still run which is totally sweetpickles. Feels faster, too. I hope build times decrease.

dbus with all the fixin's

I was a bit bored, so I fixed up the marshalling code for mozjs_dbus and now it speaks Unicode. To be fair, my changes were only a couple dozen lines. Credit where credit's due, the XUL and JS stuff is all Eric Butler, and I think most of the C++ is too (though the marshalling code looks nicked from now-defunct DBuzilla). Since it's a JS-centered approach, I think one could rejig the whole C++ component to tap into JS directly (like js-ctypes does). That saves going through nsIVariant, and allows the back-end to chew on the JS objects directly. It would probably simplify the JS slightly as well (it is already a nice, modular design).

And I'm pretending the 3 hours spent looking for the reason why my Fx build didn't pick up the mozjs_dbus extension never happened. (The install.rdf ID field said mozjsdbus. One underscore. Three hours. In theaters now...)

exacerbate and incapacitate (and, apparently, progress)

Ah, software. S for stupid, O for OMGWTF?!, F for [expletive deleted] etc.

Weird, weird bugs. Profoundly weird. So, converting from .ico to .png causes the same buffered stream bug(?) to throw a hissy fit and explode ze Prism into tiny shards of glass that hurt your soul. But, the other day when testing I chose a local .ico file, and for some reason it actually converted without crashing. To top things off, the /etc/mime.types entry for .ico was "image/x-ico" which isn't a standard MIME type. So, using a local .ico file caused the MIME service to report "image/x-ico" which in turn actually made Prism convert the .ico to a .png. Even though there is no decoder for "image/x-ico"!

Furthermore, adding a ~/mime.types file (overrides the global in /etc/) and putting the correct "image/x-icon" type there caused the thing to explode once again.

[some time passed here when I started fooling around with my imaginary girlfriend an idea, instead of writing this blog post]

My hunch paid off. Increasing the buffer size for the nsIBufferedOutputStream from 1024 to 4096 (larger than the size of the file that's being converted, it's 1920 bytes) magically disappears the crashiness (this is for .ico to .png with the correct MIME type). And, trying to convert a larger file (21941 bytes) with that 4096 byte buffer still causes a crash (this is from .png to .png). I'd allow myself to call this progress.

I might just add that this crashes a nightly Firefox when using Refractor, so it's not just my XULRunner tree that's covered in ectoplasm. Naturally, I can't recall if all this actually worked earlier.

I now have a debug XULRunner build, so I started poking at the odd crasher that bug 420777 acts as a workaround for. Converting from PNG to PNG (which should work, even though it should never be done) causes the whole shebang to SIGABRT when trying to close the buffered output-stream. Turns out it's nsBufferedStream::Close() that is the bad guy, and not nsStorageStream as I previously had guessed. More specifically the line nsBufferedStreams.cpp:109. Cesar confirmed on IRC last week that it affected him as well, so either it's a glibc bug that's in both OpenSUSE 10.2 and Ubuntu 7.10 (IIRC), or it's a Mozilla problem. I have no idea at this point, but I'm leaning towards the latter.

(A partial stack trace is available here.)

it's not a long way to Tipperary if you live on the outskirts of Tipperary

Hopefully we'll get some neat stuff done for Prism this week (we talked on IRC about menus for tray icons and the dock, and if that doesn't happen it's just a matter of triggering plasticmillion's Pavlovian response by filing a bug and it'll be done). There's also a couple of bugs to fix, I'd love to get the icon handling going smoothly (possibly whip up something based on imgLoader that can auto-detect the MIME type based on something else than what's supplied in the <link> tag or file extension -- content uses imgLoader so maybe one can do it without having to write a binary component [imgLoader's methods aren't scriptable]). At the very least, we need more graceful failure modes.

It's a really fun thing to be involved with, I feel like I'm turning into somewhat of a Mozilla evangelist. Becoming a part of the Prism team was very natural and I have to say if every single person who got started with Mozilla in some way had a Mark Finkle or a Matt Gertner (plasticmillion) as the first point of contact, there would be notices of proverbial cups overflowing with happy geeks all over Planet Mozilla. (Two really super guys, I spent most parts of Friday night vicariously getting Mozilla building using the iPhone SDK with/through Matt. The most pure nerd fun I've had in quite some time. Yes, I am easy to please.)

I've also started prodding a bit at the half-arsed JS-Dehydra thing I began back when the leaves turned last year (use SpiderMonkey to spid out the JS AST as JSON and massage in Python and make Dehydra run on that). Static checking (or, even better: automatic rewriting) of JS would be cool.
Reading all the incredibly cool posts by Dave Mandelin and Taras Glek really gets my mouth watering. I mean, a blog post like this. I don't understand half of it, and that's a blast. Learning something new implies not understanding it first. (Yes, I am a huge Feynman fan.)

flossing

It is surprisingly more satisfying than I had thought filing bugs, poking at the code and sometimes seeing p=nossralf in a check-in log. I mean, it's like a little buzz, more than a few synapses going "wheeee".

I've decided to learn a bit about the xpcshell HTTP server and writing unit tests for Mozilla in general, since Prism needs some lovin' in that department. Using the HTTP server makes sense, since it enables testing things such as the favicon downloader (and possibly aspects of Refractor as well). It may be too late to impact 0.9, but getting good unit test coverage won't make 1.0 kick any less ass.

And I need to get one of my other little projects, Tnedor, finished. It's a Python-based xbindkeys replacement with per-application settings (mouse buttons only for now). With an associated GUI application for setting up the configuration. Mouse settings on Linux suck. Bad. And reducing suck should be the goal of all software. (It's called Tnedor because Esuom sounds too Finnish and that's just wrong!)

seymour, do less

The blahgosphere is abuzz with talk of rich Internet applications, and has been for the last couple of months. Adobe's AIR was released the other day, and there's been a lot of talk about the technical merits of the platform, the security aspects, its place in the market, how it will compete with web browsers or the web in general, and so on. This is all well and good, programmers, analysts and strategists all love new platforms; poking at them, smelling them, squeezing, licking, your average verb+object bonanza.

But there's been little talk of usability, and I view this with some trepidation. The ability to easily create custom widgets and free-flowing designs will make it much easier to bring upon the desktop the curse of the skinners. Nothing good can come of this. I imagine Jakob Nielsen cries himself to sleep only to wake up later in a cold sweat.

I have yet to use a skinned desktop application (of the old steam-powered kind) that actually created a user interface that was clearly better than what could've been accomplished by using the native widget set. At best, and this happens extremely rarely, you get usability parity with a nicer visual presentation. Most of them have sucked more than a rational belief system can comprehend.

If there is a golden rule for interface design, it is this: Whenever a programmer thinks, "Hey, skins, what a cool idea", their computer's speakers should create some sort of cock-shaped sound wave and plunge it repeatedly through their skulls.

Skinning a desktop application used to be reasonably difficult, no native platform SDK provides you with the ability to do so without jumping through at least one hoop. With AIR I wonder if it's even possible to use native widgets. Add to that the fact that it's intended to actually encourage Flash or web developers to create skinned desktop applications, and it's a gaudy, blinking, oddly-shaped recipe for suck.

(One could add to this the accessibility aspect. Custom-widget designs and layouts don't respond to desktop themes that increase legibility, for example. If all AIR applications will communicate with screen readers or not is unknown to me. I know there are APIs to enable that, however.)

queuetie pie

Mercurial Queues is where it's at. The tray icon stuff a single-patch queue, and updating it against trunk is (after svn up-ing my hg base): hg qpop, hg pull -u, hg qpush and if there are conflicts, fix 'em and do a qrefresh.

And 3 hours chasing down a problem that was EOL-oriented isn't fun. At least I'm intimately familiar with Preprocessor.py and know more about the build system than before. And I've invented at least 4 new ways of cursing.

traysleeper

I filed bug 419600 to get the snot mercilessly reviewed out of me.

For generalized support, I currently assume too much about the MIME types of the image file that's handed to setIcon, but I never set out to make it anything but a Prism fit when I started (where the webapp installer nicely converts images to the proper format). It's all fixable, of course.

Also, I love Mercurial (hg mv). I check out the SVN trunk to a dir that is also versioned with hg, and clone off that locally. I think I'll try the localbranch extension to see if that makes it easier working on both bugs I find in the upstream stuff as well as the GTK tray stuff.

webrunner webrunner, going faster miles an hour

Oh yeah, the tray code for Linux now has feature parity with the Windows implementation. I never could get a native GtkWindow out of Mozilla, but instead of the whole InterfaceRequestor/Docshell/BaseWindow dance I just called nsIDOMWindowInternal's Focus method and called it a day. (SetZLevel only focuses the window currently.)

I'm contemplating menus for tray icons now. Hacked together some code that builds, but I haven't hooked up the plumbing through JavaScript. My thinking is to pass a menu's nsIPopupBoxObject to the tray code to attach it to an icon that way. Since Prism uses a 1.9 XULRunner, it will have the openPopupAtScreen method, and by getting the position of the tray icon that was clicked to bring up a menu, it can be opened at a reasonably proper location (modulo problems unknown at this time).
If I could get the actual GtkMenu out of Mozilla in some way I could use GTK's own menu positioning code for tray icons, but I don't see that happening.

Since XUL overlays can be loaded on-demand using document.loadOverlay, there's no reason why there couldn't be a webapp.xul overlay for more advanced usage of Prism (like for tray icon menus). But that's not up to me to decide. Perhaps using an extension is preferable in that case.

being Swedish, I must disagree

This is hilarious.

oopsie...

I had almost finished writing a blog post saying "it is XPCOM biting me, and I'm probably asking for it" when I realized that was indeed the case. What do you call someone who doesn't NS_ADDREF when he's supposed to? Stupid, that's what. (The jury will accept "inexperienced" too, but with me yammering on about "copying the Windows logic" without actually copying the NS_ADDREF, well, it's kinda stupid.)

Anyway, I found it by setting a watchpoint for the pointer (using "watch *((int*)0xblah)"), found out it was being freed in NS_DestroyXPTCallStub, which of course means its refcount was 0. Hard to call through XPTCall if there's no stub there to actually call through, natch.

At least it works now.

stuck inside of mozil' with the memphis blues again

Been trying to figure out why the heck I get a segfault when trying to call OnNotificationAreaClick when the tray icon is clicked. The GTK signal callback gets the correct pointer, but GDB doesn't really help with telling what it actually points to. Not even in ShowIcon, where aListener seems to point to something entirely unrelated:

Breakpoint 1, nsDesktopEnvironment::ShowIcon (this=0x858d2c0,
  aIconId=@0xbfd9ad28, aImageURI=0x858bde0, aListener=0x858d738)

(gdb) x aListener 
0x858d738: 0xb7a09ac8
(gdb) p aListener
$7 = (struct nsINotificationAreaListener *) 0x858d738
(gdb) x/wa *(void**)aListener 
0xb7a0bac8 <_ZTVN10gfxTextRun16PropertyProviderE+627120>: 0xb7715d4c

At least using this guide. It probably has to do with how nsINotificationAreaListener is defined and actually used (being QI'd through XPConnect, as the WebRunner object is the one implementing the interface). But I don't grok XPCOM enough to actually say that with any degree of confidence. Calling the OnNotification... method in ShowIcon does work at least.
Then we come to the callback which GTK calls when the icon is clicked:

Breakpoint 2, notify_cb (icon=0x8545120, data=0x858d738)
(gdb) p data
$9 = (gpointer) 0x858d738
(gdb) x data
0x858d738: 0x54746573
(gdb) x/wa *(void**)data
0x54746573: Cannot access memory at address 0x54746573
(gdb) x/s data
0x858d738: "setTitle"

The memory access thing is probably what makes it segfault. And yeah, that string is what happens to be there every single time I run it through the debugger. The callback's defined thusly:

void notify_cb(GtkStatusIcon *icon,
               gpointer data) {
  nsCOMPtr<nsINotificationAreaListener> listen; 
  listen = (nsINotificationAreaListener *)data;
  listen->OnNotificationAreaClick(0,2);
}

And hooked up like so:

g_signal_connect(mStatusIcon, "activate", G_CALLBACK(notify_cb),
                 (gpointer)aListener);

in ShowIcon (mStatusIcon is the GtkStatusIcon pointer, gpointer is typedef-ed as void*). Totally imitating the logic in the Windows tray code (where the pointer is stored as a property attached to the listener window that detects clicks on the icon). The actual segfault happens when the nsCOMPtr tries to AddRef the pointee in the assignment operator.

I would add a watch to see what actually happens to change the pointer, but since it's a multi-threaded app, GDB kills the watch as soon as the thread it's defined in gets preempted. So, what, Valgrind? Mommy, the big-boy pants are coming on... (Or I'm just doing something stupid that I'm overlooking.)

sprinkles or gummi bears?

Meh. Stupid segmentation faults. All I want to do (is have some fun) and call the nsINotificationAreaListener callback, and instead of sun coming up over Santa Monica Boulevard I get repeated punches to the groin. The joy of replacing dozens of lines of WinAPI guff with a single g_signal_connect was short-lived. (It can be called immidiately after entering ShowIcon but when the pointer is stored away and then retrieved in the callback function, similar to the Windows case, there be 'splosions.)

By the by, GtkStatusIcon is a GObject and not a GtkWidget descendant, which means it doesn't handle events like widgets do. So, no double-clicks for you!

the unbearable slightness of seeing

Okay, so the icon isn't much. But it represents nsINotificationArea support for WebrunnerPrism under Linux (not complete by any means, but it actually shows/hides the icon and sets the tooltip, which at the very least is measurable progress). Most of the time was spent wrestling with the build system, which is about as surprising as something not very surprising. A hard-coded icon file extension in webrunner.js cost me more than a few minutes of sulking too. (And what's the deal with XPM instead of PNG?)

It uses GtkStatusIcon, which was introduced in GTK+ 2.10, but seeing as how e.g. the printing changes that went into the Firefox 3.0 nightlies also require 2.10 or later, I doubt it's much of a problem. If there's any interest in this stuff upstream, that is.

Also, I'm really happy there are XULRunner SDK nightlies. Tastes great, less building.

having fun with js-ctypes

So, after humming and hawing until I was fresh out of hum and down to my last batch of haw, I figured the time was right to try and get js-ctypes working to start contributing more to Mozilla. I have a trunk Firefox debug build I update once a month or so and figured since I popped my build-your-own cherry about a year ago, with PyXPCOM enabled no less, I should be able to get js-ctypes going without too much crying.

Verdict: well that was easy. Check out the code from the svn repo into mozilla/extensions/, add it to extensions in .mozconfig like so:

ac_add_options --enable-extensions=default,spatialnavigation,python,js-ctypes

and Bob's your proverbial uncle. It built cleanly on my working Fx trunk build, and it was available to extensions and whatnot.

Now, I use Linux (openSUSE 10.2 currently) and Mark Finkle wrote on his blog when announcing js-ctypes, "compiles on Linux, but I haven’t got around to testing it." It turns out it built cleanly, but it didn't work. Some quick testing using a hastily thrown together extension showed that nsINativeTypes::Open() failed — it didn't load shared libraries (libc.so.6 in this case). After hitting mxr it turned out a PRLibSpec struct is initialized differently depending on the platform, and the code only covered the WIN32 case. That's bug #416119. The fix was easy enough, an #ifdef and a couple lines of code, but I'm still not sure the string voodoostuff is correct (I think the PromiseFlatCString call is redundant as NS_Convert... returns an nsCAutoString which is "flat" already).

That took care of the first problem and js-ctypes now loaded shared libraries in Linux. Score one for the newb. The next step was to try and call some function. I chose puts(), which is a part of libc and simply outputs a string of characters to stdout. Easy enough, right? Well, no. Since it uses a char* buffer for the string, I chose nsINativeTypes.STRING when declaring the function like so*:

const nsINativeTypes = Ci.nsINativeTypes;
var library = Cc["@developer.mozilla.org/js-ctypes;1"]
              .createInstance(Ci.nsINativeTypes);
library.open("libc.so.6");
var puts = library.declare("puts", /* function name */
                           nsINativeTypes.INT32, /* return type */
                           nsINativeTypes.SYSV, /* call ABI */
                           nsINativeTypes.STRING /* argument type */
);
var ret = puts("Hello World from js-ctypes!");

Upon trying that code I was greeted with a segmentation fault. It turned out there was no implementation for the native type STRING in nsNativeMethod.cpp. Since there's no default catch-undefined-and-return-error in the switch blocks in nsNativeMethod::Execute() and declare() doesn't care what type you specify (in reality any uint16 will work), when the function was called the call to ffi_prep_cif() in nsNativeMethod::Execute() had incomplete information about the parameters and exploded. Adding support for STRING is bug #416229. Thankfully, I could reuse the logic for WSTRING but use char* instead of PRUnichar* and fix the problem without any trouble.

The customary money shot:This was a fun little exercise in building/testing/hacking on a Mozilla project and has left me wanting to do more. Including adding support for struct types and writing some unit tests. My thinking as far as testing goes is to write a shared object/library with a set of known functions that exercise all the native types covered by js-ctypes, integrating that into the build and then adding tests to the unit test framework. I have little or no idea how to do most of those things, but figuring it out will be fun.

[*] Do note that the call ABI has to be specified. That bit me for about 30 minutes before I poked through the code thoroughly and realized the example code in Mark Finkle's original post didn't reflect that.