Sorry RT - We're Just Not Meant To Be

This time last week I got very excited about the Request Tracker (RT) being the perfect match for The Goo's higher level requirements: email alerts, tasks, bugs, web interface, command line interface etc. It sounded so right! :-)

But the relationship got off to a rocky start when it took two days to install! When we finally got it installed it was a mixture of relief and excitement. Then we started to try RT out. For three days we tried to get to know RT better but the web interface felt clunky and lacked clear workflows. We didn't even bother to look under the covers - it just wasn't right.

So, sadly this marks the end of our short, but intense, relationship with RT. At least we have a better idea of what we're looking for!

Tracking a Request for a new Care[O]Meter

Our install of Request Tracker (RT) at Trexy.com and Turbo10.com is complete and we're busily making lots of new task and bug tickets. The web interface works OK but over the next week I need to explore the rt command line interface and then create action handlers for a new Thing (i.e., rt.goo). This should give me the ability to handle multiple tasks and bug queues via The Goo and most importantly to traverse the scaffold between problem and solution and juggle things while not getting overloaded. Phew!! I've managed to duck writing a task / bug management system as part of The Goo. Which I suppose is the point:

The Goo is a mechanism for sticking Things together not implementing the underlying Things - it just tries to implement the association layer.

My new Care[O]Meter will need to talk to RT, but tasks and bugs are not the only Things I care about while coding. What about ideas? pain? projects? feelings? people? Why not have multiple queues? After all this is what happens in real life. My desk, for example, acts like a kind of queue. I use the "messy desk" filing system - more important stuff gravitates to the top. I'm considering filing Things like ideas, pain etc, into RT request queues. But it all depends on how good RT is at handling permissions and multiple queues etc - the next week will tell.

If it works well, I will no longer have to implement the care factor as a DatabaseThing and I can store it all in RT.

Yum Yum a Thing Installer

Installing Things should be really easy. So here's the plan:

shell> goo -p look.pdf
No Thing found for "pdf". Do you want to check the Thing Registry at TheGoo.org? [Y/n]
Found "pdf.goo" Thing:

Thing: pdf.goo
Author: Marcel Holan (mh@petamem.com)
Web: http://petamem.com/pdf-thing.html
Description: Handle adobe stuff. Trust me it works!
Version: 8.9

Do you want to install this "pdf" Thing? [Y/n]
Downloading pdf Thing ... done.
Unpacking ... done.
Installing "pdf.goo" done.
Continue with "goo -p look.pdf"? [Y/n]
shell>

The Goo is getting Squished

With all the progress that has been made on the Thing Registry it's time to turn my attention to the core Goo that has been released to CPAN.

The core Goo needs to be as lean and mean as possible. The modules that thrive in the CPAN environment are simple, useful and functionally elegant. This is not how I'd describe the current CPAN distribution (i.e., 0.0.9), more like "complex" and "unwieldy". So it's time to start code cutting.

The CPAN distribution needs to act as a minimal bootstrap. The rest of the Things can be dynamically loaded from the Thing Registry at TheGoo.org. This dramatically reduces the size of the CPAN distribution, which means less pain for me as a maintainer and greater simplicity for CPAN users. It also means I can release/update individual Things without making changes to the core CPAN distribution so I can incrementally stick Things back into The Goo and other contributors can too. Because my only conduit to add Things back into The Goo is via the Thing Registry it will exercise this new part of the system too.

I've removed over half the modules in the current distribution and there's more to come. It sounds harsh but if I want The Goo to thrive it needs to adapt to its environment.

Where the Trail Leads ...

Now that Trexy.com has been released to the world. I can go into more detail about what I mean by a "Trail". I've just added this page to TheGoo.org that brings together some of my previous blog entries from blog.Trexy.com.

Registering Things

So here's the plan. I've decided that Thing contribution and distribution needs to be as simple as possible. The other big decision is that Things can be written in any language - not just the mighty Perl. Hopefully we will see action handlers written in Python, Ruby, Java, C, bash etc.

This means we need a way of handling Thing contribution and distribution for non-Perl Things. CPAN is perfect for Perl distribution but does not handle other Things easily.

So, I've decided that the core of The Goo will still be released to CPAN, but all the other addon/plugin/extension Things are registered and distributed via TheGoo.org. This means TheGoo.org is the nexus of all Things.

So last night I hacked together:

• Packaging Things - a quick HowTo on what you need to start
• Register your Thing - a simple Thing registration page
• Registered Things a page listing all user-contributed Things

I want to pay tribute to Jamie Cameron and his brilliant Webmin system for inspiring me here.

Mailing List Madness

Three members of the Goo crew worked at various times to try and install a mailing list. Something like this should be really easy but each package proved problematic in one way or another. The good news is after much persistence the mailing list is here!

The mailing list is for anyone interested in The Goo. And if you're reading this that includes you! So what are you waiting for? Join now.

Keeping DRY - "Request Tracker (RT)" to the Rescue

I implemented a very simplistic task/bug tracking system while developing the two search engines: Trexy and Turbo10. It worked really well because I could stay inside The Goo and jump from task (problem) to code (solution) and back again.

But Trexy and Turbo10 are growing fast, and our task system is finding it hard to survive in this new environment: more people, dependencies, locations, tasks and bugs etc. We could spend precious coding time and effort mutating our existing task management system so it works well in its new environment ... or we could drop in an entirely new system that is more mature and can handle our enlarged requirements. We need a system that has already weathered the demanding requirements of an enterprise task/ticket system and that can stick together with The Goo.

We need a console-based "Goo" interface (for coders) and a web interface (for business developers). I think I've found what I'm looking for, the Request Tracker (RT) from bestpractical.com will save us having to repeat all this work, so we stay DRY and the RSS interface will hopefully provide the pipeline to and from The Goo.

Universal Action Handlers

You can specify the actions you want to perform on a Thing in its .goo configuration file. For example the pm.goo file is for Perl module Things. Here is an excerpt from pm.goo:

# show a profile of a Perl module
[P]rofile Goo::Thing::pm::Profiler.pm

The left hand column is the action (i.e., [P]rofile) and the right hand column specifies the action handler (i.e., Goo::Thing::pm::Profiler.pm). I want The Goo to work with lots of different Things but also lots of different action handlers. Including action handlers written in another language (e.g., Python, Ruby, Java etc). So this has got to work too:

# show a profile of a Python module
[P]rofile Profiler.py

So I'm taking the opportunity to move the foundation stones of the Goo while I still can. Then I can set it on its way.

Typeless Translator - Naturally Selected?

Here is an example of what I mean by Darwinian software development.

Last year I felt some pain: "I'm a crap typist. I need to type faster and more accurately."

So I came up with an idea: "Imagine if you could write code in abbreviated form like, SMS txt mesgs."

And a task: "Create a Perl module to help me type less: TypelessTranslator.pm."

The TypelessTranslator works as a post-edit filter. It scans my edited code for abbreviated code and expands it. For example, "fe m $member (@a)" expands to "foreach my $member (@array)". In a similar way that you, or your mobile phone, expands SMS messages. The idea is to "type less" but "read more".

Sadly, from a Darwinian software development point of view, this one is tettering on the edge of falling back into the primordial soup. It turns out to be quite tricky to implement and has an annoying habit of expanding into the wrong words. But it is an idea worth having as it may mutate into something truly useful.

Darwinian Software Development

One of my old bosses scared the hell out of me. She described the first release of the software we were working on as like a shiny new car rolling out of a garage that runs perfectly first time. Needless to say the project failed and it completely dot-bombed. Why? She was managing the software development yet didn't understand the fundamental nature of the beast.

Software must be malleable. If we're looking for metaphors I'd say it has more in common with genetic engineering than mechanical engineering. Software needs to survive in harsh environments and must adapt quickly. The adaptation process in nature is cruel but effective: survival of the fittest.

If our software is going to thrive in its environment we need lots of ideas (mutations) to be generated but only the best survive - until the environment changes again - and the process repeats.

Help is on the Way

Uwe Voelker paid TheGoo.org a visit yesterday after having a difficult time installing The Goo at CPAN. The "difficult time installing" is the bad news. The Goo is supposed to save you time, not waste it!

The really good news is that Uwe has offered to lend a helping hand! ;-)

He has already sent in some great patches and we're sending him the keys to the subversion repository.

Now is a great time to join the project. Email nige@thegoo.org if you're interested!

A moment to PAUSE?

"Release early release often" is an extreme programming mantra. But we don't want to be too "extreme". Our seventh version in as many days is now on its way to the Perl Author's Upload Server (PAUSE) at CPAN.

The release rate will calm down soon but we're still fixing install bugs so please be patient. The ALPHA version needs to work straight out of the box.