Gedit - The Goo's Editor

I've finally found a text editor that does syntax highlighting, word completion and starts quickly.

Gedit with the help of the word completion plugin does what I'm looking for.

1 Medal = 500 Karma++

After every 500 points of karma++ you now receive a medal!

I scored my first medal a couple of weeks ago. Now it's really starting to feel like playing a game - which is all part of the plan.

I'm feeling progress being made --- which is one of the preconditions to feeling "flow".

Sensations Snookered? - The Careenium to the Rescue

So I'm modelling conscious thoughts with 'perceptrons' and lower level thoughts with 'sensations' - which are fired automatically by 'receptors'. But at what point do sensations fire a perceptron?

If you touch something hot the sensation quickly gets upgraded to a perceptron - Ouch! Some receptors have a deservedly high priority - but what about more subtle sensations that need to coagulate and mingle before finally firing? - hmmm .... tricky.

Douglas Hofstadter conjures an amazing metaphor in his book, "I am a Strange Loop" to describe this process. He likens the human brain to a frictionless snooker table where tiny balls, called sims (small interacting marbles) constantly career into each other. Some sims bounce off each other, while others may stick together and form larger 'simmballs'. He suggests that consciousess emerges from the simms amd simmballs crashing around what he coins the 'careenium'.

I like the idea! So does The Goo need a careenium? Well currently I'm programming a 'See' Sensation to find filenames in emails, comments, web pages etc. Take this for example: Login.pm. Most of you will recognise this as a filename. The "." visual sensation, the mixed case, and the filename suffix (.pm) all combine to give this impression. But is that enough to fire a perceptron? Are there enough simms (i.e., sensations) for a simmball?

If you're a Perl programmer you will "see" a Perl module thanks to the ".pm" suffix and your mental model of what it is to be a Perl module. But this still may not be enough for a perceptron to fire. If Login.pm appears in an email from your boss, however, he may be referring to the Login.pm module that you wrote and suddenly there is a mental collision between your environment - the display of Login.pm in pixels on your screen and your internal simmball for Login.pm. Pop! And a perceptron is born -> Login.pm.

The firing process, in this example, is a combination of your current context, your memory (the trail) and the sensory forces in your environment (communicated via receptors). There's lots to do so back to implementing but for version one I'll settle for a very simplistic careenium.

Sensations need Receptors

I've decided I need to model "sensations" as well as "perceptrons" but the problem now is what picks up a sensation? Here again psychology offers some helpful terminology: receptor. Just like the receptors in the back of your retina - some sensors fire automatically in response to a stimulus. These low level sensors are important for classifying low level sensations (e.g., ___ ).

So practically speaking, in The Goo a Receptor generates Sensations which in combination with other Sensations may be picked up by a Sensor that may in turn "fire" a Perceptron (e.g., I see a red line).

Snipping The Gordian Knot

I recently painted myself into a corner with the design rule "everything must be a perceptron". The problem is that just like with human perception, events need to bubble up from their source until they reach a threshold and "fire". A perceptron is a perceptual event that has fired - but how do I model the preconditions for a perceptron to fire? I need something smaller than a perceptron.

Borrowing from psychology again:

Sensation is the first stage in the biochemical and neurologic events that begins with the impinging upon the receptor cells of a sensory organ, which then leads to a perception.

Sensational!

So instead of tying the design in knots trying to model perceptrons with perceptrons I've snipped the proverbial Gordian Knot and now lesser perceptual events are simply sensations that can chain sensors together to make a perceptron.

Do, Delegate, Defer + Divide = Smoother Workflow?

The todo subsystem is at the heart of The Goo. One of our ex-employees recently wrote:

"The job is great however I thought being involved in such a large organization there would be better procedures/communication in place! I miss the goo - esp the tutorials. You guys are certainly on the right track there."

But there's more to come! After my YAPC::EU presentation, Farley Balasuriya gave me some great feedback and said he'd recently seen similar ideas in 'Getting Things Done' by David Allen. I've since read the book and Allen makes some important suggestions - if the task takes less than 2 minutes Do It now - otherwise Delegate it to someone else or Defer it.

So in the pursuit of atomic todos I would add one more D - Divide It up. If the task is too big it can be a flow-stopper so it should be divided into smaller todos. I'm currently designing how todos will appear on your mental horizon - and the fours Ds will help you to wade through them.

giz - sticky editing

I'm going to release a small part of The Goo to CPAN before the London Perl Workshop in December.

giz is an $EDITOR wrapper that:

• works with your existing editor (e.g., vi, emacs, nano, pico, kwrite etc)
• helps remember where your files are located - so you can afford to forget
• injects a special comment into your Perl code to help with keyword completion
• can be called from a url (giz://filename=your_filename)
  

The Goo powered by Perl

I recently presented about how The Goo is powered by Perl at the YAPC::EU Conference in Vienna. Here are the slides:



Thanks to those who were there for the feedback and encouragement.

Group Karma++

It's a great feeling being in a state of flow. Sometimes you can get into a state of flow with a group of people, a kind of 'group flow'. This is where the team is working well and everyone is excitedly making progress while the technology is supporting their communication and ability to stay productive. I've been watching developments in the Perl6 Pugs project and the IRC log shows moments when the group hit productivity gold and achieved group flow.

One of the preconditions for flow is feeling progress being made. In the Perl6 pugs IRC log this is achieved by giving each other karma++. As you make commits to the pugs project you get a small pat on the back in the form of another point of karma++ - the more contributions you make the more karma++ you collect. You can see progress being made as the karma++ points tally up in the log.

I've been designing the scoring system for The Goo and I think karma++ is a really important part of feeling progress being made. But to help achieve flow we also need small mentally manageable tasks - atomic todos. So to help the system converge at smaller task sizes a user receives karma++ when they create a task - this should motivate them to create todos, and they receive karma++ when they complete a task too. When you view a person's context you see what channels they are tuned into and how much karma++ they have.

Sensors vs Censors

I'm reading Marvin Minsky's, "The Emotion Machine" and he makes a strong case for the role of "censors" in expert cognition.

Experts can arrive at the correct answer quicker. But how? Minsky suggests it's because the expert brain blocks unhelpful information and is not slowed down by useless processing. He refers to these automatic blocking agents as "censors". Censors, he suggests, free the expert brain to think about the important stuff. An expert learns from the mistakes of the past and censors help to avoid them in the future.

So far The Goo has been sucking in perceptrons via positive "sensors" but there is also a role for "censors" too. Spamassassin, for example, does a great job of filtering emails before they become email perceptrons. Spamassassin acts as a blocking censor.

But actively blocking events before they become perceptrons solves only part of the problem. There are still too many perceptrons coming in to stay in a state of "flow". They need to be channelled in light of your current mental context to prevent needless context switching.

So channelling perceptrons is the next step ...

Atomic ToDos

I'm having fun designing how to make the ToDo part of The Goo flow.

Flow happens when you can context switch smoothly in the face of a wave of challenging incoming tasks. The Goo needs to make it easy for users to resolve a task quickly, reward them in some way and move onto the next one. But there is a fine line between being challenged and overpowered by a task. Some tasks, for example, stop us in our tracks because the set up time is too expensive.

A ToDo task that is too unwieldy will break the state of flow. You might get stressed even thinking about where to start on a task. Stress is the flow killer!

Generally at Trexy and Turbo10 we try to break all our projects down into mentally manageable ToDos - which means we minimise stress while still making headway. No ToDo is too small and we actively avoid "projects" and "deadlines".

The trick is to break down a complex project into atomic ToDos --- indivisible, non-blocking, time slices of attention that get the job done.

Hmmm. How can I force the system to converge at atomic ToDos?

One way is to set a time limit per ToDo. How about a 'billable unit' to borrow a solicitor's term? Let's say five minutes.

Now some tasks will take less than five minutes (e.g., increase font size of heading to 20px) while others may take longer (e.g., write blog entry) but the time limit will generally encourage a smaller task size.

It also means that a user could acquire extra time over the course of the day if they 'beat the clock'. This helps to 'keep score' which is an important part of maintaining flow in computer games. User's should be rewarded in some way for beating the clock.

OK. So this is the next step for The Goo's ToDo management - atomic, 5 minute ToDos - and a cumulative scoring system based on time. Let's see if this helps find the elusive flow ...

Working with "Flow"

Game designers know implicitly that "flow" is the mental state they want their players to be in: happy, productive and engrossed in their game.

Wouldn't it be good if Microsoft Office could achieve the same effect?

Well my virus-ridden XP machine is currently showing the blue screen of death - so for me this is an entirely rhetorical question - but I hope you get my gist. If I can get into a state of flow while playing Tetris why doesn't office software try and do the same?

The Goo is some of the way there but to be honest it can't reliably get me into a state of flow. What can The Goo learn from game design? Here are some features of flow mentioned in a presentation by Daniel Johnson from the University of Queensland:

• Tasks that offer a chance of completion
• Tasks that facilitate concentration
• The task has clear goals
• The task provides immediate feedback (the need for a score or energy bar)
• The task leads to deep but effortless involvement (removes awareness of everyday life)
• A task that facilitates a sense of control over one's actions
• Concern for the self disappears during the task, yet paradoxically the sense of self emerges as stronger when the flow task is completed
• The sense of duration of time is altered

OK. In light of this it's time for the rudimentary user interface to go through another evolution ...

Bashing away at The Goo

A goal of The Goo is to help stick Things together and for a long time I've been wanting to add all my shell commands into The Goo. Thanks to some quick help from London.pm I can now capture all my bash shell history with the following ~/.bashrc setting:



export PROMPT_COMMAND='history 1 | /home/search/goo/lib/Goo/Sensor/ShellHistory.pm'

This inserts a perceptron into the database each time a command is executed and any 'things' in the argument list are automatically marked up. It also means the context is preserved between web based actions and shell actions.

Too Many Perceptrons?

In the last month over 70,000 perceptrons have flowed into The Goo.

I know that sounds like a lot but it doesn't feel too bad. It's not like your email inbox is suddenly inundated with 70,000 messages. The perceptrons are spread more or less evenly through time, and they kind of wash over you, for want of a better description.

Seeing wave after wave of perceptrons makes you feel slightly out of control but in a good way. All the perceptrons are flying in but once they go past it's reassuring to know they are frozen in the trail. Like events in a computer game some things can just fly by whilst others require special attention. The trail acts like a mental crutch collecting all the perceptrons.

Meanwhile, between waves of perceptrons, I've been bringing my cognitive science knowledge back up to date. Things have moved on since I studied 'cog sci' at the University of Queensland in the early 90's and I've really enjoyed reading Doug Hofstadter's, "I am a Strange Loop".

I'm halfway through the book and can't find too much I disagree with so far. If indeed our consciousness is, as Hofstadter suggests, a kind of "strange loop" then as a computer scientist it's very tempting to think of ways to optimise the loop!

All in all I still feel like I'm on the right trail.

Split Screen - Prior Art

Just found some prior art for a split screen - Vannevar Bush's Memex, 1945. I must be on the right trail!

New Interface - Balancing Perceptrons and Actitrons

Here is a screenshot of the really rudimentary interface that handles "perceptrons" (on the left) and "actitrons" (on the right). This tries to retain the 50/50, Ying/Yang, relationship between what you see and what you do when you're in a state of flow. The interface works with console based browsers and graphical browsers.

There is much more to be done on the interface but the essential 50/50 nature of the interplay between perceptrons and actitrons will hopefully get the user into a state of "flow" quicker.

Perceptrons - Feel the Flow

Email perceptrons are raining into The Goo. Great!

But this is only part of the problem solved. What about context switching and preventing buffer blowout?

I can only help improve context switching if The Goo is aware of the context I'm in. Added to this the context itself must be mentally manageable. Ideally I only want 7+/-2 things to juggle at a time.

Have you ever lost track of time in an "immersive" computer game? Computer game designers want you to feel "flow" as you context switch smoothly around the game. It's not a bad state to be in: your brain is working well, you're feeling good and you're getting stuff done. It feels smooth. It would be great if The Goo could help me get into, and stay in, a state of "flow".

The next phase is to capture the context I'm in by taking a snapshot of the perceptrons and actitrons in my buffer. See the thin green lines on top of the stick figure above. The perceptrons that get through will be appended to the "tail" of the trail as time goes on.

First Sensor - Goo::Sensor::MailFolder

I'm spending two hours a day implementing the next part of The Goo: perceptrons and actitrons.

I've decided on the layout of the interface and remarkably it works in the "links" console-based browser - no AJAX here!

Today I wrote my first "sensor" that captures perceptrons from email folders. Implementation options included:

• scraping user email in webmin
• fetchmail handler
• .procmail forwarding

But each of these introduce another prerequisite and configuration complexity. CPAN came to the rescue with two handy modules: Mail::Box and Schedule::Cron.

Currently a sensor POSTS perceptrons to a Goo::Server where they are recorded in a database. Each Goo::Sensor has a scan() method for searching its environment.

The next sensors to come are: tasks, chat and RSS...

I'll post a screen shot soon!

Modelling Perceptrons and Actirons

I use a hybrid database modelling methodology of ORM and E-R and over the weekend I started to model what a Perceptron and an Actitron is and how they relate to the Trail.

Here are some of the points that came out of it:

• A "perceptron" is a unit of perception that is picked up by a "sensor".
• A sensor is a software device for capturing perceptrons.
• The sensor must send where the perceptron was captured, when and who the sensor operates for.
  
• The perceptron may be observed but is not seen unless it goes via someone's short term memory buffer (7 +/- 2).
• A sensor passes perceptrons to a user or group of users or another sensor.
• An "actitron" is a unit of action which may be performed by a person or a sensor.
• An action performed by a person is always "seen" - but this may not be the case for a sensor.
• Over time a user's trail is filled with seen perceptrons and seen actitrons.
• Actitrons can happen on Things.
• A Thing is an object in your perceptual environment.

So what does this mean? Well by creating sensors your potential perceptual environment can be expanded (i.e., see more). Your own actions are remembered and can then be temporally associated with other Things which means you can make lighter context switches as time goes by (i.e., juggle more). It means you can also add future events into your trail by putting something in front of a sensor at the right time.

Actitrons + Perceptrons = Ying + Yang

I started writing a SciFi novel where in the future there are two types of things: Perceptrons and Actitrons. A perceptron is a unit of perception that somebody perceives (apologies to AI guru, Marvin Minsky for pinching his term) and actitrons are units of action.

My SciFi novel is on the back burner but I think these ideas could be useful in designing the next part of The Goo. When we work we perceive things and then perform actions as a result. A lot of time is spent moving from perceiving to acting and going back again.

Hopefully this diagram will help explain:

Seventh Heaven? Not quite.

At the start of the week every member of the team receives a "Pending Things Report" which shows the top ten pending Things. The problem is all the Things are set to care factor: 7. The care factor is not discriminating between what's not so important and what's really important. But the real problem is no one cares about the care factor - what is care factor 7 anyway?

So I'm looking at a different way of ordering Things without the need for maintaining a care factor.

Email is Br0keN

I started using email in 1990 and over the last 16 years I've progressively received more and more emails while sending less and less. In the last six months though, I've felt a change. The Russian "spam bots" are winning - exponentially so.

I now receive way more spam than real emails and I feel we're heading towards a tipping point - spamassassin does a valiant job of applying a Holtzam-esque shield but the slow, and crafty spam still gets through! I think we're converging on the "Spam Singularity".

The demise of email may not be such a bad thing --- I think there's more that can be done with a controlled hypertext system. A worthwhile message, for example, should always have a "what's next". I wonder what's next for email?

Drilling Down

I was talking to my dentist, of all people, about the CareOMeter. It turns out my dentist is full of ideas about workflow processes. His big idea for improving work processes is to capture the "what's next". He suggested that everything has a "what's next" even if it means "place this in the archive". He punctuates his points by spinning the dental drill so I wasn't in a position to argue - but I like the idea!

I didn't expect to feel this sort of pain whilst in the dentist chair.

Goo Location Creep

I've been wrestling with the problem of defining the file locations where The Goo will look to stick things together. I first implemented a monolithic config file, but although this gives fine-grained control over where The Goo should look I often find myself stopping and adding new locations as I move into new areas of the filesystem - where possible I want to avoid this feeling of going backwards.

The idea is The Goo should faithfully follow behind, seamlessly helping you to stick stuff together. The solution is for The Goo to incrementally discover the locations you're interested in and remember them. So the more you traverse your filesystem the more Things become "associated".

Help!! Information Overload - An Illustration

Subversive Heresy

I've tried various source control systems: clearcase, subversion and svk. Received software development wisdom says you *must* use a source control system. Why? Well here are some of the purported advantages:

• you can rewind the tape
• you can hack simultaneously
  
• you can branch
• you can browse (e.g., SVN::Web)
• you can blame

But in practice I rarely find myself rewinding the tape - my editor does that, and in the worst case scenario I can get the original file from production or staging. The retrieval cost of 'rewinding the tape' from the repository to a specific version is often higher than coding afresh (which version was that again?).

Simultaneous hacking on the same file in a huge code base happens infrequently but when it does resolving clashing files is a pain. The "blame game" is also pretty pointless. Mentally managing commits adds an unnecessary cognitive tax and takes time too. I don't know about you but I get a feeling of "repository drag"?

Ideally the software scaffold should not slow you down - but speed you up! I wonder if there is a better way? Fast browsing, no more commit overhead, clash prevention, automatic change logging and easy history traversal ...

Measuring Future Events

It is a goal of The Goo to help you stick Things together - now - but in the future too.

The ability to reason about the future depends on reasoning clearly about events that might, or might not happen, and their relative importance.

A recent O'Reilly book, "Mind Performance Hacks" suggests an interesting way of measuring the importance of something in the future. Paraphrasing loosely from the book it combines two Likert scales (1-7):

Likelihood of event happening:

Never                   Likely                          Certain
1         2         3         4         5         6         7

Importance/Impact of event

Not Important           Somewhat Important              Very Important
1         2         3         4         5         6         7

By combining the two scales you get a value out of 49 - and by doubling it a value roughly out of 100!

This simple heuristic gives you a measure of relative future importance. Why worry about something if it is never going to happen? On the other hand something may seem relatively unimportant yet it is certain to occur.

Stay tuned for how future events will find their way into the Care-o-Meter via this simple measure.

New Action Dispatcher - Targeting Things

The Goo has a new action dispatcher that does type-based dispatch. The action handler is chosen based on the type of Thing and it works from the web and the shell:

http://goo.localhost/r?arg1=tn7rc
shell> goo r tn7rc
shell> goo resolve tn7rc
shell> goo p HelloWorld.pm
http://goo.localhost/profile?arg1=HelloWorld.pm

But there are at least two other environments where I want to do actions on Things: email and while editing. Previously I mentioned the idea of ThereDOCs (e.g., p>>) - a way of embedding removable commands into text that help you context switch. Although there is a nice symmetry between Perl's HereDOCs <<>> - the ">>" characters are overloaded in the context of email (e.g., >> forwarding quotes) and Perl6 (i.e., hyperoperator).

So the new ThereDOCs (i.e., embedded action character sequence) will look like this: ~> and <~. The dispatcher must analyse the target of the action for 'Things' and based on type do the dispatch.

# imaging you're editing some code
use Hello::World; <~t # you want to jump to the tests for this module
use Template::Simple; <~p # you want to profile this module

# use Uri Guttman's sublime Template::Simple
Template::Simple->new->render("complicated.tpl", $tokens); <~p # jump to the profile of complicated.tpl

My::Logger::write("/tmp/look.log", $message); <~t # jump to the tail of look.log

When it comes to email I should be able to resolve a task by forwarding an embedded action to goo@somedomain.com. For example,

To: nigel@turbo10.com
CC: goo@somedomain.com
Subject: resolve~> tn5rc

Because all actions go via the Dispatcher it is also a natural place for remembering the trail of actions.

Mnemonic Thing IDs

I've just moved to mnemonic Thing IDs. Thing ID 15466, for example, has now become, tn6rc.

Why? Well mnemonics are one way we can do cognitive compression/chunking. But also because ...

• The ID itself carries meaning (e.g., tn6rc = Trexy task given to Nigel with care factor 6 Restart Cluster).
• Automatically finding Things IDs in emails, web pages, databases, logs etc. based purely on numbers can be ambiguous.
• Prefixing numeric Thing IDs with a hash # causes the Thing ID to be commented out in the bash shell and can also appear as anchors in URLs - making automatic Thing finding trickier.
  

Mnemonic IDs look more at home in a 70's airline booking system but I think they're due a comeback - TMTOWTDI - but that's also the point.

Test -> Code -> Test -> Code

How can I make this cycle quicker?

Well I just added a Perl5Tester to The Goo which jumps straight to a test for a module if one exists or generates a stub test file and then jumps to it. It sounds simple but this saves lots of time. Phew. No more directory changes between modules and tests.

Completing the Completer

I've been feeling the pain lately. My wrists are starting to hurt from typing too much. How can I type less while coding?

For the past week I've been considering changing editors. Currently I use nano because the key bindings are similar to the bash shell, it supports Perl syntax highlighting, and it works anywhere. But there is a crucial feature it lacks - the ability to do string completion in a file - I need this to save on keystrokes, short term memory and context switching.

So I've gone on a mission searching for an editor that fits my criteria: Perl syntax highlighting, in-file text completion, cross-platform, ideally works in a terminal, ability to customise key bindings, with a minimal learning curve. Here are just some of the editors I tried: mp, vim, vi, glimmer, gobby, emacs, kate, kwrite, nano, jpico, joe, jed, nedit, jedit, bluefish and others.

The winner is kwrite and an honorable mention goes to mp (minimum profit) for many reasons including this quote from the mp man page, "it helps you abandon vi, emacs and other six-legged freaks definitely."

The downside of kwrite is it won't work on a remote terminal and annoyingly flashes on start up - but apart from that it does the trick: in-file string completion.

Ok. So where does The Goo come in? Well it helps stick Things together by injecting a special comment into the file you're about to edit that is laden with Perl reserved words and context-sensitive method names. The string completion feature in your editor (e.g., vi, emacs, kwrite) sees the comment and picks up an expanded lexicon of keywords to use in string completion.

When you exit the editor the special comment is removed. Great! Less keystrokes, less context switching (no more looking up method names), and it works across different editors. Check it out:

Focussing the Email Filter

For the last week all my incoming email passed through a Goo email filter (i.e., milter).

The filter scans each incoming message for Things (e.g., HelloWorld.pm, 1212.idea, 233.pain etc) and adds a hyperlink to each Thing. The idea is I want to minimise the overhead of context switching from email to performing a task.

My first reaction was surprise at the number of Things found in my email - there are Things everywhere! Even things you don't expect. I received an email about London.pm, for example, and The Goo marked up this email with a link to a non-existent Perl module called London.pm - OOps. Generally though it works well.

I receive a constant barrage of email from "robots" on the Turbo10 and Trexy clusters. One robot, "Ernie the Exception Checker" sends details of exceptions via email. The Goo filter now automatically adds hyperlinks to the underlying programs so I can context switch quickly to the offending line of code, squash the bug and get back again.

Chicago Perl Conference

I'm doing a quick presentation on The Goo at YAPC::NA Perl Conference and here are the slides [PDF].

Critiquing your Code

The Perl::Critic module can critique your code based on Damian Conway's, "Perl Best Practices". This function is now available in The Goo too. Check out the screen shot:

Tackling Set Up Time

A big part of peforming a task is the time it takes to "set up". A painter, for example, must prepare the canvas, mix the paints, position the subject etc.

Sometimes the barrier to completing a task is not the task itself but the set up time required to do the task. Hmmm ... an improved task management system needs to take on this pain. Ideally the task description should minimise the set up time required for a task by including as much "scaffolding" and "materials" as possible. It should also help you clean up afterwards.

Hmmm ... this is definitely one for The Goo.

"Horizon of Care"

We need to log future events into The Goo "When is that conference coming up? When does registration close?"

A future event is a temporal thing and The Goo already has temporal events covered in the Trail. The Trail records the actions you perform on artefacts in your environment as "events" and as time goes by a trail of events forms.

So why don't we add future events into the Trail?

-----|---|--|--|--|--||--|-----|-----> [ The Zone ] ---------------------|------------|---->

This way you can traverse your temporal "Horizon of Care" by moving backwards and forward along the Trail - what you cared about in the past - what you are likely to care about in the future - what you want to care about now.

Polymorphic Queues - Feel the Buzz

We need to try to track the marketing "buzz" around Trexy.com and The Goo came to the rescue.

Thanks to the polymorphic queues of Things in The Goo we could easily add a new queue: Buzz. This means any marketing buzz can be forwarded to The Goo where it is recorded in a queue along with a "care factor".

Getting Sticky

To enable members of the team to email The Goo I needed to create an email filter to parse incoming emails. The email parser is called via .procmailrc and is implemented with the help of the Email::Filter Perl module.

But there's much more to be done. Why not apply an email filter across all emails in the organisation that helps to stick Things together? The idea is for a special email filter that analyses the content of an email and associates it with other Things by embedding links. The recipient of the email can quickly jump to, or perform actions on, these associated Things.

Emailing The Goo

Members of the Trexy team can now directly email The Goo! :-)

This means a really light context switch for those members of the team who spend most of their time working with email - all they need to do is carbon copy The Goo and a ticket/task/Thing is automatically lodged into the CareOMeter.

Action Time

I've just finished implementing a new action handling table for TheGoo at Trexy. It enables me to bind an action to a handler for that action.

For example: [P] -> Profile -> PerlProfiler.pm.

This means typing "goo -p HelloWorld.pm" on the command-line will cause the PerlProfiler.pm to execute on HelloWorld.pm. But it works in the Web environment too:

http://localhost/goo.cgi?action=p&filename=HelloWorld.pm

There is a new internal dispatch mechanism that maps an action to an action handler based on the "type of Thing". All this means that I can start to add functions back into The Goo quickly and the system can expand to handle lots of different Things (e.g., Javascript, HTML, SQL, Python, Ruby, Perl6 etc).

Goo Wobble

On the weekend I wondered if re-implementing the IDE-like features of The Goo was really a good idea? Was I wasting my time? Reinventing the wheel? Am I nuts?

So I downloaded "Eclipse", the IDE of the moment, and installed the Perl EPIC plugin. The raison d'etre of The Goo is to help programmers stick Things together, and there is one feature I simply can't do without: the ability to traverse the structural associations in your program. For example, I often want to traverse the inheritance hierarchy - what's in the superclass? When I clicked on the "Outline View" in Eclipse it took me no further than the program itself. I'm sure the class browser is more impressive for Java programs but for Perl programs I came to a deadend fast. :-(

I don't know about you but my programs hardly ever work in isolation - they always involve other Things. When I write a program I often have to context switch between these Things (e.g., program -> output, program -> file, program -> database, program -> log, program -> shell, program -> other programs etc). The Goo should help you to make all these context switches - fast.

So I closed Eclipse, noted the stream of uncaught Java exceptions in my console, and jumped back into the bosom of The Goo. No more wobbles.

The Goo at Trexy.com

We are now starting to heavily rely on The Goo at Trexy.com. Lots of tasks, ideas, bugs, etc. have been filed into the Care[O]Meter. I can now see at a glance what's happening across all team members and what I need to do too. We are more productive, communication overhead is lower, and nothing is "falling off the radar".

I haven't used the "I feel like ..." functionality as much as I had hoped but this is because I have too many tasks with a high care factor! Maybe when the pace slows down feelings can come back into the equation? But will it ever? The harsh reality is, some tasks just need doing, whether you feel like it or not. Hmmm. I still want to design a human-friendly, task management system and emotions matter - more design coming soon.

The "strategic" and "tactical" functions of The Goo are working well but we have lost "operational" functions while migrating from the console-based Goo to the web/console-based Goo. I'm really missing the ability to view my current trail, traverse backlinks, automate Perl module making and test making. What am I going to do about this? Check out the Goo related tasks in my CareOMeter:

Sparky powering The Goo?

Sparky is a lean, mean webserving machine I developed for Turbo10 and Trexy. I found configuring and performance tuning Apache + mod_perl so painful I ended up writing my own webserver!

Sparky is doing a great job of serving millions of hits per day for Turbo10 and Trexy. It's faster than mod_perl, consumes much less memory, has few prerequisites and is written in pure Perl. Last week I added POST request handling so Sparky now serves The Goo too.

The next phase in the development of Sparky is to fuse its currently separate Request and Response objects into a Context object (ala Catalyst). But I'm not sure how far I should go? Should I emulate the Catalyst dispatch mechanism too? Will this retrofit easily with Uwe Voelker's Catalyst port of The Goo?

I don't want to make yet another Catalyst (e.g., Jifty) but Sparky can't help but benefit from distilling the lessons learnt from these frameworks and others (CGI::Application, CGI::Prototype etc). Hmmm. No matter what happens I need to improve Sparky for Turbo10 and Trexy and a new Context object with a simple dispatch mechanism is definitely on the cards.

The Goo on the other hand will have its own application-specific dispatch mechanism no matter what underlying webserver is used. Given a 'Thing' in a URL (e.g., goo.cgi?thing=HelloWorld.pm&action=p), we need to look up its type (e.g., Perl 5 Thing), the action to be performed (e.g., p -> profile) and dispatch to the appropriate handler (e.g., Goo::Thing::pm::Perl5Profiler).

Each Goo user will also need to run a local "Goo Server" to access local files (i.e., a local file finder) and a light weight standalone server like Sparky is required here. Hmmm. Stay tuned ...

Finding Files and Doing Things

I've integrated the file finder with the Care-O-Meter. This means I can jump to a file quickly. Here is the latest screenshot of the Care-O-Meter (see the file finder on the right):

This means I need a separate form for updating the locations of Things in the file system:

Cool. Now I don't have to bother about remembering where something is located - no more tiresome directory traversals - up, down, up, up, down. The Goo automatically looks in these locations when it sticks Things together.

Goo URLs? Yes, please.

Hypertext is the original system for traversing associations between Things.

In a previous blog entry (Goo URLs? No Thanks) I mentioned that I didn't want to encumber The Goo with having to address Things using http://really_long_unambiguous_urls/that/are/a/pain/to/type. But The Goo sticks Things together by association and hypertext offers the path of least resistance!

For example, ideally I want to browse my Perl code in hypertext and then jump straight to an editor and then jump back again as quickly as possible (to minimise context switching). A recent patch by Jonas Fonseca to the Elinks browser unfortunately didn't quite do what I needed but he suggested an even better solution: Goo URLs!

To enable your own protocol in Elinks all you need to do is add this to ~/.elinks/elinks.conf:

set protocol.user.goo.unix = "/home/search/dev/goo %u"
set protocol.user.goo.unix-xwin = "/home/search/dev/goo %u"

I now have Goo URLs! This means if I want to edit a given method I can use a URL like this:

goo://?method=someMethodDeepInTheModule&filename=/tmp/MyModule.pm

Clicking on this URL launches an external text editor (e.g., nano, vi, emacs etc), starting at the appropriate line and means I can edit the file in place and jump back out again - quick!

It turns out the same method works in Firefox too. If you search on Trexy you will find my search trails for "firefox registered protocol". But here is a summary of what I found. To add a new protocol to Firefox do the following:

1. Type "about:config" in the location bar.
   
2. Right click, select New --> String.
3. Name of string should be: "network.protocol-handler.app.goo"
4. Value should be the path to your executable: "/home/search/dev/goo.sh"

This activates goo:// URLs in Firefox. I use KDE on Linux so I need to launch a window to wrap the application with a shell script (e.g., /home/search/dev/goo.sh):

#!/bin/bash
# launch the editor in a new window
/usr/bin/konsole -e /home/search/dev/goo "$1"

This means the Goo can be fully integrated with a graphical browser (Firefox) and a console based browser (Elinks) too.

There is now no technical impasse to traversing from high level tasks and ideas all the way down to editing, compiling and testing a small Perl script buried in the file system. Cool! ;-)

Care-O-Meter Feedback

The team has been using the Care-O-Meter for a week and thanks to their feedback I've made quite a few changes:

• private emotions - how you feel about doing a job/task is now kept private
• free text searching - you can search all Things for keywords
• you can choose to notify a specific email address when the status of a Thing changes

More changes are on the way!

More New Goo ...

I haven't posted a blog entry recently because my Care-O-Meter has filled up with lots of Things with a high care factor - sadly Goo blog entries are being pushed down. But today I feel like doing one!

Goo developments continue. I've written a file finder for quickly finding Things in GooSpace. Here is a screen shot.

The motivations for this are:

• reduce context switching - I need to jump from the Care-O-Meter to the underlying files as quickly as possible
• reduce the need to navigate hierarchical directories - no more up, down, up, up, down and all the context switching that entails - just jump to the file I want!
  
• simplify the .goo configuration files - we now no longer need to store locations per Thing
• quickly resolve filename clashes that are found in multiple locations
• gracefully handle mapping Perl::Module::Names to the underlying file
  

Each user has a list of directory locations in which to look for file Things and I've made a design decision to store this in the database instead of flat configuration files. Goo actions will also be stored in a MySQL database along with each user's trail.

In the next week I hope to hook up with elinks so I can find a file and then edit it in place and then jump back to the Care-O-Meter.

An Emotional Goo Debut

After spending two weeks reviewing "tracking", "task" and "bug" management systems I decided to stick with The Goo. Why?

The other task management systems were too harsh. Is that task priority 87 or 88? What's more none of them tuned in to how people really use them. I don't know about you, but I always end up scouring over the task/bug/project/todo list looking for what I feel like doing next.

So why not create a task management system that lets you tell it what you feel like doing?

Enter the Care-O-Meter. It shows a ranked list of Things you care about that you can assign a care factor to and search by what you feel like doing next. Thanks to Elinks I can access it via the console and the rest of the team at Trexy.com and Turbo10.com can access it from their graphical browsers too!

This is an alpha version so it's not 'industrial strength' like RT. Then again it is designed to be really malleable (i.e., squishy) and thanks to RT's brilliant idea of polymorphic "tickets", it lets you care about lots of different "things" (i.e., bugs, tasks, ideas, pain, projects etc).

Anyway why not see for yourself? Check out the new tour.

Uwe Voelker has even offered to turn it into a Catalyst web application! Stay tuned ...

Email Meets Web Meets Console

The Goo is all about sticking Things together. There is one big queue of Things I deal with that feels largely disconnected from the rest of my work yet is very important: incoming email.

Important events come in via email (e.g., server down, database replication failed, friend is getting married, have a baby etc) but also unimportant events (e.g., email spam).

I want The Goo to help file events into my Care-O-Meter, manage my event horizon and generally help stick Things together automatically so I don't have too.

So why not pass this queue of incoming email via a Goo-powered filter first? The email filter can scan incoming emails for other Things and associate them with existing Things. But perception, and association is only part of the problem. The Goo needs to take actions too. Actions include adding hyperlinks to all Things found in the email, inserting an event high up into my Care-O-Meter (e.g., friend is getting married) or performing an automatic action (e.g., restart server), or injecting a special Goo-signature into the email that contains URL-based actions to minimise context switching.

Now that would help stick Things together. ;-)