Injecting Events into the Trail

The Goo Trail records the actions or events that you do to Things in your environment. But what about incoming events that arrive from your environment? Server down. High load on cluster. Girlfriend on phone.

I spend about half my time making Things happen and the other half responding to "events".

Ideally I don't want to be buffeted by incoming events: "Chill out everybody! Form an orderly queue and please take note of my Care[O]Meter." Some Things deservedly fall below my threshold of "care" (e.g., spam email) while others deservedly displace everything else I was doing (e.g., server has gone down).

I want the Care[O]Meter to act as a lens on incoming events. Focusing my mind's eye on the important stuff while maximising my productivity and minimising the number of context switches I need to make during the day as I process useless events (e.g., spam email).

The Care Factor

The Care[O]Meter is one of the more unusual ideas behind The Goo - it is tied to the idea of the Emotional Programmer. A purely dry software construction memory isn't fun. I want a Memex with feelings.

That said, I have reservations about including it in the ALPHA release. One technical downside is it relies on a MySQL database which is a hefty prerequisite. A conceptual downside is it relies on the 'Care Factor'. We all care about different things while programming and it is presumptuous of me to expect that my fellow programmer will care about the same Things.

Hmmm. This is an important subsystem which I definitely want to share and I think advances the prior art. So does it stay or does it go?

I've decided it has got to stay but it means I'm going to model the "Care Factor" really *simply*. Here's how: any DatabaseThing that has a "care_factor" column gets juggled by the Care[O]Meter. No care_factor means it will never appear on the horizon of your Care[O]Meter. It also means you get to set the care_factor which means I don't have to worry about making you care about the same Things I do.

OK. That's decided then.

The "Associator"

The Goo just needs to provide a simple mechanism to stick Things together. As to what Things it can stick together, it should be as agnostic as possible (apart from the mighty Perl). It should provide an easy way for users to plug in their own Things and action handlers. Ideally action handlers could be written in any language, not just Perl.

A Thing is really just a "handle" on something in your environment: a file, a module, a script, database entity, a url, a log etc. A Thing is an artefact in your environment that you need to perform actions on. At the moment we have three types of Things: FileThing, DatabaseThing and a Thing. This covers an awesome number of artefacts but we also need to deal with Things at the application layer so here are a few more: URLThing, EmailThing and ApplicationThing.

The mechanism to stick Things together needs to firstly capture the trail of actions as we context switch between Things in our mental working zone. The TrailManager looks after recording the temporal associations between Things. But we also want to traverse the structural associations between Things too. It is the job of the "Associator" to distill and traverse the relationships between a Thing, its environment, and its creator.

Coding the Installer - Next Steps

We are working hard to get the install process running smoothly. It's not up to scratch yet but it will be soon. Please be patient if you are trying to install The Goo - this is an Alpha release. And if you can't wait, why not join in the coding? Email me at nige@thegoo.org for the keys to the repository.

Here is what the install process should do:

• create a per user ~/.goo directory
• create an SQLite database to store your trail. Found here: ~/,goo/goo-trail.db
• create a Things subdirectory ~/.goo/things to store all your .goo config files

And that's it for the moment. Each user has ultimate control over their Goo installation, including their Goo trail.

Subversion Repository is Here

You can now browse the code online.

Now is a great time to get involved with the project. We need all hands on deck to get to a BETA release. If you're interested in contributing, email me (nige@thegoo.org) and I'll send you the keys to the repository.

A mailing list is coming soon ...

The Goo Released

It was great to finally let The Goo out of the bag at the London Perl Workshop!

The audience seemed to really tune in to what I was saying and I hope my central idea of developing a Memex-inspired software construction tool got across. A Lisbon-based Perl Monger gave me some great feedback afterwards and he is very interested in coding on The Goo.

The Subversion repository is coming soon ... in the meantime you can download The Goo. Please note this is an ALPHA release - so expect bugs.

Alpha Release is Here

Thanks to the great help of the "expert programmer from PetaMem" we have a tar file ready for CPAN that you can download now.

I'm looking forward to sharing The Goo with everyone at the London Perl Workshop today. This is an alpha release but it also means it's a great time to get involved with the project. We are setting up a subversion repository and mailing list so stay tuned.

Some Thing for my Bro

My brother Andrew had his birthday on the weekend and I was trying to think of some "Thing" to get him ...

The best present we received as boys was a computer called a "Dick Smith Wizard". Imagine a Commodore 64 meets an Atari. My Uncle, who himself was a computer programmer, advised my mother to forget about the Atari and get us a *real* computer. The Wizard had only 16K of RAM, which by today's standards makes it more like a calculator. Nonetheless, the cartridge that spent the most time plugged in was not "Tank Attack" or "Crazy Chicken" but the "Basic" programming cartridge. At 7 and 8 years old the two of us were a little programming pair: extreme, underaged programming. I would do the 'vision thing' and Andrew would patiently explain two dimensional arrays to me. We had loads of fun and learnt lots too.

So in the same spirit of fun I'm going to make him a Thing for his birthday. His lingua franca is Python so that means:

> goo -m py.goo

London Perl Workshop Talk Submitted

I just submitted my slides for the London Perl Workshop this Saturday. It contains lots of screenshots of The Goo in action so the file size is pretty huge.

Click here to download the presentation:

• http://thegoo.org/thegoo-lpw-2005.odp (6632031 bytes)
• http://thegoo.org/thegoo-lpw-2005.pdf (21853598 bytes)

Perl6 Thangs

I've been avidly reading about the development of Perl6 and especially the awesome code cutting powers of Autrijus and the team at pugscode.org.

I'm also really keen to start using Perl6. So much so I want to make a new Thing for Perl6.

This would normally be easy. Just create a .goo configuration file Perl6 Things:

goo -m pugs

But the suffix of Perl6 Things is not "pugs" it's ".pm"! Bugger ... I've already got a Perl5 config file for ".pm" Things "pm.goo".

But this will happen from time to time and The Goo needs to be flexible enough to deal with it. Fortunately the solution is straightforward - we just need a slightly more sophisticated set of action handlers in pm.goo that can inspect the #!/shebang/line to determine what to do next.

I'd like to demonstrate The Goo working with Perl5 and Perl6 at the workshop at the end of the week.

Living Things

We need to model the team of people working in The Goo. So that they can email reports when bugs get fixed and tasks get done. I thought for a moment about defining people in the goo.goo config file but that puts too much onus on the config file parser, it doesn't extend easily and isn't as fun as making a "person" DatabaseThing.

This is how we will model people who support your activity in The Goo (i.e., other programmers, testers etc).

Care[O]Meter is back on!

Things are now starting to move fast. The Care[O]Meter is back working. I can now see what's happening and jump quickly to the next Thing. My programming pair can see it too.

Making Bugs

That was quick. Phew. Great I can make bugs again!

Going into The Zone

I'm in deep at the moment! I'm trying to get The Goo into a format ready for CPAN. This means lots of changes! ... Luckily I'm using The Goo to make The Goo.

Because it's a simple text based system I can use it directly on the staging machine at TheGoo.org. But not all the features are working and I'm missing them. Like: Bugs!!! - there's lots but no where I can quickly log them, Tasks!! Lots of these too but sharing them is slow. No [Z]one yet so I keep dropping balls :-(. The Care[O]Meter is also busted - so I have to manage what to do next.

I'm working in tandem with "the expert Perl programmer from Petamem" and it would be great if we could see the scaffold for this beast together.

Ok ... back to the [Z]one ...

The Goo @ London Perl Workshop

Cool! ;-)

I'm confirmed to speak at the London Perl Workshop.

> goo -m feeling
> Select a feeling? [1] Happy [2] Frustrated [3] Angry [4] Tired
> [1] Happy!!!

Now if I can just get my login on CPAN working! ... we may even have an alpha release ready in time.

Planning the Alpha Release

Here are the minimum features I'd like to include in the alpha release later in the month:

• Things - we need to include a few example Things (e.g., Perl modules, configuration files, log files, tasks, bugs)
• Goo Trail - this is a crucial subsystem
• The Zone - show your current working space (the tail of the Trail)
  
• ThereDocs - jump quickly between Things
• BackLinkManager - show all the backlinks that point to a given Thing
  
• Care-O-Meter - a very simple Care-O-Meter (no emotions just yet ;-))
  

Working with PetaMem will really help structuring the project for release. A lot of the functions I use need to be removed because these "Things" are specific to my environment. For example, the two search engines I have developed with the help of The Goo, Turbo10.com and Trexy.com use a homegrown templating system and many of the Things in the environment relate to this. These Things won't transfer well to another environment and it doesn't make sense to bundle them with The Goo.

The main purpose of the alpha release is to share the ideas behind The Goo and to demonstrate a proof of concept - it really is an alpha release. We need to distribute a minimal version which will hopefully act as a framework in which others can plug in their own Things. I'm interested to hear comments and suggestions too ...

<<HEREDOCS to THEREDOCS>>

Perl has a great feature for embedding a document straight into your program: HEREDOCS. The programmer can declare, "here is a document" and clearly delimit it from the rest of the program. The syntax for HEREDOCs starts with a <<MARKER and ends with a MARKER on its own line.

Sometimes while programming you don't want to include a document, but instead jump to it. I'm constantly needing to jump to another Thing and this context switching can often be time consuming. Too many context switches can lead to "buffer blowout" (a kind of mental coredump).

"I want to jump to 'there'" as quickly as possible. "There" is often a template, test file, command line or another program - but importantly it could be any "Thing".

While editing Things in The Goo you can use THEREDOCs to jump from one Thing to another. A THEREDOC is an action followed by two arrow characters (e.g., p>>). The letter corresponds to the action you want to carry out on a Thing.

I was about to launch into a written description of THEREDOCs but the best way to explain it is to show you. Slide show coming soon ...

Synaptic Silliness

OK. So I'm going to implement a few more Things: pain, ideas, projects, people and events (more detail on these Things coming soon).

These Things will be stored in a database. Now, I could do it the traditional relational way ... many-many relationships between projects and people etc. But maintaining the relationships between entities in a database adds an extra implementation tax to the system that is paid regularly in the form of constraint checking, coding for update anomalies, transaction checking, locking and the need to maintain many-many relationship tables (i.e., referential integrity).

Hmmm ... I want an easy life. I'm also keen to try something new.

I've decided to associate Things in the database a bit like our brain does. Cognitively speaking, the more a memory engram fires around the synaptic gaps of our brain the more indelible the synaptic trail becomes (emotions help too). Hmmm ... maybe the only associations that really count in the database are the ones that 'fire'.

So I've decided to associate Things in the database by processing the trail. An "Associator" will analyse the trail for associations between Things. You can think of this as a many-many table for all the entities in your database.

The associations table records the associations between Things so you can afford to forget them.

Goo URLs? No Thanks.

The Goo works by using a filename as a kind of tiny URL for files in your workspace. The Goo maps this filename to a location. It can be tricky if we have a file with the same name and different locations - which one do we choose?

Hmmm ... I want to keep everything really light and I think it would be a drag if The Goo required you to use formal URIs: goo://my_thing_in_unambiguous_location/thing.thing). It should just work. Generally the user should not need to worry about location - The Goo should stick to a Thing wherever it is. But what about filename clashes?

The principle of DRY (Don't Repeat Yourself) helps here. If there is a name clash, do you need both files? Is one incorrectly named? If it is the same file aren't you repeating yourself? If an ambiguity arises The Goo can ask the user to choose a file and optionally delete one.

Computer science loves hierarchy - just think of inheritance, file systems, URIs etc. Personally I'm sick of traversing hierarchies to get to what I need to do. Up, up, down, down, down, no not there, up, up, down. Crazy! Ideally I just want to associate one Thing to another - this means overlaying the hierarchical file systems we're stuck with at the moment. So rather than working in a strict hierachical environment my ideal is more like a blob of associated Things - an association layer - coloured green. ;-)

Decisions, Decisions ...

I'm trying to decide what is the best way to distribute The Goo.

But ideally I'd like users to contribute their own Things. So we need a system for contribution and distribution.

We could go with a simple tar.gz file, which would simplify distribution, but what about contribution? What's a good system for user contributed Perl code? CPAN.

The Goo Tour

Here is a tour which helps to explain what The Goo does: http://thegoo.org/goo-tour.pdf.

Changes @ TheGoo.org

Yesterday I improved the look and feel of TheGoo.org. I don't want to blow the "buffer" of first time visitors, so the main page has just three links: about, download and blog.

The about page tries to gently introduce The Goo before encouraging visitors to take a tour. The tour is a simple slide show of annotated screenshots. A second slide show will demonstrate how to make your own "Things". After all, There's More Than One Way To Do It (TMTOWTDI) and I want Goo users to make their own Things. It would be great if users could contribute extra Things to The Goo.

The yet-to-be-designed download page will provide simple install instructions and a guide on what to do next!

If someone wants to read about the background of The Goo the blog is the place to look.

Structural Association vs Temporal Association

Some Things are connected structurally. For example, an image that is embedded in a Word document is structurally associated with the Word document. Other types of associations are more subtle. Where did the image come from? Was it cropped, resized from the original? There may only be a temporal association between the image in the Word document and the original. But it may be important to traverse this temporal association later. The Goo Trail is attempting to capture the temporal associations between Things in your working environment so you can afford to forget.