Hackathon 2011!

We've still got a little hacking time left in the 2011 Horde Hackathon, but we're about to relax for dinner for a bit, so now is a good time to wrap up some of the things that have been accomplished already:

  • We have a plan for making the Horde UI consistent between all applications and application views, as well as to make it look much better. Also to reduce the number of views we maintain, hopefully increasing our developer productivity
  • Some really snazzy automated updates coming for the demo server
  • The develop branch of nag now supports ajax completion of tasks, creating tasks, and basic viewing of tasks
  • Trean finally has a migration from folders to tags, with basic working functionality for viewing bookmarks with tags. This was on my list FOREVER.
  • Lots of good discussion about how we fit into the larger PHP and groupware ecosystem
  • New timezone support from Jan
  • New weather support to replace the now-commercial weather.com feed
  • New documentation tools

... and almost certainly a ton more that is slipping my mind right now. Going to miss everyone once we head our separate ways again tomorrow, but hopefully the momentum will keep up, and I'm really looking forward to next year!

Improved URL auto-linking in Horde

Horde now uses John Gruber's regex pattern for matching URLs in text. This regex is used in the Horde_Text_Filter_Linkurls class, which already had a solid pattern, but the new one improves it in several ways:

  • Much better support for unicode characters
  • We now auto-link URLs like example.com/foo
  • Support for some limited parentheses matching so that URLs that contain matching parentheses are properly matched, even if a parentheses is the last character in the URL

Horde's pattern differs from the one posted by Gruber in a few ways, as well:

  • It does not match mailto: URLs. This is to better match how Horde has historically separated linking of web addresses from email addresses; mailto: links are handled by Horde_Text_Filter_Emails.
  • It matches URLs that start with ://
  • URL protocols are limited to 20 characters to avoid excessive memory use when PCRE does backtracking
  • "+" is allowed in a protocol (so svn+ssh:// works)

A good-sized list of test data has been incorporated into the Horde_Text_Filter test suite, but let me know if there are kinds of URLs that used to work but now don't, or if you see any other problems with the new auto-linking. Hopefully the new pattern will help polish the experience for those using the new Horde 4 alphas!

Stream wrapper for strings

I've added a StringStream class to the horde/Support package that lets you access a string as a stream, using fgets, fread, fseek, ftell, etc. - all of the standard stream functionality. 

The neat thing about this implementation is that it doesn't duplicate the string, and it also doesn't use global scope to pass the string to the stream wrapper. The first is accomplished through references; not such a huge trick. The second is harder, since there isn't an obvious way to pass arbitrary data to a stream wrapper implementation. You can pass an object to fopen() as the $path, and if it has a __toString() method it will be used - an interesting trick that I'm still looking for a good use for. Unfortunately the string-ified value is what gets passed to stream_open() inside the stream wrapper implementation.

But! You can use the stream context to pass arbitrary parameters. So the $context parameter becomes the vehicle to pass the Horde_Support_StringStream object to the stream methods, which are implemented in Horde_Stream_Wrapper_String.

Here's an example:

$str = 'large data passed as a string';
$stream = new Horde_Support_StringStream($str);

$fp = $stream->fopen();
while (!feof($fp)) {
    $chunk = fread($fp, 1024);
    // Do something chunked
}

Using Horde_Xml_Element to quickly generate XML from arrays

Horde's Xml_Element package, part of the PHP 5 Horde 4 framework, is a SimpleXML-like wrapper around the DOM extension that gives programmers a midpoint between DOM's power and SimpleXML's simplicity. Most of the syntax is just like SimpleXML - access attributes as array offsets and elements with object accessors - but you can also add new nested elements with object syntax and use namespace prefixes in element names. The biggest extra feature, though, is probably the fromArray() method.

Horde_Xml_Element::fromArray lets you very easily convert an array into a Horde_Xml_Element object, which you can then work with or dump directly to XML:
<?php

// To populate an <entry> element:
$e = new Horde_Xml_Element('<entry/>');
$e->fromArray(array(
    'id' => 1,
    'title' => 'Hi neighbor!',
    'author' => array('name' => 'Chuck', 'email' => 'chuck@horde.org'),
));
echo $e->saveXmlFragment();
Produces:
<entry><id>1</id><title>Hi neighbor!</title><author><name>Chuck</name><email>chuck@horde.org</email></author></entry>
With the Horde/Feed package, which uses Horde_Xml_Element internally, this can be simplified further:
<?php

$e = new Horde_Feed_Entry_Atom(array(
    'id' => 1,
    'title' => 'Hi neighbor!',
    'author' => array('name' => 'Chuck', 'email' => 'chuck@horde.org'),
));
echo $e->saveXmlFragment();
Which produces the same output. Horde_Xml_Element will also let you set attributes and namespaces with a simple syntax. "#" denotes an attribute, and ":" separates namespaces:
$e = new Horde_Xml_Element('<atom:feed/>');
$e->registerNamespace('dc', 'http://purl.org/dc/elements/1.1/');
$e->fromArray(array(
    'atom:entry' => array(
        'atom:link' => array(
            '#rel' => 'self',
            '#title' => 'Feed title',
            '#dc:date' => '2011-01-31 01:00:00',
            '#href' => 'http://planet.horde.org/atom/',
        ),
    ),
));
echo $e->saveXmlFragment(true) . "\n";

Which produces:

<atom:feed xmlns:atom="http://www.w3.org/2005/Atom">
  <atom:entry>
    <atom:link xmlns:dc="http://purl.org/dc/elements/1.1/" rel="self" title="Feed title" dc:date="2011-01-31 01:00:00" href="http://planet.horde.org/atom/"/>
  </atom:entry>
</atom:feed>

 

Quick Tasks in Nag, Horde's task manager

Horde recently added a new library: Horde_Date_Parser. It started off as a port of the ruby library Chronic, but was completely revamped to support multiple locales. I also added even more tests and several that don't pass in Chronic pass in Horde_Date_Parser.

To show off the new library, I've added a "quick add" feature to Nag, the Horde task manager. This adds a tasks/quickAdd registry method, that takes a single string. Dates in the string are parsed out as due dates, and multiple tasks can be added by separating them with newlines. Child tasks can also be created with indentation (including * or - lists).

For example:

laundry tomorrow
* buy detergent tonight 

Will create two tasks: a parent task named "laundry" with a due date of tomorrow, and a child task named "buy detergent" with a due date of tonight.

In addition to the API call there's a pretty UI for this:

Gives:

Others have these features of course. However our version handles some cases that some of them don't:

MySQL Conference Wrapup

Absolute highlight of the MySQL conference: being congratulated by Monty Widenius for creative use of merge tables, and receiving a batch of good feedback on our keynote (video).

Speaking at the MySQL Conference

I'm really excited to be participating in the closing keynote of this year's MySQL conference. We'll be talking about our experiences behind the scenes of the Obama compaign. For me and my coworker Leigh Heyman, that means the software and hardware systems behind my.barackobama.com, which took in over $500 million in contributions and sent over 1.3 billion emails.

I also recently spoke with another coworker, Josh King, at the BostonPHP user's group about the various communications systems we ran - Josh about the Neighbor-to-Neighbor tool for canvassing, and myself about email. There are some comments and photos on the meetup page, and a podcast recording on the BostonPHP website.

Using Horde/Controller from the command line

Horde/Controller is a new Horde 4/PHP 5 component that builds on Horde/Routes to provide the "C" in an MVC architecture. It's heavily based on the Maintainable framework, which is in turn heavily based on Rails, so it will look similar if you're familiar with either of those. Both Mad and Rails have the concept of tasks that can be run from the command line; I thought it would be nice to use the same routing and controller architecture to allow command line scripts.

After a little hacking, the Horde_Controller package in git now has Cli request and response objects and this script is all you need to give command-line access to a set of controllers:

#!/usr/bin/env php
<?php

require 'Horde/Autoloader.php';

// Set up our request and routing objects
$request = new Horde_Controller_Request_Cli();
$response = new Horde_Controller_Response_Cli();
$mapper = new Horde_Routes_Mapper();
$mapper->connect(':controller/:action');

// Create our controller context.
$context = array(
    'mapper' => $mapper,
    'controllerDir' => dirname(__FILE__) . '/../lib/controllers',
);

// Dispatch.
try {
    $dispatcher = Horde_Controller_Dispatcher::singleton($context);
    $dispatcher->dispatch($request, $response);
} catch (Exception $e) {
    var_dump($e->getMessage());
}

Error handling can obviously be improved, and your controllers need to produce plaintext output for this to be friendly, but it gives you the flexibility to add new commands to your script just by dropping in new controller classes to the controllerDir.

Arguments are parsed with the Horde/Argv package, using a new allowUnknownArgs parameter that auto-creates simple text values for options unknown to the parser. This lets you take arbitrary command line arguments in your controller without having to tell the dispatcher script what arguments you expect ahead of time (of course the trade off is that the dispatcher can't validate the arguments, so the controllers are responsible for that, just like they are with web requests).

Here's an example of invoking the index() method of a MigrateController class:

$ ./dispatch.php migrate

To invoke the currentVersion() method of the same controller:

$ ./dispatch.php migrate/current_version

To pass an argument named target_version with the value "11" to the controller's index() method:

$ ./dispatch.php migrate --target-version=11

It's still a work in progress, but I'm happy with the possibilities.

Reasons to host your own groupware

With the proliferation of hosted services and the cloud (5 yard penalty, unlicensed buzzword use, try again next post), a case has been made that people, and companies, shouldn't host things like their own email anymore. I've always been a proponent of keeping your own data, and privacy and data portability issues aside - sometimes services disappear. I don't mean GMail dropping off the face of the net for an hour here or there, but a service like I Want Sandy simply shutting down. This is of course an argument both for controlling your own data and for open source software. It's articulated well here:

http://itdied.com/2008/12/in-defense-of-shutdowns.html

The calculations change if you're paying for the service; at that point you have some level of commitment. But for free services? Keep backups.

Git support in Horde Chora

Horde's version control browser, Chora now has initial git support. All three Horde git repositories (some more info available at http://horde.org/source/git.php) are browseable through the repository menu in the upper right at http://dev.horde.org/h/chora/.

Chora provides a traditional tree-based view of the repository, with histories available in each individual file. We'll definitely have commit-based information (as we already support changesets in CVS and patchsets via cvsps), and we're looking at having more of a git-centric history view as well. But if you need to interface with your repository through PHP code, or if you're used to a primarily tree-based version control browser, or if you already use Chora or Horde, you might find this useful.