Quick Tasks in Nag, Horde's task manager

Horde recently added a new library: Horde_Date_Parser. It started off as a port of the ruby library Chronic, but was completely revamped to support multiple locales. I also added even more tests and several that don't pass in Chronic pass in Horde_Date_Parser.

To show off the new library, I've added a "quick add" feature to Nag, the Horde task manager. This adds a tasks/quickAdd registry method, that takes a single string. Dates in the string are parsed out as due dates, and multiple tasks can be added by separating them with newlines. Child tasks can also be created with indentation (including * or - lists).

For example:

laundry tomorrow
* buy detergent tonight 

Will create two tasks: a parent task named "laundry" with a due date of tomorrow, and a child task named "buy detergent" with a due date of tonight.

In addition to the API call there's a pretty UI for this:

Gives:

Others have these features of course. However our version handles some cases that some of them don't:

MySQL Conference Wrapup

Absolute highlight of the MySQL conference: being congratulated by Monty Widenius for creative use of merge tables, and receiving a batch of good feedback on our keynote (video).

Speaking at the MySQL Conference

I'm really excited to be participating in the closing keynote of this year's MySQL conference. We'll be talking about our experiences behind the scenes of the Obama compaign. For me and my coworker Leigh Heyman, that means the software and hardware systems behind my.barackobama.com, which took in over $500 million in contributions and sent over 1.3 billion emails.

I also recently spoke with another coworker, Josh King, at the BostonPHP user's group about the various communications systems we ran - Josh about the Neighbor-to-Neighbor tool for canvassing, and myself about email. There are some comments and photos on the meetup page, and a podcast recording on the BostonPHP website.

Using Horde/Controller from the command line

Horde/Controller is a new Horde 4/PHP 5 component that builds on Horde/Routes to provide the "C" in an MVC architecture. It's heavily based on the Maintainable framework, which is in turn heavily based on Rails, so it will look similar if you're familiar with either of those. Both Mad and Rails have the concept of tasks that can be run from the command line; I thought it would be nice to use the same routing and controller architecture to allow command line scripts.

After a little hacking, the Horde_Controller package in git now has Cli request and response objects and this script is all you need to give command-line access to a set of controllers:

#!/usr/bin/env php
<?php

require 'Horde/Autoloader.php';

// Set up our request and routing objects
$request = new Horde_Controller_Request_Cli();
$response = new Horde_Controller_Response_Cli();
$mapper = new Horde_Routes_Mapper();
$mapper->connect(':controller/:action');

// Create our controller context.
$context = array(
    'mapper' => $mapper,
    'controllerDir' => dirname(__FILE__) . '/../lib/controllers',
);

// Dispatch.
try {
    $dispatcher = Horde_Controller_Dispatcher::singleton($context);
    $dispatcher->dispatch($request, $response);
} catch (Exception $e) {
    var_dump($e->getMessage());
}

Error handling can obviously be improved, and your controllers need to produce plaintext output for this to be friendly, but it gives you the flexibility to add new commands to your script just by dropping in new controller classes to the controllerDir.

Arguments are parsed with the Horde/Argv package, using a new allowUnknownArgs parameter that auto-creates simple text values for options unknown to the parser. This lets you take arbitrary command line arguments in your controller without having to tell the dispatcher script what arguments you expect ahead of time (of course the trade off is that the dispatcher can't validate the arguments, so the controllers are responsible for that, just like they are with web requests).

Here's an example of invoking the index() method of a MigrateController class:

$ ./dispatch.php migrate

To invoke the currentVersion() method of the same controller:

$ ./dispatch.php migrate/current_version

To pass an argument named target_version with the value "11" to the controller's index() method:

$ ./dispatch.php migrate --target-version=11

It's still a work in progress, but I'm happy with the possibilities.

Reasons to host your own groupware

With the proliferation of hosted services and the cloud (5 yard penalty, unlicensed buzzword use, try again next post), a case has been made that people, and companies, shouldn't host things like their own email anymore. I've always been a proponent of keeping your own data, and privacy and data portability issues aside - sometimes services disappear. I don't mean GMail dropping off the face of the net for an hour here or there, but a service like I Want Sandy simply shutting down. This is of course an argument both for controlling your own data and for open source software. It's articulated well here:

http://itdied.com/2008/12/in-defense-of-shutdowns.html

The calculations change if you're paying for the service; at that point you have some level of commitment. But for free services? Keep backups.

Git support in Horde Chora

Horde's version control browser, Chora now has initial git support. All three Horde git repositories (some more info available at http://horde.org/source/git.php) are browseable through the repository menu in the upper right at http://dev.horde.org/h/chora/.

Chora provides a traditional tree-based view of the repository, with histories available in each individual file. We'll definitely have commit-based information (as we already support changesets in CVS and patchsets via cvsps), and we're looking at having more of a git-centric history view as well. But if you need to interface with your repository through PHP code, or if you're used to a primarily tree-based version control browser, or if you already use Chora or Horde, you might find this useful.

November Horde Board Meeting Summary

The November board meeting, already over a week ago (there was also this little election here in the U.S. that occupied a wee bit of my time), was quite productive. I'll give the link to the full log up front; I'm just going to talk about the process of moving libraries over to both Horde 4 code and to the new git repositories. Porting framework packages over is our current Horde 4 priority (over new applications, but not over Horde 3.3 application releases), and the process will be:

  1. Write down the new coding standards for Horde 4 libraries. These have been developing somewhat organically, but they need to be codified - what is the package layout, what capitalization, what PHP 5 features to emphasize, what not to use, etc.
  2. Focus on converting our existing libraries. Libraries will be moved to the main git repository when:
    1. The code has been reviewed, updated for the new coding standards, potentially refactored
    2. Unnecessary dependencies have been eliminated
    3. Wherever possible they should work without assuming a Horde configuration, $registry instance, etc.
    4. There should be at least basic unit tests
  3. When a library is moved to git, it will be deleted from CVS HEAD
  4. Applications will be tweaked (in CVS) to work with the newly refactored Horde 4 packages

Some libraries (Autoloader, Log, Support, ...) are already in git as they are already Horde 4 code; several others such as Db, Feed, Argv, ... will join them soon in the move from CVS. The rest - all 80-some other libraries in the current Horde Framework - will be converted over next.

October Horde Board Meeting Summary

The October Horde Board meeting was this past Tuesday, the 7th. It was one of our more decisive meetings, with decisions to:

  • Move to git for future (Horde 4+) version control. Code will be pushed to a clean git repository as it is modified for Horde 4 and PHP 5. Git pushes will generate email to a new list, commits@lists.horde.org. Everything currently in CVS will remain available, and Horde 3.x will continue to be developed in and released from the existing CVS structure.
  • Aim for shorter release cycles of major versions. When backwards compatibility is broken we will increment the major version number (Horde 3 to Horde 4) as we do now; we'll just do that more often. Feedback indicates that being able to do feature upgrades of an application but not Horde, or vice versa, is not a big advantage for most people.
  • Use individual package versions much more extensively with Horde 4. We want it to be possible to release an application upgrade that relies on new framework functionality without requiring a new major version; instead, many changes will be possible by just requiring a specific package version.
  • Finally, and most importantly, we will be focusing on getting the last Horde 3.3 releases out (see http://wiki.horde.org/ReleaseManagement), and then shifting our major focus to Horde 4. Our goal is to have an initial release of Horde 4.0 within 6 months of starting serious development on it; and to aim for major version releases approximately every year thereafter.

The full minutes are available in the board list archives; thanks to Ben Klang for running the meeting, posting the minutes, and doing the initial summarization.

Distributing resources with Horde_Support_ConsistentHash

Let's say you've just read Yahoo!'s Best Practices for Speeding Up Your Web Site for the first time, and you want to tackle some easy ones first: serve your static assets (javascripts, css stylesheets, and images) from multiple servers. Alternately, if you've read the Yahoo! rules a bit more closely, you might not be worrying too much about assets due to the empty browser caches most of your users have, so what you really want to do is distribute cached user data across a set of cache servers.

We're going to skip the details of why the simplistic ways to do this are a bad idea, and go right to consistent hashing.

If you're not familiar with it already, you should read up to understand what it solves and how it works. Once you have that covered, here's how you can do it using Horde_Support_ConsistentHash, part of the recently released horde/support package.

For our example, you have three cache servers; the third one is much more powerful so you'd like it to handle the bulk of users. You're going to distribute cached user data across them based on username. Easy does it:

$h = new Horde_Support_ConsistentHash;
$h->add('cache1.example.com', 1);
$h->add('cache2.example.com', 1);
$h->add('cache3.example.com', 4);
$cacheServer = $h->get($username);

This has all kinds of nice properties: you can bring up new servers gradually by increasing their weight (just read the list of nodes from a database or configuration file), and doing so - or taking a bad server out of the pool - won't cause most users on an unaffected server to be switched, reducing cache thrashing and probably saving your butt if you really have as much traffic as you think you do.

Horde_Support_ConsistentHash has no external dependencies (even on other Horde_Support files) and can be installed with an easy:

pear channel-discover pear.horde.org
pear install horde/support-beta

Finally, if your cache backend is memcache, the PECL memcache extension can take care of this for you by setting memcache.cache_strategy = "consistent". Obviously the Horde implementation can help you out in far more situations than just memcache, or if you're stuck with an older memcache package.

Horde_Oauth released

As promised in my what's-coming summary, initial Oauth support has been added to Horde. The 0.1.0 release is a standalone PEAR package installable from pear.horde.org; the only Horde-related dependency is the horde/Http_Client package. You can install both easily:

pear install horde/oauth-beta

PHP 5 is required (5.2+ is recommended, earlier is your own ball of wax), and openssl is needed to use the RSA-SHA1 signature method (required if you want to use this with Google's Data APIs; other sites support HMAC-SHA1, and the package includes both along with Plaintext for completeness. The code is based on the example OAuth.net PHP code, but is refactored, split into multiple files, packaged, made Horde-friendly, etc. It is released under a BSD license.

Still to come:

  • Integrated consumer support for the Horde UI, with a Prefs container for managing the access tokens that a user has for external sites, a callback script, and a UI that knows how to request access to external sites.
  • OAuth server support, both standalone in the package to whatever extent makes sense, and integrated into Horde.
  • A UI for managing access tokens granted by Horde users to third parties.