51 Elliot

at December 07, 2008

Cross Country Skiing, Gatineau Park, Quebec

at December 05, 2008

Staghorn Sumac, Gatineau Park, Quebec

at November 23, 2008

Finally, Aura is covered up for winter. Good thing as it's supposed to snow heavily this week. Friends Liam and Paul helped me out and tried a new technique of rolling the front of the tarp and "stitching" it just with strands of rope. It's really effective.

What a brutally cold day to be out, though. There is already about two inches of ice in the harbor. While we were working on getting Liam's boat covered, we started hearing this hacking sound. It turned out to be one of the maintenance staff, trying to hack an aluminum work boat out of the water with an oar. The whole thing, including a big 30Hp Mercury motor were frozen solid into the ice. We had a good laugh over the silliness of it all... forgetting to take the boat out in the first place, the futile hacking with the oar, and the blasted freezing cold. At least there was somebody out there more miserable than us.

A few minutes later we started hearing a huge crunching sound. I ran to the harbor wall and looked out towards the entrance. A yellow sailboat was making it's way in like a Russian icebreaker at the north pole. The bow was riding up, the motor was gunned wide open, and chunks of ice were flying all over port and starboard. I ran back and told Liam and Paul, "You've gotta see this!" We all just stood there mouths agape, until Paul got his cellphone out and snapped a couple of pictures. Right about then they got hopelessly stuck, and we were practically rolling on the ground laughing. For some reason, being out in the bitter cold made watching these shenanigans seem really hilarious. We got back to work on Liam's boat, taking care because he didn't want the metal grommets on the tarp to scratch his hull. We had to undo a few of the knots till it met with his satisfaction. When we finally finished, the lads in the harbor had somehow managed to force that poor yellow sailboat up to the crane, which they were now busy manipulating into position for a haul-out. A jagged trail of broken ice streamed off behind them.

"Forget about grommets", I told Liam, "If you're concerned about scratching your hull, those are just the two fellows you ought to call!"

There was a roaring fire in the fireplace when we got to the clubhouse, and dinner was still being served. It hit the spot nicely.

at November 12, 2008

The end of the sailing season has come, and Aura is now high and dry, on the hard, laid up for the winter. She still has to be covered over and the motor prepped and stored, but the big work of hauling out is done.

It was a good season of sailing. With her new sails, Aura looked like a new boat and performed well. A used #2 genoa given to me by fellow Tanzer 22 sailor John Lydon gave us some enormous sail area for light wind races and was a deciding factor in us crossing the finish line ahead of some others.

One crisp and sunny October afternoon as I finished putting new pads on the boat cradle, enjoying all the sawing, hammering, and general puttering around in the harbor, I reflected on the hours of behind-the-scenes work that go into owning a sailboat. My crew this year (on weekly PHRF and JAM races) were good sports, fun to sail with, and did a great job for being relatively new to racing. It did turn out to be a bit of a chore to get everyone to show up consistently though, and I've realized that to most people, sailing is seen a relaxing way to spend a couple of hours, not a competitive sport. When you've spent countless hours working on a boat doing a hundred little things that have to be done, there's more of you in the boat, and you're more dedicated to it. That's probably why my best crew member is also the owner of an Aloha 28 and a dedicated sailor. For the rest, the novelty wore off about halfway through the season, and we withdrew from the Fall series of races.

But there's a certain enjoyment in just being around boats. The environment is good to be in. There's the wind, and the water, the promise of the open river and places to explore. And sailing up the river under the bright white canvas, feeling the forces of nature and the laws of physics working to propel you effortlessly forward... it's all very different from motoring along in a machine. It instills a sense of wonder.

As I wrapped up my work on the boat cradle that day and headed back to the club house for a hot drink in the Dragon bar, a chilling breeze blew in across Lac Duschenes. A lone sloop was tacking upwind for one last run into the setting sun. I shivered and ducked inside where the fireplace was blazing warmly. It's a bit sad to see the end of the season come so soon, but I don't think I'd want to be out sailing in the biting November wind. Better tuck away the memories of hot Sunday afternoon cruises and picnics on the water for another year.

Now it is time to prepare for a long, cold winter.

at October 13, 2008

Meech Lake, Gatineau Park, Quebec.

at October 13, 2008

Cat-tails beside the trail, Gatineau Park, Quebec.

at October 13, 2008

Lac Meech, Parc de la Gatineau, Quebec.

at September 30, 2008

at September 03, 2008

Mont Blanc du Tacul, Mont Maudit and Mont Blanc from the Aguille du Midi, Chamonix, France.

at September 03, 2008

Cafe, Como, Italy.

at September 03, 2008

A peaceful sunset, Lago di Como, Italy.

at September 03, 2008

A train passes through Val Bernina near Pontressina, Switzerland.

at September 02, 2008

Iain Williams climbing the South Ridge of Piz Balzet in Albigna, Switzerland.

at September 02, 2008

Dusk from the Chamanna d'Escha.

at September 02, 2008

Climbers descend from the Aguille du Midi, Chamonix, France.

at September 02, 2008

Crossing the upper Vadret de Portabella below the summit of Piz Kesch in the Rhaetian alps.

at September 02, 2008

Original 14th Century panels on the Kapellbrücke, Lucerne, Switzerland.

©2oo8 Photo by Darren DeRidder

at August 29, 2008

Guests enjoy a pleasant evening at the Albigna Hut, Switzerland.

at August 22, 2008

The famous Biancograt, Piz Bernina.

at August 22, 2008

Chocolate, St. Moritz, Switzerland.

at August 22, 2008

Diavolezza and the Pers glacier from Bovalhutte, Morteratsch in the Engadine.

at August 22, 2008

Shores of Lake Como, Varenna, Italy.

at August 22, 2008

Hotel Victoria, Varenna, Italy.

at August 22, 2008

Cairn construction in the Albigna valley, Switzerland.

at August 22, 2008

Slab climbing above the water, Albigna, Switzerland.

at August 22, 2008

A window box in St. Moritz, Switzerland.

at August 22, 2008

Mist hangs above hills in Chamonix, France.

at July 07, 2008

This week, amid news - just in time for the 4th of July - that a Federal US judge has ordered Google to turn over the confidential user information of every single user who has ever watched a Youtube video to Viacom, the giant media conglomerate who is suing Google over some TV clips that end-users uploaded, my thoughts turned again to the issue of privacy, and freedom, how the two are inextricably related, and how poorly this seems to be understood by many people who consider themselves citizens of a free country.

Viacom, by the way, are the brilliant folks who stole a Youtube user's video without permission, aired it on TV, and then filed a DMCA takedown notice against said user, for infringing on "their" copyright.

While the invading horde of Viacommies is bad enough, it pales in comparison to the threats posed by warrantless wiretapping programs and their ilk (including the deceptively-named Patriot Act). It's becoming more and more apparent that we all need to take a careful look at privacy, and why it is essential to freedom and democracy.

There is an argument that goes, "If you're a good citizen then you shouldn't have anything to hide; if you don't have anything to hide then you shouldn't be worried about privacy."

This is dysfunctional thinking.

The right to privacy should be protected because it is an essential pillar of freedom and democracy. Without privacy, there is no such thing as freedom. And, as Amy Teimman wrote: "Let's face it: just because we have nothing to hide doesn't mean we want to have our lives uploaded to government servers."

"...the absence of privacy has the ability to influence"

I'm not talking about someone planning to build bombs, share music, import French wine, pick their nose in front of Congress, or other equally nefarious crimes. I'm not talking about the concern people have for keeping their credit card information safe online, concealing their web-browsing activity, or protecting their kids from an online predator. Those are valid concerns, too, but there is an even more important over-arching issue at stake...

If your private thoughts were broadcast on a television screen above your head for everyone to see, its fair to say we'd all be more careful what we thought about. We would be forced to practice thought control to give others a good impression. Mental self-discipline is great, but this imaginary scenario illustrates a critical point: the absence of privacy has the ability to influence.

Moreover, the one who usurps privacy is the one who wields that influence. And such control at the cost of privacy is the opposite of freedom. Democracy and freedom are upheld by the individual's right to form thought and opinion and to communicate within the haven of privacy.

You should have the right to speak to or email a friend privately. You should have the right to discuss political views privately. You should have the right to pray and gather for religious worship privately. You should have the right to check out a book from the library without a government agency tracking your selection. You should be able to conduct online research, reading and communication, privately.

When you have lost these, you have lost your freedom. We're in a time where freedoms and privacy are being usurped in the name of security, and as Benjamin Franklin said, "He who sacrifices freedom for security deserves neither."

Your privacy is your freedom.

Image by xark.typepad.com

at July 03, 2008

The news of the passing of Esbjorn Svensson on June 14th came as a very sad shock to me and many others around the world who became familiar with the extraordinary musical creativity of the Esbjorn Svensson Trio (EST).

EST quickly became one of my favorite musical groups when I special-ordered and picked up their album "Strange Place For Snow" after having it recommended by a German friend with notably fine taste in European jazz.

Their music was energetic, buoyant, fresh and hauntingly beautiful. The loss of a rising jazz leader so early in his career is really a tragedy. I encourage those with a musical ear to check out EST.

at June 27, 2008

Here's how to make a bootable USB flash drive (thumb drive) on Windows XP without using a floppy disk.

Download Virtual Floppy 2.0.

It should create a virtual floppy called drive b. Open up My Computer, right click on drive b, and select Format. Select the option to create an MS DOS startup disk.

Download mkbt.

Open a dos prompt (run cmd) and cd to the mkbt directory, wherever you put it, i.e.
> cd mkbt/mkbt20/

Copy the bootsector from b: to a file:
> ./mkbt -c b: bootsect.bin

Format the flash drive to FAT 16. In My Computer, right click on the Flash drive (mine was F:) and select format. Use FAT, not FAT 32.

From the dos prompt, copy the bootsector image to the flash drive:
> ./mkbt -x bootsect.bin F:

Make sure you get the drive letters right. Finally, copy all the files from b: to f:.

The flash drive should be bootable media now, with plenty of space for extra files you might need for doing BIOS upgrades and such.

at June 12, 2008

Update: When this post came out, a few zealous RETE proponents had a field day bashing some of the ideas I presented here and referring to my dismissal of RETE (as a panacea for policy control) as "mental laziness". Since then, other people have come to similar conclusions, including Martin Fowler. In his article RulesEngine he states, "there's a lot to be said for avoiding more general rules systems", and advocates building simple custom, domain-specific rules systems. I happen to agree with Martin Fowler on this point.

Here is how to build a high-performance rules-based policy server in less than two hours using proven, free open-source software that can outperform incumbent rules-engine products by up to two orders of magnitude in common scenarios of business logic and policy. In addition it is easier to use and more secure than its commercial counterparts.

All of the information in this article is an established part of what we in the engineering field refer to as "the literature" - the large base of common knowledge encompassed by published resources - and is taken from free, publicly available sources. Links to online references are included.

Rules engines are found in various forms and may be alternately referred to as an expert system, business process management system, a policy decision function, policy manager, or business rules engine. These network components make decisions based on rules that are programmed by administrators. Rules engines are all over the place. Examples include Service Oriented Architectures (SOA), workflow management systems, and logical components of many large scale Internet applications. They're used in the financial industry, where they decide such things as whether or not you qualify for a new credit card, or what your maximum mortgage should be. They are also an important part of Internet and telecommunications applications.

Many kinds of applications don't require transactions to be processed in real-time. It doesn't matter if there's some latency on the request, or in some cases, if it takes hours to generate a report. But high-traffic Internet and telecommunications applications need to support millions of users and thousands of transactions per second with a turn-around time measured in milliseconds.

A friend of mine described rules engines as "web-time" applications at best, where page-load times of several seconds are typical. They tend not to scale well to multiple thousands of transactions per second with milliseconds latency. Trying to use such a system to provide such functionality may be a case of using the wrong tool for the job.

Certain rules engines claim to be far better at processing rule sets than naive implementations, by using a RETE algorithm. RETE is an development of the branch of computer science known as Artificial Intelligence. It was originally developed as a solution for processing the very large sets of facts and rules generated in machine-learning algorithms and is used especially in forward-chaining inference engines.

Let's summarize what RETE advocates mean by a naive implementation and why they claim to be "often several orders of magnitude better". In part, it relates to a simplified logical structure, illustrated in the following example:

Example 1:
IF A and B do X
ELSE IF A and C do Y

Example 2:
IF A {
IF B do X
ELSE IF C do Y
}

The second example avoids double-checking condition "A". Likewise, a RETE algorithm takes a collection of rules and builds a network of nodes that eliminates redundancy in the logic. It also remembers the results of evaluating criteria, and allows them to be re-evaluated if the facts change. Admittedly, this example is overly simplified, but it gives you a general idea.

Programmer optimize their control flow in a similar way. In a machine-learning environment where data sets grow very large, RETE is able to sort through the thousands of rules more efficiently than a human programmer. However, many kinds of applications have policies and rules that are fairly straightforward and don't involve thousands of machine-learned factoids.

This article by Loic Tregan of eBay from the W3C Workshop on Rule Languages for Interoperability questions the usefulness of RETE with greater scrutiny. To quote a portion of the article:

"Using two industrial implementations of the RETE and a sequential engine, both coming from the same vendor, we found a x10 factor degradation in performance, start-up time and used memory. Do we really need RETE ? ...We believe many transactional, stateless systems are best suited with simple sequential rules processing that do not require forward-chaining.

Tom Debevoise, author of "Business Process Management With a Business Rules Approach: Implementing the Service Oriented Architecture", wrote this article to suggest that RETE is not an appropriate choice for the majority of business rules:

"A good business rule mining team will strive to build concise, self-contained logic in the statements. Unless, they are building a diagnostic or expert system, the outcome is usually a small ruleset for each business area.

The article goes on to describe how well-written business rules tend to adhere to some common principles that make them less likely candidates for RETE processing.

It turns out that modern day compilers are also well-optimized tools for evaluating conditions, and they do a pretty good job of it. Modern compilers can take a run-on if-else clause or a poorly implemented control structure and simplify it down to a fairly streamlined set of instructions.

So why not use compiler optimizations to streamline the evaluation of conditional program flows such as rule sets? Mainly because you don't want to have to write your rules in C and compile them.

A scripting language, on the other hand, doesn't need to be compiled first. Most rule sets are best expressed as simple scripts, anyway. Some rules engines refer to their rules as "scripts", although they may have a much steeper learning curve than most scripting languages . Indeed, it has been said that expert systems "qualify as programming languages, although certainly with a narrower range of application than most programming languages"[1].

So why not implement rules in a scripting language? Well, there is a perception that scripts are slow. Unfortunately, it's not really an accurate perception. There are some circumstances where scripts can perform very well. We need to take a look at what happens when you run a script.

Most of the time, when you run a script, it gets run through an interpreter that turns your script into byte code. This phase is called compiling (scripts do get compiled at runtime). Next, the byte code is executed by the script engine, be it Spidermonkey, Zend, Perl or what have you. The next time you run the script, the whole process starts over again. And this process of converting the script into bytecode is where the real performance pinch can begin to be felt.

There is however, a nice solution to this problem called a bytecode cache. What it does is it holds onto the bytecode that the interpreter produces, and keeps it on hand for the next time the script runs, instead of compiling it all over again. This saves a lot of time.

A server, however it is implemented, must communicate with other network components via some protocol. In an IP network, solutions like XML-RPC and SOAP have become popular choices. Either one is better than deciphering bytes when you're trying to get things working.

The basic requirements so far consist of Operating System, HTTP Server, Scripting Language, and Database, which sounds a lot like like LAMP. The basic Linux-Apache-MySql-PHP combo in any of its themes and variations will do. LAMP doesn't have commercial backing per se, but it is a robust, secure and proven technology with a vast market share. It has been leveraged to provide carrier-grade, highly scalable services in an IP network.

With this proposal in mind, let's proceed to take a look at the installation and configuration of the components we need to build and test our rules-based policy engine.

Here are the basic components you can use to build a prototype. You might recognize that the same configuration is used in hundreds of Internet applications from Amazon to Facebook to... Zembly. It's the ubiquitous Linux, Apache, mySQL, and PHP combination and the popular APC cache. Many top-tier web applications use this combination to execute policy functions, and application logic. Much work in Service Oriented Architectures (SOA) and Software as a Service (SaaS) has made use of this approach [PDF article, IT Professional, January 2007].

For deployment, the ideal platform is a multi-CPU dedicated server, but for experimenting, a LAMP stack running on a VMware virtual server is a good option. You can use VMware Server or VMware Player to create a virtual machine. Mac users can use VMware Fusion. I won't go into the details of installing these products and setting up a virtual server, as the product documentation itself is pretty good.

VMware Fusion - Setting up Ubuntu jeOS from an iso disk image

I chose to use jeos (pronounced "juice") from Ubuntu. Jeos is an edition of Ubuntu server that's been trimmed down especially for use on virtual machines. It comes with a minimal set of applications installed, so if your not comfortable with configuring a Linux server or prefer a graphical desktop environment, you may wish to install Ubuntu Desktop instead. For production I'd recommend Ubuntu Server or CentOS. Again, the installation of the operating system is well documented on the Ubuntu website, so please follow the instructions there.

Ubuntu jeOS Installation

Apache, PHP and MySQL can be installed from the command line with this one-liner:
> tasksel install lamp-server

You may wish to install Webmin, a web-based server administration tool. It's not necessary, but it's useful for server administration. The instructions for installing on Debian / Ubuntu can be found here.

A screenshot of Webmin's System Logs configuration

Next, install phpMyAdmin, a web based MySql administration tool.
> apt-get install phpmyadmin

Visit the phpMyAdmin web site for installation and configuration instructions. You should be able to launch a browser on your host operating system and connect to the the Apache web server on the virtual machine. Run 'ifconfig' on the VM to find it's IP address, then connect to http://[ip address]/phpmyadmin. If this doesn't work for you straight away, see http://www.jumpbox.com/node/816 for troubleshooting tips.

phpMyAdmin Screen

The last piece of software to set up is APC, the Alternative PHP Cache. Prior to installing APC, you may need to install the following as well:
> apt-get install make
> apt-get install php5-dev
> apt-get install apache2-threaded-dev
> apt-get install php-pear

Now you should be able to fetch APC using PECL:
> pecl install APC
> vi /etc/php5/apache2/php.ini

Add the following lines to the php.ini file:
extension=apc.so
apc.shm_size = 32

Copy the APC user interface page to your docroot:
> cp /usr/share/php/apc.php /var/www

Additional info on installing APC:
http://www.linuxweblog.com/node/576
http://www.vbulletin.com/forum/archive/index.php/t-165367-p-2.html

At this point you should be able to point the web browser on your host system to the webserver running on your VM and connect to:
http://[vm's IP address]/apc.php

You should see the GUI for APC at that URL.

APC "Alternative PHP Cache" GUI

Setting up all of this is the hardest part of the whole affair. Once you have the environment set up, the rest will be a breeze (I promise)!

LAMP is an ideal solution for providing policy-based decision-making functions for network applications, and in fact, is already used in this capacity in a number of massively multi-user web applications. A key component to providing the required speed and scalability is a byte-code cache, such as the perennial favorite of LAMP developers: the Alternative Bytecode Cache. When dealing with the typical set of policies for many kinds of applications, this approach has benefits over some well-known rules engines that can include better performance, ease of use, and speed of implementation.

Now that the basic requirements are in place, let's review the system.

A Complete System

The first point may come as a surprise: if you've set up your LAMP server and APC cache, you've already got your rules engine. You can start using it (almost) right away. A LAMP stack is many things, and one of the things it is, is a platform to evaluate application logic, including business rules, policies, etc. There will be a few improvements and features to add - nothing difficult - but in its most basic form, the whole thing is ready to roll.

File-based, Scripted Rules

APC works with files. The rules that we write will be stored as files in Apache's docroot directory, and they'll be accessed via HTTP like any other web application. Personally, I like the idea that the rules are files. They're like config files, and it makes it really easy to have an htaccess-protected staging area and another live-deployment area (firewalled, private network segment, password protected, etc).

A CGI-Like Interface

Once you write a script and put it in the web-server's document root, it's accessible over HTTP (via a web browser, wget, curl, a telnet session, the Firefox RESTClient plugin, or a custom web-client). To Apache, the scripts are just another web page written with PHP. They're very similar to CGI scripts (in fact, they are CGIs because they conform to the Common Gateway Interface specification) and will output some kind of text (like XML, for example). You may want some other output format, but that part of it is trivial.

Testing via a Form

The rules/scripts can be tested through a simple web page with an HTML form and a POST query. A custom-built client would just open a socket and write out the appropriately formatted bytes. There's an elegant simplicity about the attribute-value pairs submitted in a POST query, but alternatives like XML-RPC are also nice for readability. SimpleXML, included in PHP, is a handy way to parse it.

Caching Benefits

APC cache transparently caches the bytecode of your scripts. You'll be able to see this using the APC GUI, and it will provide a dramatic speed-up over the per-query parsing and compiling required when there is no caching.

Parameter Caching

Your script is likely to use other criteria that may be stored in a database or come from system calls (like date and time) or other external sources. Rather than query these every time, APC cache can also be used to store variables for use in consecutive executions of your bytecode. APC cache allows you to store variables in simple attribute-value-pairs, with a duration to indicate how long it should be cached for. Once it expires, it'll be removed from the cache and the variable can be re-fetched. This is great for caching variables that don't change very often. For example, you might cache information about the weather and only update it every 5 minutes. If you get 1,000 queries in the meantime, you've just saved 999 queries to your weather server.

Additional features, like the ability to group rules together into rule-sets, and a user interface, are both trivial to do in the LAMP environment.

You may also want to optimize your installation of mySQL to enable query caching. This can offer some noticeable improvement when the same queries are generated frequently.

Getting Over the "Scripting" Stigma

I realizing there are many computer experts out there who enjoy criticizing PHP almost as much as they enjoy denouncing Microsoft, so I'd like to share something I picked up from the world of music. In my experience, people who denigrate a certain style of music are typically not musicians themselves. I've worked with some very talented musicians over the years, and I've always found the best ones are also the most gracious, and have a vast palette of musical appreciation. Similarly, in programming, the best programmers I've known have an appreciation for a variety of languages and technologies, recognize that there's a time and a place for each one, and are not be too hasty to condemn one or the other. Sometimes, the criticism is just due to a lack of familiarity. I hope that this will encourage readers and software developers working in the area of business process managment, policy and application servers, rules engines, web services, the Internet and telecommunications to explore the possibilities offered by this elegant technology.

at June 01, 2008

The last canning plant in Canada to use Ontario-grown fruit is closing. What this means is that as of this summer, you will no longer be able to buy canned fruit that was grown in eastern Canada. CanGro, the multi-national corporation which owns the facility in St. Davids near Niagara Falls, has decided it is more profitable to move the entire operation to China, and will begin shipping Chinese-grown fruit to the Canadian market under their Del Monte brand.

This puts about 200 farmers in the Niagara region out of business, with a production of over 7000 tons of fruit annually. They no longer have a market for their produce. Hoping to save the industry in Canada, one farmer offered to purchase the CanGro factory if the fruit-canning contract would be returned to the local plant. He offered to put up $5 million dollars, but needed the Canadian government to match that amount to make the deal. The plant owners agreed to the deal. They agreed to give back the canning contract and met with the local farmers and government representatives. The Canadian government only had to pony up $5 million, and the Ontario fruit industry would be back in business.

But the Canadian government already had a proposal of their own: they were offering $30 million dollars for the farmers to rip up their fruit-bearing trees and burn them! I'm not making this up.

The deadline for the cannery purchase approached, and on the final day, there was still no reply from the Ontario's Minister of Economic Development and Trade, Sandra Pupatello. She was on a plane, on her way to China to attend meetings on deepening trade agreements with the Chinese. And much of the equipment from the plant had already been removed and was on its way to China to be used in the operation over there.

It would have cost $5 million for the government to help buy the plant and save the Ontario fruit industry, which in turn would pump a lot back into the Canadian economy, i.e. "economic development". Or they could pay $30 million dollars to rip the fruit industry out of Ontario by the roots, give it to China, and wipe out any possibility that the industry could recover in Canada, destroying thousands of acres of mature fruit trees in the midst of a world-wide food shortage.

They chose to spend $30 million tax-payer dollars to ruin Canada's fruit industry.

Sanda Pupatello, Ontario Minister of Economic Development and Trade, came on CBC radio, trying to offer an explanation. She tried to say "The door isn't closed" on the Ontario fruit industry, that the government was "very interested" in finding ways to keep the industry in Canada, and that they were continuing to explore various options. She said that the deal to purchase the CanGro factory wasn't appropriate because the government felt they needed more consultation with "a broader spectrum" of representatives from the business sector and other interested parties. She indicated that the business plan wasn't as thorough as they wanted it to be.

Farmers are business people. The large multi-national corporate "business sector" like Del Monte and CanGro care more about the profit margins they can realize by sending it all overseas. Who's input should we really be listening to?

at May 26, 2008

Racing begins! And we need a motivated crew. Hence the new flag:

at May 19, 2008

This spring I undertook what could be considered a rite of passage among sailboat owners: refinishing the keel. The steps (shown in the video below) were:

scraping down the keel with paint scrapers (didn't work very well)
grinding the keel with an evil-looking twisted wire brush attachment
grinding the keel some more with a rough fiber sanding disk
more wire brush grinding
wash-down with Metal-Ready (phosphoric acid)
3 coats of POR-15 anti-rust paint
1 coat of Interprotect 2000
Epoxy fairing compound in hull-keel joint and pitted areas
4 more coats of Interprotect 2000
VC-17m on anti-fouling bottom paint
Launch, and watch all that hard work disappear underwater

at April 13, 2008

I've written before about the intersection of art and technology. As a musician and an engineer, I'm always interested in art that is techie and tech thats artful. Last week I had the distinct pleasure of discovering a little piece of technology that I think is one of the coolest things I've ever seen in my life, ever. It's a software-based piano synthesizer that turns the whole world of digital piano technology on it's head.

Piano players can't really lug a piano around easily, so you never know when you show up at a gig what kind of instrument you'll have to play on. I've seen everything from a Bosendorfer to a beat up electronic Casio with the sustain pedal polarity wired in reverse. The notes wouldn't stop until you stepped down on the pedal, exactly the opposite of how it normally works. It was like trying to drive down the freeway backwards.

Like a lot of other pianists, I sprung for a decent digital stage piano that I can take to gigs with me. It's also good for recording, because miking a recording on an acoustic piano is hard to do properly. With digital, you can go back and fix your mistakes! For a number of reasons, a digital piano is really useful. But they have never been able to fully replicate the sound of a real acoustic grand piano.

In recent years, software based "sampled" digital pianos have gotten really good. Some take up hundreds of megabytes or gigabytes of disk space, with each note on the piano recorded at different velocities for up to 60 seconds, with and without the sustain pedal. Playing a moderately polyphonic piece of music requires a ton of memory and CPU power. If you have too many notes sustaining at once you can get clips and drop-outs. No matter how good they sound they are just not the same as playing a real piano.

Pianoteq takes a totally different approach. Rather than trying to record every nuance of an acoustic piano and replay the sound samples in software, Pianoteq builds a mathematical model of the physics of a piano. Everything from the length of the strings to the hardness of the felt hammers to the impedence of the soundboard is taken into consideration... and presented in the clean user interface so you can tweak it. You can build a10-meter long piano if you want to! As a bonus, there are no gigabytes of sampled data to download; the whole program was less than 14 megabytes! It mathematically calculates what the waveform of the sound will be, taking all the parameters into consideration, and then constructs the sound image in real-time.

At some point in their musical development, an instrumentalist comes to the realization that a truly fine instrument can actually make them play better. After you've spent years working on your chops, you sit down with a really great instrument and notice all sorts of ideas and techniques suddenly bubbling out of your creative reservoir. That's when you can really tell a great instrument from the mediocre. And you know, nothing is going to compare with a 9ft Bosendorfer, but after playing just about every model of grand piano out there, and dozens of digital pianos and software synths, I sat down with this Pianoteq software and was blown away. It made me play better. In one evening of jamming with it I came up with more ideas, changes and riffs than I have in months.

After keeping a careful eye on a dozen or so digital sampled piano packages, waiting for something truly convincing to come along, the wait is over. Pianoteq has what it takes.

Although I agree with another reviewer that some of the preset piano models sounded a little bit flat in the mid-range right out-of-the-box, I'll be quick to point out that the onboard EQ and reverb settings, not to mention the individual harmonic overtone settings, let you tweak the tone color to your heart's content. Actually, I suspect whats happening is that Pianoteq is almost too realistic, and those of us who've gotten used to the heavily-processed sampled piano sound just are not used to the natural purity of tone coming out of our headphones. And no other piano software gives you the range of configurable parameters to construct any type of piano model from a 9ft concert grand to a1980's vintage Yamaha CP80 electric stage piano.

Created by a pianist-turned-mathematician, Pianoteq is a great example of the combination of art and technology to produce something brilliant.

at March 25, 2008

CBC Radio, The Current, 24 March 2008 featured an interview with Rev. Richard Cizik, Vice President for Government Policy with the National Association of Evangelicals, the most powerful religious organization in the United States with over 30 million members. He is encouraging his flock of 30 million to "proselytize in the fight against climate change as well as the battle to save souls."

On a trip last August, he along with other evangelicals and scientists went to Alaska to see the effects of climate change. Asked about the impact of the trip, he said it was "an extraordinary experience for me personally as well as for all evangelicals."

The group saw first hand the impact of climate change, habitat destruction, and species extinction. Stressing the need for cooperation between the religious and scientific communities, Cizik recounted "...all these things that we together believe threaten the planet."

"It's one of the most important conversations that has to occur if we're going to be able to merge the concerns of these two worlds of religion and science on behalf of something that is surely one of the greatest challenges facing the planet - that of climate change."

In Alaska, the group saw melting of glaciers, rising sea levels of Shishmaref and the "great ghost" fir trees destroyed by bark beetles on the Kenai peninsula. Cizik said the impact of climate change is hitting close to home and that seeing it in the US is "critical" for Americans to understand.

It's clear he is passionate about the issue. "People are impacted too! Inupik Indians forced to consider leaving and losing entire way of life... the impacts are human!"

Although the group members didn't shy away from topics of creation and evolution, according to Cizik the debate is not how the earth was created (God is already well aware of that, he quips) but rather, what we do with the earth He created. He cites the Genesis 2:15 mandate to protect and care for the earth. "More import than how it all came to be is what is happening to it NOW."

The enmity between religious and scientific communities is a thing of the past, according to Cizik. "Science helps us to know what creation is telling us about itself - science is our partner in this. Science has been viewed as an enemy of the faith... that view has frankly gone by wayside for most people."

He admits that there is residual impact from the fundamentalist attitude, saying it is why large segments of the American public are skeptical about climate change, and why the GOP congress has been reluctant to act.

"This is new territory for a lot of people", he added.

Having demonstrated a much more forward-looking attitude than the religious right in recent times, it nevertheless surprised many when the Vatican revised its 1500 year-old list of Seven Mortal Sins to include "causing damage to the environment". The revised list :

1.) genetic modification
2.) carrying out experiments on humans
3.) polluting the environment
4.) causing social injustice
5.) causing poverty
6.) becoming obscenely wealthy
7.) taking drugs

Cizik calls it "a reappraisal of theology itself," and says that evangelicals have been slowest to this. Times are changing, however.

"The mainline has been on the sidelines... the irony is the community that has been least equipped has become the go-to religious community in Washington on policy changes with respect to climate change.

Cizik believes the influence of the religious community is the only way that the republicans are going to move in the policy debate. It has, he says, moved George W Bush from barely even recognizing it to saying, in his last State of the Union address "it is here".

Speaking of the beginnings of his own environmentalism, Cizik said he made a turn about face at a conference in Oxford, UK several years ago.

"I thought I was well positioned as a mugwump... don't push me, the science is split; I dont need to make decision, I don't have to engage evangelicals in this."

But the scientific evidence he saw at that conference swayed him.

"Wow... I saw with my own eyes the impacts that climate change is having upon the world. It's like I had a St. Paul's conversion. It was not by looking at the earth per se... I had it looking at the science."

He recalls thinking, "How can this be? That I myself in this case for 18 years a representative of the association here in Washington, that I could decide based on whatever evidence we had that I had no dog in this fight, so to speak? And yet I couldn't have been more wrong!"

"We all have blind spots. We evangelicals have been the blindest of all. It was a conversion for me at Oxford in England that brought me to this place. Well, I turned around and went in another direction. Yes, In some cases we evangelicals need to repent and turn around. I don't apologize for that term. I say, what is it if you turn around and go another direction except a conversion?"

Cizik insists that principles of caring for the earth are taught all the way from Genesis to Revelation, and says the religious community has been misguided by political interests.

"We've participated in an unholy alliance with big business. Those are the two components of the Republican party, the conservative moralists, the evangelicals, fundamentalists... along with big business. We're the two wings of this political party. We're 40 to 50 percent at least if not more. And we've participated in an unholy alliance over the years in which we've gotten very little andthey've gotten a lot. And one of the things they've gotten is a free run over our energy policy and our environmental policy. This is what's got to change."

Cizik is encouraging people not to put politics ahead of faith by "focusing on one or two hot-button issues". He criticizes the religious right for putting their tent in the wrong camp, and says that pro-lifers who say abortion is the main issue need to have a broader agenda. "We see hundreds of millions of lives at stake because of climate change today. I say broaden your understanding of what it means to be pro-life." The younger generation, he says, is keenly aware of these issues that will be among the most important issues they face throughout their lives. Many have been left politically homeless. Cizik sounds as if he may be among them. "I don't know who I would vote for," he admits. But he claims he predicted years ago that no non-green candidate would be elected to the republican party in 2008.

Cizik explains that historically, environmentalism has been associated with democratic blue state politics, leading many Evangelicals to view it with suspicion. Evangelicals who oppose evolution have also therefore opposed science, as a proponent of evolution, and have been skeptical of media. He calls it "a tribal kind of attitude", a "narrow fundamentalism" that is not based on sound principles of faith and wrongly ignores "undeniable realities."

The entire segment can be heard on CBC's web site at http://www.cbc.ca/thecurrent/2008/200803/20080324.html

at March 20, 2008

Here are a few things to bear in mind when creating Selenium IDE tests.

Setting and Getting Variables

store | 10 | x

This obviously sets x=10. There are a couple of ways to reference it: ${x} or storedVars['x']. They're not the same.

The first, ${x}, is from Prototype. Its a shortcut and returns a copy of storedVars['x'], it doesn't actually point to the real value. The second, storedVars['x'] directly references the array of Selenium's stored user variables. I recommend using storedVars['x'] and not ${x} (even though it's shorter).

You can't assign anything to${x}. Within a single Selenium command, if you modify storedVars['x'], the value of ${x} won't get updated until the next command. Stick with storedVars and you'll be ok.

When you want to use a previously stored variable as an argument to a subsequent command, use the "javascript{...}" notation. For example:

assertElementPresent | javascript{storedVars['myElementsXpath']}

XPATH from Firebug

You tend to use xpath queries a lot in Selenium tests. Sometimes figuring out the xpath expression for an element is a real pain. Firebug can help. Just click on the "inspect" tab and highlight the element you want. Right-click on it in the HTML code display and select "Copy Xpath" from the context menu. Paste it and give it a try. You may have to replace "/html/body/" with "//" though.

Xpath can get interesting. I like Zvon's tutorial.

Using storeEval

There are ways of defining functions even in Selenium IDE, but a simple approach is to use storeEval to run some Javascript code and put the result in a variable you can use for subsequent commands. For example, say you need to click on ButtonA if its present, or else click on ButtonB if its present. If neither one is present you want to do something else.

In this example I've got my xpath expressions for ButtonA and ButtonB stored as variables (just to illustrate the aforementioned javascript{} syntax). I'm going to fetch them and use them in my evaluation.

storeElementPresent	javascript{storedVars['ButtonA_Xpath']}	ButtonA
storeElementPresent	javascript{storedVars['ButtonB_Xpath']}	ButtonA
storeEval	(storedVars['ButtonA']==true)?storedVars['ButtonA_Xpath'] :((storedVars['ButtonB']==true)?storedVars['ButtonB_Xpath']:null);	linkToClick

That's one way to do it. Another is to use conditional logic and flow control to set up an if-else structure. Check out the Selenium IDE FlowControl post for info on how to do conditional logic-based flow control.

In the end, if you're testing a web application you've designed and your test case is getting this ugly, you'd better re-evaluate your design. If you're testing something other than a web app though, for example a test case that spans multiple sites, you might need to get creative.

at March 11, 2008

As with all good things that must come to an end, the pure spartan simplicity that defined Google at the start has begun to cede ground to the ever-encroaching clutter.

Usability trouble comes in various guises but could be summed up in a word as "inconsistency". If I had a mantra for usability it would be "meet expectations". Which is impossible when your products set conflicting expectations. With all the additions, acquisitions, and new beta projects coming out of Google now we're starting to see the inevitable usability dragon rearing its ugly head.

It happens, for example, when you click on "My Account" in one application and it takes you to the app's dashboard, click it in another application and go to your Google account preferences. I don't want to have to remember that two identical-looking links in identical fonts in identical places on Google apps have completely different functionality.

Dear Google, please fix the inconsistency.

at March 11, 2008

... a bit last night.

at March 10, 2008

When porting the original flowcontrol plugin for using goto and while loops in Selenium IDE (original Sideflow post) I found the following techniques useful for digging into the internals of Selenium IDE.

using getEval / storeEval

The functionality Selenium provides is great, but at times you may need to do something beyond the ordinary. One of the most useful commands in Selenium IDE for this is getEval. Get eval executes an arbitrary string of JavaScript code. If you've upgraded to the new Selenium IDE 1.0 Beta 1 you can also use the new "runScript" command.

getEval with alert

Most people will have tried this, I'm sure. Some say this is bad form, but I use this sort of thing all the time and it's usually the fastest way to get what I need for debugging.

|getEval | alert("simple is always better"); |
|getEval | alert("first command = "+testCase.commands[0].command); |
You'll want to remove these kinds of alerts once your test case is ready for production.

Selenium Internals

Selenium's greatest fault may be its poor documentation. Technical documentation is basically non-existent. These "can-opener" scripts can show you what's under Selenium's hood. They also show you how to open up a window and write stuff to it, an alternative to plumbing the depths of Selenium's inscrutable logging internals.

This will crack the Selenium object open for you:
|getEval| var result = "<h2>Selenium.prototype</h2><ul>"; for (var i in Selenium.prototype) { result += "<li>this." + i + " = " + Selenium.prototype[i] + " <br>";} result+="</ul>"; var debugWin = window.open("about:blank", "debugWin"); debugWin.document.open(); debugWin.document.write(result); debugWin.document.close();

Here's the browserbot object:
var result = "<h2>this.browserbot</h2><ul>"; for (var i in this.browserbot) { result += "<li>this." + i + " = " + this.browserbot[i] + " <br>";} result+="</ul>";var debugWin = window.open("about:blank", "debugWin"); debugWin.document.open(); debugWin.document.write(result); debugWin.document.close();

Try it with TestLoop.prototype, currentTest, testCase, and, of course, 'this'.

Looking at Constructors

Another handy trick is to check an object's constructor. It can be used sort of like a typeOf.
|getEval| alert(currentTest.currentCommand.constructor.toString());

How all these pieces fit together, and hacking new and nifty things out of them is left as an exercise for the reader (you).

Note: According to Matt Raible's blog post on the Selenium User's Meet-up, the next release of Selenium is going to feature some major changes including the merging of Google's webdriver code. So a lot of this is likely to change.

at February 13, 2008

tldr; https://github.com/73rhodes/sideflow

This extension provides goto, gotoIf and while loop functionality in Selenium IDE. Selenium IDE is a plugin for Firefox that automates the testing of web-based applications. There is an excellent flow-control extension at wiki.openqa.org/display/SEL/flowControl for the Selenium RC and TestRunner components, but it does not work with Selenium IDE (the Firefox plugin) directly. This makes it difficult to develop controlled test cases within Selenium IDE, and there are times when the frame-based TestRunner cannot be used (such as when the website under test employs a frame-buster script).

I've ported the existing control-flow extension to work in the Selenium IDE Firefox add-on. The image below shows a sample test case using goto, gotoIf and a while loop, all running successfully in Selenium IDE.

The file can be downloaded from Github and should be saved as "sideflow.js" to your hard drive. Then the Options settings in Selenium IDE should set the Selenium Core extensions to include the path of this file, similar to the image below.

I've published the extension on github. Go ahead and download it from there.

github.com/73rhodes/sideflow

Update 26 February, 2008:
This mod of the FlowControl extension works only with the Selenium IDE Firefox add-on. If you want to use the TestRunner component or Selenium RC, please use the original FlowControl extension.

Update 19 February 2011
I've moved the extension over to github.

Update 7 July 2011
I've updated the example to use the newer syntax required in the latest versions of Selenium IDE.

Please note that if you are using Selenium RC, you should use the original flow-control plugin here: wiki.openqa.org/display/SEL/flowControl

Unfortunately I haven't had the time or the need to make this compatible with both Selenium RC and Selenium IDE. Please feel free to fork the code on github and make a pull request if you would like to help out.

Update 24 July 2012
I added a "push" command to the plugin, with a simple example on the Github page. This allows you to easily create a collection of items and implement "for each"-style functionality to iterate over it. Here's the original announcement.

at February 11, 2008

I haven't gone away, but my computer did, for a while. It was a bad stick of RAM. In the process of getting it back, I formatted my drive. Then I had to fork over cold hard cash for a new copy of Windoze XP. I got it only because the software I use for composing and recording music, Sonar, requires it.

I refuse to use Vista. Bill Gates was recently interviewed on Gizmodo and basically admitted that Vista was a failure. Voted one of the 10 Worst Tech Products of 2007 at Yahoo Tech, then as one of the worst tech products in history over at CNet UK. MicroSoft now has a class action lawsuit against them for their bungled "Vista Capable" PC sticker program where even a MicroSoft executive claimed he'd been burnt... suckered into buying "a $2100 email machine" that won't run any of Vista much-touted features.

If you want eye candy, you have a lot better options. You could get a Mac. Or, you can use one of many free Windows shell replacements, like LiteStep. Or if you find shell replacement a little daunting, there are several options for user-friendly Linux-based operating systems. Ubuntu is a really popular choice, and the new gOS looks really good, too. See www.thinkgos.com.

In fact, complete prepackaged eGos PCs are available for $199 at Walmart or possibly your local computer store.

Stunning 3D visual effects have been part of Beryl (now called Compiz Fusion) for a long time already. They don't even require fancy hardware and will run on systems several years old with minimal RAM.

I plan on never buying another operating system again. For a long time people have been waiting for Linux to mature to the point where it was nearly as user friendly as the commercial operating systems. Now MicroSoft has handed the open source community a huge advantage by releasing a product that is so much worse than the competition.

at January 17, 2008

CTV News has reported that the Canada has named the United States on a watch-list of countries with "greater risks of torture". The list is part of a workshop for Canadian diplomats meant to teach them to recognize signs of torture and places where it is more likely to be found. It specifically mentions "U.S. interrogation techniques" including "forced nudity, isolation and sleep deprivation". Presumably the document was compiled before the use of waterboarding in American interrogations became widely publicized (waterboarding has been declared illegal by the United States in the past and is an internationally recognized form of torture[1]).

Siting the list, a lawyer for a Canadian boy who has been illegally held and tortured at the infamous U.S Guantanamo Bay prison for five years claims that he should be returned to Canada for a court hearing. The lawyer said that the boy has been abused, that his rights have been violated under international law, and that Canada is not doing enough to help him.

And the lawyer making these incriminating statements is a Lieutenant Commander in the US military.

at January 14, 2008

To follow the previous post, Noam Chomsky has written a new article, "We Own the World". It is excellent.

He explains clearly the intellectual and moral malaise that permeates current discussion of US policy towards Iran and the world in general.

Photo by John Soares, linked from www.chomsky.info

at January 13, 2008

Noam Chomsky is a professor of linguistics and philosophy at the Massachusetts Institute of Technology. He has lectured and written extensively on US foreign policy and social issues. I've always been very interested in his perspectives, but until I saw this video, I'd found his other lectures somewhat dry. This interview does a good job of presenting an informed critique of the current political climate in the US regarding Iran in a concise and articulate manner. Noam Chomsky is a very respected voice, and what he says on Iran in this video is well worth hearing.

at January 12, 2008

As a computer systems engineer, I've always thought electronic voting machines are a Bad Idea. Here is a video demonstration of a defect in the electronic voting machines being used in US elections.

Sunday, December 7, 2008

Friday, December 5, 2008

Sunday, November 23, 2008

Wednesday, November 12, 2008

Monday, October 13, 2008

Tuesday, September 30, 2008

Wednesday, September 3, 2008

Tuesday, September 2, 2008

Friday, August 29, 2008

Friday, August 22, 2008

Monday, July 7, 2008

Thursday, July 3, 2008

Friday, June 27, 2008

Thursday, June 12, 2008

Sunday, June 1, 2008

Monday, May 26, 2008

Monday, May 19, 2008

Sunday, April 13, 2008

Tuesday, March 25, 2008

Thursday, March 20, 2008

Tuesday, March 11, 2008

Monday, March 10, 2008

Wednesday, February 13, 2008

Monday, February 11, 2008

Thursday, January 17, 2008

Monday, January 14, 2008

Sunday, January 13, 2008

Saturday, January 12, 2008