Tuesday, January 20, 2015

Fluent 2014 Talk Summaries, continued

"Reading, Writing, Arithmetic... and JavaScript?"


Pamela Fox, the person behind the JavaScript-based programming curriculum at Khan Academy, discusses how age affects a person's ability to learn programming and says that most 13 year olds are capable of learning basic programming skills that will help them explore other fields like art, history and language.  Visit Khan Academy for more information on their Computer Programming curriculum.


My rating: 3/5.  Introduces a great educational resource for young people. Plus, I've been a fan of Pamela's work ever since I read her articles on JavaScript widgets at Coursera.

[video]

"Virtual Machines, JavaScript and Assembler"


Popular podcaster Scott Hanselman delivers a humorous talk describing how the basic features of an operating system exist in both the cloud and the browser.

My rating: 3/5. Entertaining for programmers, though light on technical take-aways.

[video]

"The Humble Border-Radius"


In this talk Lea Verou demonstrates how border radii can be used to create a variety of shapes and animations, as well as upcoming CSS specifications for different corner shapes. While interesting, I think what web designers really want is complete flexibility to design arbitrary shapes, and border-radius seems like a very awkward mechanism for accomplishing that.

My rating: 3/5. Interesting CSS hacks.

[video]

Monday, January 19, 2015

Fluent Talk Summary: Brendan Eich, "JavaScript: the High and Low Roads"

In this lighthearted talk, Brendan Eich (inventor of JavaScript) discusses high and low-level improvements in upcoming versions of JavaScript. Acknowledging that Web development is hard ("like having a chain-saw in place of a hand"), Eich says the upcoming version of JavaScript, ES6 "Harmony", will address many difficulties facing web developers, with improvements like a built-in module system, observable objects, and many other features that will make "transpilers" and other "syntactic sugar" libraries unnecessary.

The future ES7 version of JavaScript aims to bring many low-level improvements for high-performance and scientific computing, including new value objects, operator overloading, and SIMD intrinsics. New value objects proposed in ES7 include: int64, uint64, int32x4 and int32x8 (SIMD), float32 (useful for GPUs), float32x4 and float32x8 (SIMD), bignum, decimal, rational, and complex. ES7 will also introduce operator overloading and new literals like 0L (int64), 0UL (uint64), etc. Support for SIMD (Single Instruction, Multiple Data) will lead to native performance for applications like game programming and signal processing. To demonstrate further, Eich reveals the first-ever public demo of the Unreal Engine 4 from Epic Games, showing stunning 3D graphics running at a full 60 frames per second, with no plugins, in a Firefox build with special SIMD capabilities.

Guided by the Extensible Web Manifesto, the high road of developer-friendly features and the low road of safe, low-level language improvements will converge in a virtual machine that offers native performance while being very developer friendly.

In a nutshell - "Always bet on JS!"

My rating: 4/5. Informative, insightful and entertaining for programmers.

Sunday, January 18, 2015

Dynamic Types FTW

There's a belief among some programmers, particularly the OO classical inheritance folks, that static typing is a bulwark of security against all kinds of disasters that may happen in your code.  They feel that the compiler will ensure their programs work correctly, by generating errors and warnings and refusing to compile their program until all the mistakes are fixed.

Some developers mistakenly speak in terms of "strong typing" and "weak typing". Put that way, who wouldn't prefer "strong" over "weak" typing? In reality the terms "strong typing" and "weak typing" are undefined in the language of computer science. They're misnomers that people sometimes use to talk about strict (explicit) type definition with static type checking, versus dynamic (runtime) type checking with implicit type conversion.

It's unfortunate that this line of reasoning causes some people to avoid languages that offer dynamic types. Dynamic types can be very useful, and static typing actually offers very little in the way of ensuring program correctness.  I finally sat down and watched Eric Elliott's presentation "Static Types are Overrated: The Dynamic Duo - Loose Types and Object Extension" and was treated to a bang-on description why static types aren't all they're cracked up to be.  Eric does a good job of explaining and clarifying things about JavaScript and programming in general that I intuitively believed but had a difficult time putting into words or backing up with solid examples.

I borrowed a bunch of points from Eric's talk when I spoke at a recent Ottawa JavaScript meetup. I was presenting on Flow, a static type checker for JavaScript, but I wanted people to see it as something that could be added to their regular code linting process rather than something to enforce a statically typed programming style. Here are some of them.
  • It's a myth that dynamic loosely typed languages like JavaScript aren't suitable for building large, complex applications.  There are dozens of examples from Facebook to 37signals, Dow Jones, Adobe, Flickr, LinkedIn, Walmart, etc).
  • Type correctness does not guarantee program correctness. This is one of the biggest cargo-cult myths out there: thinking that static, strict type checking will save you. It won't.
  • When languages lack dynamic types, people fake it with ugly hacks - void pointers, variadic functions, abusing array accessors, type-casting of all sorts, generics and ugly template classes, to name a few. I don't know any C or Java programmers who haven't used and abused at least a few of these techniques.
  • "Any sufficiently advanced C / Fortran [Java, etc] program contains an ad hoc, informally specified, bug-ridden, slow implementation of half of common Lisp" - Greenspun's Tenth Rule. (There are no rules 1 through 9, by the way. Greenspun's Rules of Programming start and end with number 10.)
  • Older statically typed languages are now introducing dynamic types because they really are useful. Objective-C added them a long time ago; more recently C++11 has added dynamic types, Java has "generics", and libraries like cppscript and boost provided ways to do dynamic types in earlier versions of C++.
  • Functional programming benefits from functions that can operate on any type which implements its requirements (for example, arguments have a valueOf method). Otherwise known as duck typing. This permits things like map, filter, forEach etc. to be used generically.
  • Correctness can only really be assured through proper unit and integration testing. 
  • Code that is well organized in small, simple modules, linted, unit tested, peer reviewed, and integration tested, is very unlikely to contain type errors. 

The last thing I would add is that while static types are over-rated, static code analysis tools like JSLint, Tern.js and Flow are mostly underrated and underutilized. If you use something like Flow or Tern.js to provide hints and insights into potential type mismatches, the chances of type errors become essentially zero. Ideally they should be as up-front as possible, i.e. integrated directly into your code editor. I wrote a JSLint plugin for SublimeText and recently wrote a Flow plugin as well. They provide immediate static code analysis warnings as I edit my code, so I can fix problems before I commit anything to the source code repository.

Wednesday, November 12, 2014

Three Simple Rules for Escaping Callback Hell

A lot of newcomers to Node.JS complain about "callback hell" and the "pyramid of doom" when they're getting started with the callback-driven continuation passing style.  It's confusing, and a lot of people reach for an async / flow-control module right away.  Many people have settled on using Promises, a solution that brings some unfortunate problems along with it (performance, error-hiding anti-patterns, and illusory behavior, for example).

I prefer using some simple best practices for working with callbacks to keep my code clean and organized. These techniques don't require adding any extra modules to your code base, won't slow your program down, don't introduce error-hiding anti-patterns, and don't convey a false impression of synchronous execution. Best of all, they result in code that is actually more readable and concise, and once you see how simple they are, you might want to use them, too.

Here they are:
  1. use named functions for callbacks
  2. nest functions when you need to capture (enclose) variable scope
  3. use return when invoking the callback

The Pyramid of Doom

Here's a contrived example that uses typical node.js callbacks with (err, result) arguments. It's a mess of nested functions: the so-called Pyramid of Doom. It keeps indenting, layer upon smothering layer, until it unwinds in a great cascading spasm of parenthesis, braces and semi-colons.


Named Callbacks

The Pyramid of Doom is often shown as a reason to use Promises, but most async libraries -- including and especially Promises -- don't really solve this nesting problem.  We don't end up with deeply nested code like this because something is wrong with JavaScript. We get it because people write bad, messy code.  Named callbacks solve this problem, very simply. Andrew Kelley wrote about this on his blog a while ago ("JavaScript Callbacks are Pretty Okay"). It's a great post with some simple ways of taming "callback hell" that get skipped over by a lot of node newcomers.

Here's the above example re-written using named callback functions. Instead of a Russian doll of anonymous functions, every function that takes a callback is passed the name of the callback function to use. The callback function is defined immediately afterwards, greatly improving readability.


Nest Only for Scope

We can do even better. Notice that two functions, sendGreeting and showResult, are still nested inside of the getGreeting function. Nested "inner" functions create a closure that encloses the callback function's own local variable scope, plus the variable scope of the function its nested inside of. These nested callbacks can access variables from higher up the call stack. In our example, both sendGreeting and showResult use variables that were created earlier in the getGreeting function. They can access these variable from getGreeting, because they're nested inside getGreeting and thus, enclose its variable scope.

A lot of times this is totally unnecessary. You only need to nest functions if you need to refer to variables in the scope of the caller from within the callback function. Otherwise, simply put named functions on the same level as the caller. In our example, variables can be shared by moving them to the top-level scope of the greet function. Then, we can put all our named functions on the same level. No more nesting and indentation!


Return when invoking a Callback

The last point to improve readability is more a stylistic preference, but if you make a habit of always returning from an error-handling clause, you can further minimize your code. In direct-style programming where function calls are meant to return a value, common wisdom says that returning from an if clause like this is bad practice that can lead to errors.  With continuation-passing style, however, explicitly returning when you invoke the callback ensures that you don't accidentally execute additional code in the calling function after the callback has been invoked. For that reason, many node developers consider it best practice. In trivial functions, it can improve readability by eliminating the else clause, and it is used by a number of popular JavaScript modules.  I find a pragmatic approach is to return from error handling clauses or other conditional if/else clauses, but sometimes leave off the explicit return on the last line in the function, in the interest of less code and better readability. Here's the updated example:

Compare this example with the Pyramid of Doom at the beginning of the post. I think you'll agree that these simple rules result in cleaner, more readable code and provide a great escape from the Callback Hell we started out with.

Good luck and have fun!

Monday, October 13, 2014

How Wolves Change Rivers

A beautifully filmed short video from Yellowstone National Park that reminds us of the importance of wildlife for the health of the whole planet:


How Wolves Change Rivers from Sustainable Man on Vimeo.

Wednesday, September 17, 2014

doxli - a help utility for node modules on command line

Quite often I fire up the node REPL and pull in some modules I've written to use on the command line. Unfortunately I often forget the exact way to call the various functions in those modules (there are a lot) and end up doing something like foo.dosomething.toString() to see the source code and recall the function signature.

In the interest of making code as "self-documenting" as possible,  I wrote a small utility that uses dox to provide help for modules on the command line. It adds a help() function to a module's exported methods so you can get the dox / jsdoc comments for the function on the command line.

So now foo.dosomething.help() will return the description, parameters, examples and so on for the method based on the documentation in the comments.

It's still a bit of a work in progress, but it works nicely - provided you actually document your modules with jsdoc-style comments.

All the info is here: https://www.npmjs.org/package/doxli

Sunday, September 7, 2014

REST API Best Practices 4: Collections, Resources and Identifiers

Other articles in this series:
  1. REST API Best Practices: A REST Cheat Sheet
  2. REST API Best Practices: HTTP and CRUD
  3. REST API Best Practices: Partial Updates - PATCH vs. PUT
RESTful APIs center around resources that are grouped into collections. A classic example is browsing through the directory listings and files on a website like http://vault.centos.org/. When you browse the directory listing, you can click through a series of folders to download files.  The folders are collections of CentOS resource files.



In Rest, collections and resources are accessed via HTTP URI's in a similar way:

members/ -- a collection of members
members/1 -- a resource representing member #1
members/2 -- a resource representing member #2

It may help to think of a REST collection as a directory folder containing files, although its highly unlikely that the member data is stored as literal JSON files on the server. The member data should be coming from a database, but from the perspective of a REST API, it looks similar to a directory called "members" that contains a bunch of files for download.

Naming collections


In case it's not obvious already, collection names should be nouns. Use the plural form for naming collections. There's been some debate over whether collection names should be plural (members/1) or singular (member/1). The plural form seems to be most widely used.

Getting a collection


Getting a collection, like "members" may return
  1. the entire list of resources as a list of links, 
  2. partial representations of each resource, or 
  3. full representations of all the resources in the collection. 
Our classic example of browsing online directories and files uses approach #1, returning a list of links to the files. The list is formatted in HTML, so you can click on the hyperlink to access a particular file.

Approach #2, returning a partial representation (ie. first name, last name) of all resources in a collection is a more pragmatic way of returning enough information about the resources in a collection for the end user to make a selection to request further details, especially if the collection can contain a lot of resources. Actually, the directory listings on a website like http://vault.centos.org/ display more than just the hyperlink. They include additional meta-data like the last-modified timestamp and file size, as well.  This is helpful for the end-user who's looking for an up-to-date file and wants to know how long it will take to download. It's a good example of returning just enough information about the resources for the end-user to be able to make a selection.

With approach #3, if a collection is small, you may want to return the full representation of all the resources in the collection as a big array.  For large collections, it isn't practical, however. Github is the only RESTful API example I've seen that actually returns a full representation of all resources when you fetch the collection. I wouldn't consider  #3 to be a "best practice", or recommend it for most use cases, but if you know the collection and resources will be small, it might be more effective to fetch the whole collection all at once like this.

The best practice for fetching a collection of resources, in my opinion, is #2: return a partial representation of the resources in a collection with just enough information to facilitate the selection process, and be sure to include the URL (href) of each resource where it can be downloaded from.

Only when a collection is guaranteed to be small and you need to reduce the performance impact of making multiple queries, consider bending the rules with approach #3 to return all the resources in one fell swoop.

Here's a practical example of fetching the collection of members using approach #2.

Request

GET /members
Host: localhost:8080

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

[
  {
    "id": 1,
    "href": "/members/1",
    "firstname": "john",
    "lastname": "doe"
  },
  {
    "id": 2,
    "href": "/members/2",
    "firstname": "jane",
    "lastname": "doe"
  }
]

In this example, some minimal information is returned about each of the members: first and last name, id, and the "href" URL where the full representation of the member resource can be downloaded.


Getting a resource


Getting a specific resource should returns the full representation of that resource from the URL that contains the collection name and the ID of the specific resource you want.

Resource IDs


RESTful resources have one or more identifiers: a numerical ID, a title, and so on. Common practice is for every resource to have a numeric ID that is used to reference the resource, although there are some notable exceptions to the rule.

Resources themselves should contain their numerical ID; the current best practice is for this to exist within the resource simply as an attribute labelled "id". Every resource should contain an "id"; avoid using more complicated names for resource identifiers like "memberID" or "accountNumber" and just stick with "id". If you need additional identifiers on a resource, go ahead and add them, but always have an "id" that acts as the primary way to retrieve the resource. So, if a member has "id" : 1, it should be fairly obvious that you can fetch his details at the URL "members/1".

An example of fetching a member resource would be:

Request

GET /members/1
Host: localhost:8080

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "id": 1,
  "href": "/members/1",
  "firstname": "john",
  "lastname": "doe",
  "active": true,
  "lastLoggedIn": "Tue Sep 16 2014 08:37:42 GMT-400 (EDT)",
  "foo": "bar",
  "fizz": "buzz",
  "qux": "doo"
}

Beyond simple collections


Most of the examples you see online are fairly simple, but practical data models are often much more complex.  Resources frequently contain sub-collections and relationships with other resources. API design in this area seems to be done in a mostly ad-hoc manner,but there are some practical considerations and trade-offs when designing APIs for more complex data models, which should be covered in the next post.

Productivity and Note-taking

I told a friend of mine that I wasn't really happy with the amount of time that gets taken up by Slack and "communication and sched...