Monday, April 7, 2014

REST API Best Practices 2: HTTP and CRUD

This post expands a bit further on the previous post regarding HTTP operations for Create / Read / Update / Delete functionality in REST APIs.

APIs for data access and management are typically concerned with four actions (the so-called CRUD operations):
  • Create - the ability to create a resource
  • Read - the ability to retrieve a resource
  • Update - the ability to modify a resource
  • Delete - the ability to remove a resource

RESTful APIs based on HTTP typically implement these actions using the standard HTTP methods:
  • POST - to create a resource
  • GET - to retrieve a resource
  • PUT - to update a resource by replacing it
  • PATCH - to update part of a resource
  • DELETE - to remove a resource

Many folks point out that there's not an exact mapping between CRUD operations and HTTP methods, but this list represents best practice and follows the HTTP specification.

For background, the HTTP 1.1 specification defines "safe" and "idempotent" methods [1].  Safe methods don't modify data on the server no matter how many times you call them. Idempotent methods can modify data on the server the first time you call them, but repeating the same call over and over again won't make any difference. Here's a partial list:


The safe and/or idempotent nature of these HTTP methods provides some further insight into how they ought to be used. Notice that POST is neither safe, nor idempotent. A successful POST should create new data on the server, and repeating the same call should create even more copies on the server. GET, on the other hand, is safe and idempotent, so no matter how many times you call it, the data on the server shouldn't be affected.

GET - use it to fetch resources, but don't "tunnel" request parameters through to the server as a way to alter the state of data on the server - as a "safe" method, calling GET shouldn't have side effects.

PUT - use it to update an existing resource by replacing it with a new representation. The data you PUT to the server should be a complete replacement for the specified resource. Although PUT can in theory be used to insert new resources, in practice it's not advisable. Note that after the first PUT request, repeatedly calling the same PUT method with the same data won't change the data on the server more than it already has been (a condition of idempotent methods).

PATCH - use it to update an existing resource by changing some of it's properties. Unfortunately, the PATCH method isn't supported everywhere, but if you want to provide partial updates, it's the right method to use [2]. Newer version of frameworks like Ruby on Rails support the PATCH method; if your server doesn't provide it you'll need to consider some alternatives that I'll cover in the next post.

POST - use it to create new resources. The server should create a unique identifier for each newly created resource. Return a 201 Created response if the request was successful. Best practice appears to be to return the unique ID in the response. POST is also frequently used to trigger actions on the server which technically aren't part of RESTful API, but provide useful functionality for web applications.

DELETE - use it to delete resources; it's pretty self-explanatory.


Friday, March 21, 2014

REST API Best Practices: a REST Cheat Sheet

I'm interested in REST API design and identifying the best practices for it. Surprisingly, a lot of APIs that claim to be RESTful, aren't. And the others all do things differently. This is a popular area, though, and some best practices are starting to emerge.  If you're interested in REST, I'd like to hear your thoughts about best practices.

REST is not simply JSON over HTTP,  but most RESTful APIs are based on HTTP. Request methods like POST, GET, PUT and DELETE are used to implement Create, Read, Update and Delete (CRUD) operations. The first question is how to map HTTP methods to CRUD operations.

To start, here's a "REST API Design Cheat Sheet" that I typed up and pinned to my wall. Its based on the book "REST API Design Rulebook", and the HTTP RFC. I think it reflects standard practice. There are newer and better books on the subject now, but this list covers the basics of HTTP requests and response codes used in REST APIs.

Request Methods

  • GET and POST should not be used in place of other request methods
  • GET is used to retrieve a representation of a resource
  • HEAD is used to retrieve response headers
  • PUT is used to insert or update a stored resource
  • POST is used to create a new resource in a collection
  • DELETE is used to remove a resource

Response Status Codes

  • 200 "OK" indicates general success
  • 200 "OK" shouldn't be used to return error messages
  • 201 "Created" indicates a resource was successfully created
  • 202 "Accepted" indicates that an asynch operation was started
  • 204 "No Content" indicates success but with an intentionally empty response body
  • 301 "Moved Permanently" is used for relocated resources
  • 303 "See Other" tells the client to query a different URI
  • 304 "Not Modified" is used to save bandwidth
  • 307 "Temporary Redirect" means resubmit the query to a different URI
  • 400 "Bad Request" indicates a general failure
  • 401 "Unauthorized" indicates bad credentials
  • 403 "Forbidden" denies access regardless of authentication
  • 404 "Not Found" means the URI doesn't map to a resource
  • 405 "Method Not Allowed" means the HTTP method isn't supported
  • 406 "Not Acceptable" indicates the requested format isn't available
  • 409 "Conflict" indicates a problem with the state of the resource
  • 412 "Precondition Failed" is used for conditional operations
  • 415 "Unsupported Media Type" means the type of payload can't be processed
  • 500 "Internal Server Error" indicates an API malfunction
A note about the PATCH method. There are good reasons to consider using the HTTP PATCH method for partial updates of resources, but because it's not supported everywhere, and because there are workarounds, I haven't added it to my cheat sheet yet.

Wednesday, March 19, 2014


Kasa  © 2010 Darren DeRidder.
Umbrella detail, Kinosaki, Japan

Tuesday, March 11, 2014

When Agile Went Off the Rails

Whenever I hear a company say "We follow an agile development process", I subconsciously grimace. Agile methodology is something of an oxymoron. Oh, the core ideas of it were (are) excellent, but somewhere along the way it turned into a bunch of codified management processes, and became exactly the sort of thing the Agile Manifesto was trying to eliminate. As soon as the management folk came along and tried to formalize agile philosophy into a methodology, it stopped being agile.

Now one of the original authors of the Agile Manifesto has come out with a piece, originally titled "Time to Kill Agile", in which he makes this point. Dave Thomas has been hugely influential in the software development field. Aside from being one of the authors of the agile manifesto, he's written a lot of other stuff, and he's the guy who coined the phrases "code kata" and "DRY" (Don't Repeat Yourself - the maxim developers follow to effectively organize their code).  Evidently, Dave considered that "Agile" was probably already dead and renamed the piece "Agile Is Dead (Long Live Agility)!"

The problem with being a critic of Agile, or anything else that's become ingrained in a big software development organization, is that you'll come up against folks who've been drinking the kool-aid for long enough that if you call into question the value of Agile (or Oracle, or Java, or whatever) you'll likely be viewed as an idiot.

So this - having one of the original authors of the Agile Manifesto come out and make the same kind of criticism of "Agile" methodology that I've been skeptical of for years - is gold.

It all boils down to people over processes. Agile software development cannot be implemented as a set of methodologies, and the managers, consultants and companies that have turned it into a set of rules, processes and formalities, have demonstrated that they totally don't get what the authors of the Agile Manifesto intended in the first place.  In my humble opinion.

Dave Thomas has some good advice for teams that want to develop software with agility. He advocates an iterative approach to development, and choosing options that enable future change. He recommends thinking of "agile" in the form of an adverb (agilely, or "with agility"). Programming with agility. Teams that execute with agility.

I've found that when it comes to managing a project, simple is usually better. What's worked best in my experience is, in a nutshell, to simply encourage communication. Make sure everyone understands the overall objective, how they can contribute to it, what progress has been made and what challenges remain, and importantly, give everyone the opportunity to have their work fully recognized and appreciated on a regular basis. Given the opportunity to work on a challenging project and the chance to have their contributions seen and appreciated by colleagues, most developers will bend over backwards to do their best work.

There's no right or wrong way to do this, but one technique I found effective was a brief (set-a-timer with a hard cut-off) meeting at the beginning of the week where we laid out the objectives for the week ahead, a quick information-gathering hike around the office at the end of the week, and an email re-cap on Friday afternoon highlighting the teams progress. Showing the percentage-towards-completion of major tasks was also a big motivator, as developers began to take pride in seeing their areas of responsibility make steady, visible progress towards completion.  We didn't formalize or get locked into one way of doing it, so when our company got acquired and our management structure changed, we adapted pretty easily.

  • People over processes.
  • Working software over documentation.
  • Collaboration over contracts.
  • Adaptability over planning.

Saturday, February 22, 2014

Itsukushima Jinja

UNESCO World Heritage Site, Itsukushima Shrine, Hatsukaichi, Hiroshima Prefecture, Japan.

Itsukushima Jinja
© 2014 Darren DeRidder. Originally uploaded by 73rhodes

Thursday, December 5, 2013

Node.JS Module Patterns using simple examples

Slides for a recent talk at Ottawa.JS on "Node.JS Module Patterns using simple examples" are available. The slides have been updated to include a brief intro to Common.JS, examples for exporting named and anonymous functions, objects and prototypes, and an explanation of "exports" vs. "module.exports".

Saturday, September 28, 2013

JavaScript Efficiencies

I'm working on an article about patterns for structuring Express.JS apps, which is taking too long, so I decided to write this instead: Here are a few tips and tricks for JavaScript programming that I like.

Comment switches

Comment switches let you comment out entire blocks of code with a single character, or switch between two different blocks of code with a single character, which can be useful when prototyping.
// */
Removing the first slash '/' toggles between these two print statements. See the original post for more examples of comment switches.

Iterate by Counting Down

You can iterate n times concisely, like this:
var n = 1000;
while (n--) { ... }

Defaulting Arguments

This is a handy way to provide a default value for undefined arguments in a JavaScript function.
function foo(bar) {
   var bar = bar || "Some default";