Friday, November 13, 2015

Reacting to React

While I'm on the blog here, I figured I'd take a minute to write my thoughts on React / React Native.

Recently I've had occasion to play with React Native a bit. This post is not so much about first impressions, though, but about my perspective and some preconceptions about things like React in general.

I run a big JavaScript meetup group called OttawaJS. I get to see lots of interesting presentations on the latest and greatest technology, and sometimes give talks myself. Over the past few years I've seen an absolute deluge of new frameworks and libraries for web and mobile development, and like a lot of people in the tech community, have suffered from "new framework fatigue".  Perhaps this is one reason why, when React came around, I didn't get immediately excited about it. Beyond that, here are some general observations, personal biases, and preconceptions.

1. When Mark Zuckerberg famously said "HTML5 isn't ready", what he really should have said is Facebook wasn't ready for HTML5. The good folks at Sencha proved the point by building an HTML5 clone of the Facebook app that outperformed the native one. Anyway, because of that bit of history, I realize that just because a big corporation like Facebook is behind something doesn't mean it's automatically awesome.

2. What I just said about Facebook is the same for any big company that produces and promotes a framework. This is one reason I steered clear of Angular; it came from Google. I've worked in big enterprises and have been developing software long enough now that I'm no longer naive enough to believe that big enterprises build open source frameworks out of the generosity in their hearts for the betterment of mankind. The frameworks and tools that I tend to prefer, and in fact the ones that have generally proven most successful over time, are ones that have been written by a single author, usually out of personal interest, and slowly built up a community around them. The bigger the company behind it, the more skeptical I am.

3. The marketing hype machine was in high gear. This is always a bad sign. React had barely been introduced before there were conferences about it and a flood of videos and articles. Paul Graham said something similar about Java a long time ago. About how good languages and frameworks don't need to be marketed, and how anything with a big corporate marketing engine behind it smells funny.

4. There are some cool functional programming concepts in React but they're mixed in with the classical object model. This kinda points to confusion about what the designers thought they were trying to build.

5. Generating HTML programmatically actually does suck. Other languages and framework have tried this, and they also sucked. I think Ember's Glimmer engine demonstrates that you can minimize DOM updates and get lots of great performance improvement without re-inventing the way HTML is written.

6. React seems to have done a lot to make people aware of cool things like immutability and components. And that's really great. However I think those concepts have value outside of React and can be applied without jumping on the React bandwagon.

7. Gut feeling. It just doesn't feel like React is the final holy grail solution we've all been waiting for. It brings some cool concepts to the table and has helped shift developers' obsession with things like two-way data binding and REST API's and made them start thinking about alternatives. Also really great. However I think web components and the native web stack are heading in the right direction and will ultimately replace a lot of what client-side MVC frameworks currently do.

So that's why I've told people I'm not betting on React becoming the defacto standard way to build web/mobile apps. At best, I think some of it's most useful concepts will be extracted, cleaned up and incorporated in smaller standalone implementations. Redux is the prime example of that.

KCJS keep calm and javascript.

Successful Software Projects

Successful software projects don't happen by accident. A recent post by Jeffrey Ventrella on Slow Programming resonates with developers who've been through enough projects to know the pros and cons of the old top-down design approaches versus the popular new "iterative development" and agile approaches.  The author wrote about his team's fast, iterative development process:

At the job, we were encouraged to work in the same codebase, as if it were a big cauldron of soup, and if we all just kept stirring it continuously and vigorously, a fully-formed thing of wonder would emerge. It did not.

The fallacy of fungible engineers - viewing engineers as resources that can be assigned interchangeably to various projects, is at the heart of the problem. Unleashing a bunch of developers on a project to "vigorously stir the cauldron" without a solid high-level design is clearly not a great idea. Iterative development may be a more realistic approach to software development, but you can't do away with up-front planning altogether.  As Ventrella says, "You can't wish away the design process."

So is iterative development a bad thing?  I don't think so. It's important to note that new paradigms are usually reactionary - they introduce a different approach to fix the failings of the old one.  Out of necessity, they're usually overly reactionary. The new approach condemns the old practices in order to get people to change, but it often fails to recognize that underneath there's probably a core of ideas still worth considering.

There are valuable aspects of both the "slow programming" approach and the "iterative" approach. It may not be necessary to define all the detailed requirements up front, but projects can benefit from asking a few simple questions to lay the groundwork before development starts.  These are things like the optimal technology stack for your problem space, documentation, and what you want your testing to look like.  Too often, these decisions are done in an ad hoc manner, late in the development cycle, or simply ignored altogether.  I feel that addressing these questions at the start of project is important for success. Here they are, roughly in the order of importance, as I see them.

  • Design: design is often the worst part of software products created by large enterprises. Almost no thought goes into it; a product gets built starting with the data model; layers of code are built up on top of it until the data is finally vomited onto the user's screen in a great pile of meaningless clutter. Good design is not part of the corporate DNA of a large company with a legacy of hardware development. This makes visual design an issue of low hanging fruit for smaller competitors, who can quite easily win over potential clients with the usability and elegance of their product, even if it lacks the feature set of a more mature offering.  Conversely, great design can't be done by a non-technical "designer" without a deep technical understanding of the product; it's a mistake to view design as a separate activity that can take place in isolation from the technical implementation. Graphic designers are generally bad at understanding the technical details of a project, and sometimes force engineers into making poor compromises for the sake of preserving a "look and feel".  Engineers are generally quite bad at user interface design, unless they also have a background and strong aptitude in the visual arts. Good engineers are rare; good engineers who are also artists are even rarer. Unless you have one of them, you'll need a design / engineer pair who can work in close collaboration to define the initial product vision. It is not going to magically emerge from a set of requirements handed down by Product Level Managers. Ask yourself: Is there a strong vision for the aesthetic sense of your product? Is it informed by a strong technical understanding as well as a good aesthetic sensibility?  Can it be demonstrated that the design decisions are grounded in good usability guidelines and not just the whim of a non-technical "graphic designer"? Are you leveraging existing best-in-class frameworks for UI design? Can you point to the examples that inspire the aesthetic vision of the product? Do you know what you want your product to look like?  Is it beautiful? Does it inspire people to want to use it?
  • Testability: how do you want to be able to test your code? Do you want unit tests?  End to end functional testing? What do you want the testing process to look like?  Will it be the same in the developer environment and the build environment? What tools will you consider? Do you want continuous integration? What's considered bad enough to break the build? Do you value test coverage reporting? How much test coverage is enough?
  • Documentation: it can make or break a project and often it is left as an afterthought, to be completed by "technical writers" who have little if any actual technical knowledge of how the thing actually works. Can you make it "self documenting"? How? What do you want the documentation to look like? In what formats will it be available? Will you have a developer's guide? An install guide? A user guide? How will you keep it up to date? How will you know if it's out of date?
  • Surface area: how do you define the points at which clients interface with your product (it's surface area)? A lot of this boils down to good API design. What are the best practices that you wish to implement? Which existing products / services exemplify what you want to achieve? What are the existing successful APIs, trends, industry best practices you want to emulate. Against what do you measure the usability of your API? Who are the current best-in-class leaders and how do you build upon their examples?
  • Technology stack: Quite often the default in large enterprises is Java. This is unfortunate because Java is rarely if ever the best technological solution to a particular problem - it's just the one that large numbers of mediocre programmers happen know. Think outside the box - one developer with a killer tool-set is deadlier than a whole team of mediocre Java programmers. Have you checked out ThoughtWorks Technology Radar? How do you place bets on which technologies will succeed going forward?  Do you have a strong sense of picking technology winners that comes from being in the trenches? Or are you a manager trying to make a "safe" choice?  Are your choices based on cargo cult or bandwagon tendencies?  Does your framework of choice have a track record of solid bets, critical mass of users, a core team with a clear focus and commitment to keeping things small - aka. the "do one thing well" philosophy? Will you ask for permission or for forgiveness? As a lead developer, are you willing to tell your managers that the technology and technical design decisions are ultimately your decisions to make?
  • Tooling: will you standardize on set of tools or let developers use whatever ad hoc collection of tools they prefer? Will you utilize linters, and static code analyzers like PMD, Findbugs, and Flow? Where in the development process where they be used - in the developer environment by triggering these tools as each file is edited? Will you trigger build failures after committing flawed code? Will you depend on a monolithic IDE like Eclipse or use a lightweight code editor like Sublime?
  • Code organization / architecture: The default, of course, is the Big Ball of Mud design pattern.  Are you a devotee of Object Oriented design? Are you enlightened by Functional Programming? Prefer an Abstraction Oriented architecture? What is your high level view of how it all fits together with a view towards delegating responsibilities? How to you define the boundaries between components? Can you specify, in draft form, an APIs to act as a  'contract' between components that can then be delegated off to different developers? How will you communicate your architectural vision and get buy-in from team members?
  • Prototype: have you built a proof of concept? Will you? How? Do you intend to productize the prototype, or is it meant to serve only as a reference for what you want the final product to be like? If you plan to evolve your prototype into a production application, do you have a clear path for doing so? For example, have you steered clear of design decisions in the prototype that will be difficult to back out of, like introducing dependencies on libraries or techniques that you won't be able to release in a production environment? As a real world bad example: our start-up commissioned a prototype for a Blackberry "app". It ran on simulator, used polling instead of push notifications, probably did not use the right technology stack for what we needed, was nowhere near production quality, and resulted in a lot of wasted effort. As a real-world good example: having learned from our Blackberry experiences, we created a mobile security demo using a relatively new technology that allowed rapid development, but made sure the API specification was well defined, knowing that it would not be difficult to re-implement the API in any server side language we chose.
  • Delegation: how can the project can be split up into sensible areas of responsibility as it grows? If the project has started out with one full-stack developer, will it be easy to carve off the API, UI, mobile app, etc. into standalone projects that another developer or team can take ownership of? A lot of this comes down to having well defined and documented APIs as a contract between components.
  • Security: Its not enough to trust that your product is going to be secure because it uses Framework X.  Upon inspection, our initial subscriber-facing web interface turned out to be susceptible to session hijacking, cross-site scripting, and SQL insertion, and security issues continued to be present despite being overseen by senior developers.  Security audit tools revealed even more weaknesses. What tools will you use to audit the security of your product (Skipfish, Nessus, OpenVAS, etc.)? Will you carry out manual penetration testing on your product? Will security audits be performed automatically, on a regular basis? Will they be integrated into your build environment or QA environment?

The questions you need to ask may vary, but the important thing is to have a plan, communicate the plan well, empower people to own their part of it, make the design objectives clear (including all of the points above), and make sure everyone knows how their piece fits into the overall product vision.

Thursday, January 22, 2015

Fluent 2014 Talk Summaries 3

"Speed, Performance and Human Perception"

In this performance-related talk, Ilya Grigorik explains that performance is not only a function of speed, but of meeting user expectations in a way that allows them to complete tasks effectively, with insightful examples.


My rating: 4/5. Provides useful insight on usability.

"Delivering the Goods"

Paul Irish discusses optimization in this keynote talk, describing the "critical path" and how requests impact page load times. Chrome developer tools are used to explain page load sequences and timing. Recommendations include: eliminate render-blocking JS; minimize render-blocking CSS; serve content in the original HTML response and use gzip compression. Google Page Speed Test is a tool to automatically recommend such optimizations.


My rating: 5/5. Beneficial recommendations for all developers of web-based software.

Tuesday, January 20, 2015

Fluent 2014 Talk Summaries, continued

"Reading, Writing, Arithmetic... and JavaScript?"

Pamela Fox, the person behind the JavaScript-based programming curriculum at Khan Academy, discusses how age affects a person's ability to learn programming and says that most 13 year olds are capable of learning basic programming skills that will help them explore other fields like art, history and language.  Visit Khan Academy for more information on their Computer Programming curriculum.

My rating: 3/5.  Introduces a great educational resource for young people. Plus, I've been a fan of Pamela's work ever since I read her articles on JavaScript widgets at Coursera.


"Virtual Machines, JavaScript and Assembler"

Popular podcaster Scott Hanselman delivers a humorous talk describing how the basic features of an operating system exist in both the cloud and the browser.

My rating: 3/5. Entertaining for programmers, though light on technical take-aways.


"The Humble Border-Radius"

In this talk Lea Verou demonstrates how border radii can be used to create a variety of shapes and animations, as well as upcoming CSS specifications for different corner shapes. While interesting, I think what web designers really want is complete flexibility to design arbitrary shapes, and border-radius seems like a very awkward mechanism for accomplishing that.

My rating: 3/5. Interesting CSS hacks.


Monday, January 19, 2015

Fluent Talk Summary: Brendan Eich, "JavaScript: the High and Low Roads"

In this lighthearted talk, Brendan Eich (inventor of JavaScript) discusses high and low-level improvements in upcoming versions of JavaScript. Acknowledging that Web development is hard ("like having a chain-saw in place of a hand"), Eich says the upcoming version of JavaScript, ES6 "Harmony", will address many difficulties facing web developers, with improvements like a built-in module system, observable objects, and many other features that will make "transpilers" and other "syntactic sugar" libraries unnecessary.

The future ES7 version of JavaScript aims to bring many low-level improvements for high-performance and scientific computing, including new value objects, operator overloading, and SIMD intrinsics. New value objects proposed in ES7 include: int64, uint64, int32x4 and int32x8 (SIMD), float32 (useful for GPUs), float32x4 and float32x8 (SIMD), bignum, decimal, rational, and complex. ES7 will also introduce operator overloading and new literals like 0L (int64), 0UL (uint64), etc. Support for SIMD (Single Instruction, Multiple Data) will lead to native performance for applications like game programming and signal processing. To demonstrate further, Eich reveals the first-ever public demo of the Unreal Engine 4 from Epic Games, showing stunning 3D graphics running at a full 60 frames per second, with no plugins, in a Firefox build with special SIMD capabilities.

Guided by the Extensible Web Manifesto, the high road of developer-friendly features and the low road of safe, low-level language improvements will converge in a virtual machine that offers native performance while being very developer friendly.

In a nutshell - "Always bet on JS!"

My rating: 4/5. Informative, insightful and entertaining for programmers.

Sunday, January 18, 2015

Dynamic Types FTW

There's a belief among some programmers, particularly the OO classical inheritance folks, that static typing is a bulwark of security against all kinds of disasters that may happen in your code.  They feel that the compiler will ensure their programs work correctly, by generating errors and warnings and refusing to compile their program until all the mistakes are fixed.

Some developers mistakenly speak in terms of "strong typing" and "weak typing". Put that way, who wouldn't prefer "strong" over "weak" typing? In reality the terms "strong typing" and "weak typing" are undefined in the language of computer science. They're misnomers that people sometimes use to talk about strict (explicit) type definition with static type checking, versus dynamic (runtime) type checking with implicit type conversion.

It's unfortunate that this line of reasoning causes some people to avoid languages that offer dynamic types. Dynamic types can be very useful, and static typing actually offers very little in the way of ensuring program correctness.  I finally sat down and watched Eric Elliott's presentation "Static Types are Overrated: The Dynamic Duo - Loose Types and Object Extension" and was treated to a bang-on description why static types aren't all they're cracked up to be.  Eric does a good job of explaining and clarifying things about JavaScript and programming in general that I intuitively believed but had a difficult time putting into words or backing up with solid examples.

I borrowed a bunch of points from Eric's talk when I spoke at a recent Ottawa JavaScript meetup. I was presenting on Flow, a static type checker for JavaScript, but I wanted people to see it as something that could be added to their regular code linting process rather than something to enforce a statically typed programming style. Here are some of them.
  • It's a myth that dynamic loosely typed languages like JavaScript aren't suitable for building large, complex applications.  There are dozens of examples from Facebook to 37signals, Dow Jones, Adobe, Flickr, LinkedIn, Walmart, etc).
  • Type correctness does not guarantee program correctness. This is one of the biggest cargo-cult myths out there: thinking that static, strict type checking will save you. It won't.
  • When languages lack dynamic types, people fake it with ugly hacks - void pointers, variadic functions, abusing array accessors, type-casting of all sorts, generics and ugly template classes, to name a few. I don't know any C or Java programmers who haven't used and abused at least a few of these techniques.
  • "Any sufficiently advanced C / Fortran [Java, etc] program contains an ad hoc, informally specified, bug-ridden, slow implementation of half of common Lisp" - Greenspun's Tenth Rule. (There are no rules 1 through 9, by the way. Greenspun's Rules of Programming start and end with number 10.)
  • Older statically typed languages are now introducing dynamic types because they really are useful. Objective-C added them a long time ago; more recently C++11 has added dynamic types, Java has "generics", and libraries like cppscript and boost provided ways to do dynamic types in earlier versions of C++.
  • Functional programming benefits from functions that can operate on any type which implements its requirements (for example, arguments have a valueOf method). Otherwise known as duck typing. This permits things like map, filter, forEach etc. to be used generically.
  • Correctness can only really be assured through proper unit and integration testing. 
  • Code that is well organized in small, simple modules, linted, unit tested, peer reviewed, and integration tested, is very unlikely to contain type errors. 

The last thing I would add is that while static types are over-rated, static code analysis tools like JSLint, Tern.js and Flow are mostly underrated and underutilized. If you use something like Flow or Tern.js to provide hints and insights into potential type mismatches, the chances of type errors become essentially zero. Ideally they should be as up-front as possible, i.e. integrated directly into your code editor. I wrote a JSLint plugin for SublimeText and recently wrote a Flow plugin as well. They provide immediate static code analysis warnings as I edit my code, so I can fix problems before I commit anything to the source code repository.

Wednesday, November 12, 2014

Three Simple Rules for Escaping Callback Hell

A lot of newcomers to Node.JS complain about "callback hell" and the "pyramid of doom" when they're getting started with the callback-driven continuation passing style.  It's confusing, and a lot of people reach for an async / flow-control module right away.  Many people have settled on using Promises, a solution that brings some unfortunate problems along with it (performance, error-hiding anti-patterns, and illusory behavior, for example).

I prefer using some simple best practices for working with callbacks to keep my code clean and organized. These techniques don't require adding any extra modules to your code base, won't slow your program down, don't introduce error-hiding anti-patterns, and don't convey a false impression of synchronous execution. Best of all, they result in code that is actually more readable and concise, and once you see how simple they are, you might want to use them, too.

Here they are:
  1. use named functions for callbacks
  2. nest functions when you need to capture (enclose) variable scope
  3. use return when invoking the callback

The Pyramid of Doom

Here's a contrived example that uses typical node.js callbacks with (err, result) arguments. It's a mess of nested functions: the so-called Pyramid of Doom. It keeps indenting, layer upon smothering layer, until it unwinds in a great cascading spasm of parenthesis, braces and semi-colons.

Named Callbacks

The Pyramid of Doom is often shown as a reason to use Promises, but most async libraries -- including and especially Promises -- don't really solve this nesting problem.  We don't end up with deeply nested code like this because something is wrong with JavaScript. We get it because people write bad, messy code.  Named callbacks solve this problem, very simply. Andrew Kelley wrote about this on his blog a while ago ("JavaScript Callbacks are Pretty Okay"). It's a great post with some simple ways of taming "callback hell" that get skipped over by a lot of node newcomers.

Here's the above example re-written using named callback functions. Instead of a Russian doll of anonymous functions, every function that takes a callback is passed the name of the callback function to use. The callback function is defined immediately afterwards, greatly improving readability.

Nest Only for Scope

We can do even better. Notice that two functions, sendGreeting and showResult, are still nested inside of the getGreeting function. Nested "inner" functions create a closure that encloses the callback function's own local variable scope, plus the variable scope of the function its nested inside of. These nested callbacks can access variables from higher up the call stack. In our example, both sendGreeting and showResult use variables that were created earlier in the getGreeting function. They can access these variable from getGreeting, because they're nested inside getGreeting and thus, enclose its variable scope.

A lot of times this is totally unnecessary. You only need to nest functions if you need to refer to variables in the scope of the caller from within the callback function. Otherwise, simply put named functions on the same level as the caller. In our example, variables can be shared by moving them to the top-level scope of the greet function. Then, we can put all our named functions on the same level. No more nesting and indentation!

Return when invoking a Callback

The last point to improve readability is more a stylistic preference, but if you make a habit of always returning from an error-handling clause, you can further minimize your code. In direct-style programming where function calls are meant to return a value, common wisdom says that returning from an if clause like this is bad practice that can lead to errors.  With continuation-passing style, however, explicitly returning when you invoke the callback ensures that you don't accidentally execute additional code in the calling function after the callback has been invoked. For that reason, many node developers consider it best practice. In trivial functions, it can improve readability by eliminating the else clause, and it is used by a number of popular JavaScript modules.  I find a pragmatic approach is to return from error handling clauses or other conditional if/else clauses, but sometimes leave off the explicit return on the last line in the function, in the interest of less code and better readability. Here's the updated example:

Compare this example with the Pyramid of Doom at the beginning of the post. I think you'll agree that these simple rules result in cleaner, more readable code and provide a great escape from the Callback Hell we started out with.

Good luck and have fun!