Dummy

How we built BayesHive on Yesod

14 July 2013

The web server component of BayesHive is built using Yesod, a web framework for Haskell that combines type safety and good performance with lightweight syntax for routes, persistence and templates. This post is written for other users of Yesod or Haskell, or those who are comparing Yesod and maybe Haskell to other development environments.

What is BayesHive?

BayesHive is what you get when you ask functional programmers to re-invent statistics and data analysis software. At the core of BayesHive is a functional programming language, but we want users with little or no programming experience to use BayesHive as well, so we have built a user interface that allows new users to write code without them even knowing it.

BayesHive is based on the user interacting with three different kinds of entities: documents, data and models.

Documents: Users write documents containing text in English (or any other language!), code in the Baysig language, and questions consisting of Baysig expressions. These documents are, depending on which tradition you come from, examples of literate programming or reproducible research.

Data: Data sets are uploaded by users and are immediately given a type in a type system a little like Haskell's but with first-class but non-extensible records like those in Standard ML. What is the type of your grandfather's spreadsheet? It is a list of records where every field name comes from the column header. What is the type of a timeseries? It is a function from Time to values, just like in functional reactive programming.

Models: In contrast to almost most other data analysis environments, in BayesHive/Baysig we do not transform data or even apply functions to data in order to calculate "insight". Instead, we build statistical models for the raw data and use Bayesian inference to derive the probability distribution over the model parameters. These models and the inference step are written in the Baysig language itself, but we want new users to be able build and run those models quickly, so we provide model builders, point-and-click interfaces that generate Baysig code for some common modelling tasks (linear and nonlinear regression, ANOVA and dynamical systems models).

Model builders: web app "wizards"

From early on we decided that there should be multiple model builders corresponding to common classes of statistical models, and that they should be fairly autonomous in how they work. It quickly became apparent that we needed a wizard-like user interface to allow the model builders to iteratively elicit information from the user to define the models they had in mind for their data.

Model builders use a modified Fuel UX wizard
Model builders use a modified Fuel UX wizard

The complexity of these interfaces drove us towards using the AngularJS JavaScript framework on the browser side of things, since its data binding capabilities makes it easier to build highly responsive web UIs. This approach was inspired by Michael Snoyman's very encouraging blog post on the topic, and we took Michael's original Yesod-Angular code as a basis to work from. We have since extended this code to work with the Angular ui-router package. This allows us to embed our wizard-like model builders within a single-page JavaScript application presenting an "explorer" type view of a user's data, documents and models.

State-based routing with Yesod and ui-router

Using our Angular.UIRouter module, we can define a hierarchical set of user interface "states", each of which has associated Hamlet files defining the user interface appearance and Julius files defining its behaviour. For example, we have a set of states for handling documents called doc, doc.edit, doc.view and doc.options. The "abstract" doc state defines layout and behaviour that is common to all document handling operations, while the subsidiary states define the details of editing documents (using a CodeMirror text panel and a timer to save document changes to the server), viewing them (documents are rendered to HTML on the server, and we have code to maintain a status display as the documents are rendered, then to splice the document HTML into the page) and for setting document options.

This degree of modularisation makes developing complex user interfaces much easier, and it was essential for developing our model builders, each of which has up to six or seven different stages of varying complexity. (One stage of the dynamical systems model builder includes a whole system for doing basic symbolic analysis of systems of ordinary or stochastic differential equations; another is a whole interface for simulating and plotting simulation results of such systems.)

The layout and behaviour definitions for all the states are collected together using Template Haskell code that traverses the directory tree of state definitions and constructs the state dispatch structure needed to initialise the Angular ui-router system. This is all transparent to a user of the system: a single Template Haskell function called buildStateUI processes all the definitions within a monad type called AngularUI, and the resulting value can be rendered as a Yesod Handler by calling a runAngularUI function.

Although it's a feature of Yesod that's sometimes quoted as a disadvantage, we've found Yesod's widespread use of Template Haskell to be very powerful and convenient. It would have been hard to integrate the ui-router system into a web app cleanly without it.