Onward! 2016
Sun 30 October - Fri 4 November 2016 Amsterdam, Netherlands
co-located with SPLASH 2016
Wed 2 Nov 2016 16:30 - 16:55 at Zürich 2 - Session 2 Chair(s): Jeff Huang

The most successful systems for ``big data'' processing have all adopted
functional APIs. We present a new programming model we call {\em function
passing} designed to provide a more principled substrate on which to build
data-centric distributed systems. A key idea is to build up a persistent
functional data structure representing transformations on distributed
immutable data by passing well-typed serializable functions over the wire and
applying them to this distributed data. Thus, the function passing model can
be thought of as a persistent functional data structure that is {\em
distributed}, where transformations to data are stored in its nodes rather
than the distributed data itself. The model simplifies failure recovery by
design–in the event of a failure, data is recovered by replaying function
applications atop immutable data loaded from stable storage. Deferred
evaluation is also central to our model; by incorporating deferred evaluation
into our design only at the point of initiating network communication, the
function passing model remains easy to reason about while remaining efficient
in time and memory. We formalize our programming model using
small-step operational semantics, and we provide an open-source
implementation of our model in and for the Scala programming language, along
with a case study of several example frameworks and end-user programs written
atop of this model.