Strange Loop

Spores: Distributable Functions in Scala

FULL TITLE: Spores: Distributable Functions in Scala

ABSTRACT: Functional programming languages are often sold as the future of distributed programming. Yet, despite all of the hype, a number of fundamental requirements of distributed systems remain quite painful to achieve in practice. You can't really send closures, and persisting data typically requires a lot of boilerplate, is slow, or is locked-in to a specific serialization format.

This talk presents some of our efforts at better supporting distributed programming in Scala. We present a new fast pickling framework designed for Scala, as well as Spores-- composable pieces of mobile functional behavior.

Scala's new pickling framework has a few attractive properties: (1) using the framework requires little to no boilerplate, (2) using Scala's implicit parameters, users can add their own easily-swappable pickle format enabling users to persist to a number of formats, binary, JSON, or your own custom format, (3) using the type class pattern, users can provide their own custom picklers to override the default behavior of the pickling framework, (4) static, macro-based generation of picklers enables significant performance improvements. In microbenchmarks our framework achieves a factor 6 speedup over Java Serialization, and performs on par or up to 3 times faster than popular fast Java serialization frameworks like Kryo. We'll show how this new framework can be used as a drop-in replacement for other serialization frameworks.

Spores are well-behaved, composable pieces of functional behavior that can be cleanly pickled, transported over the wire, unpickled, and more. We'll cover some important application areas for spores, interesting patterns that they can make possible, and we'll go on to hint at new and interesting distributed frameworks that they could enable.

Heather Miller

Heather Miller

Scala Team, EPFL

Heather Miller has been a member of the Scala team, and a PhD student under Martin Odersky, since 2011. Alongside of her research which touches on providing better language support for distributed programming, and deterministic abstractions for parallel and concurrent programming, she is a regular contributor to Scala— she was on the team that designed Scala's Futures & Promises, is responsible for improving several aspects of Scala documentation (API docs, guides, tutorials, quickrefs, including, and leads the now 90,000-student-strong Coursera massive open online course, “Functional Programming Principles in Scala.”