Speaker Focus: Mark Volkmann on Software Transactional Memory

Mark Volkmann is a partner at Object Computing (OCI) and is doing a talk at Strange Loop called Tackling Concurrency with Software Transactional Memory. I had a chance to ask him a few questions about the talk and his experiences working with Clojure.

Alex: What is software transactional memory anyways?

Mark: STM is a technique for dealing with concurrency that is similar to database transactions. It greatly simplifies coding for concurrency because, unlike with locks, developers aren't required to identify up front which objects they will be accessing. Developers simply surround sections of code that require access to a "consistent" state for a group of variables with a special syntax that demarcates transaction boundaries.

Alex: Your talk abstract mentions Clojure, which uses an STM to manage mutable state changes. Is your talk about Clojure? What do you find interesting about Clojure's use of STM?

Mark: The main goal of my talk to is describe the general characteristics of STM and then describe the specific characteristics of one implementation, that of Clojure. It's important to emphasize that, just like with garbage collection, there are many ways to implement STM. It's difficult to make general statements about STM since implementations can differ quite a bit. Learning about the specifics of the Clojure implementation will give developers ideas for questions they should ask about other implementations. They may get ideas for improving STM implementations, including that of Clojure. Most importantly, they will gain the ability to reason about the impact of STM on their applications.

Some things that I find interesting about the Clojure STM implementation include its use of Java's concurrency classes such as AtomicLong and ReentrantReadWriteLock. It's also interesting how it maintains a chain of past values for each variable that is modified by a transaction commit.

Alex: STM is one approach to managing concurrency and is touted to be a simpler model than using locks to manage shared memory. Why is it simpler from a programming perspective?

Mark: In general STM makes it easier to write "correct" concurrent code. This is because certain kinds of errors just cannot happen. For example, with a lock-based approach, developers are on their honor to only modify objects for which they have obtained the appropriate locks. If they forgot to lock an object before modifying it, well that's okay, they get to modify it anyway. Not so with STM. In the case of Clojure and Haskell, variables are immutable unless they are a special kind of variable (for example, Ref in Clojure or TVal in Haskell). Those special kinds of variables can ONLY be modified inside a transaction. Forgetting to start a transaction isn't allowed.

Alex: From a performance point of view, does STM promise to be more scalable or higher performance than shared locks?

Mark: Definitely not! Anything that can be achieved with STM can also be achieved with locks. Using locks is just more work to implement and more prone to programming errors. STM is all about simplifying the task of developing concurrent code. It is often compared to garbage collection. Developers can manage memory themselves in a language like C or let the language runtime manage it for them in a language like Java. Memory can be managed correctly in C, but it's more work to implement and more prone to programming errors.

Alex: What are the major disadvantages to software transactional memory?

Mark: The main disadvantage is probably performance. When a transaction attempts to modify a variable that has been changed by another transaction since the current one began then it has to retry. That means throwing away all the work it has done and starting over at the beginning of the transaction. It is possible that a transaction will have to retry many times before it can complete successfully. This is typically minimized by keeping transaction durations short and avoiding concurrent transactions that differ greatly in their duration and modify the same variables.

Keep in mind that when garbage collection implementations began appearing, many people said they were too slow and that the only way to achieve reasonable performance was to hand code memory management. Few people feel that way today. I think it's likely we'll see a similar shift in opinions about STM as implementations improve.

Another temporary disadvantage is that we haven't yet developed good tools to help with identifying how often each transaction retries and why they retry (for example, learning which variables had write conflicts). Having this knowledge would aid developers in writing better performing code that uses STM.

Alex: What approaches have you taken to learn about STM and in particular Clojure's implementation?

Mark: Read, read, read. There are many articles about STM on the web. I've also spent a large amount of time studying the source code for the Clojure STM implementation and asking questions about it on the Clojure mailing list and IRC channel.

Alex: Thanks Mark, that was great. I'm really looking forward to your talk at Strange Loop.

Mark Volkmann will be speaking at the Strange Loop conference in St. Louis on October 22nd. For more information check out the schedule and prices, the full session list, or register now.