Strange Loop

Next: September 12-14 2019

/

Stifel Theatre

/

St. Louis, MO

Distributed Systems with Gevent and ZeroMQ

Concurrency is hard! At least, it seems hard. You might feel that unless you take up Erlang, you'll never be able to express distributed systems in an easy way. I'll let you in on a secret: the magic of Erlang, and Go, and any other language with good concurrency primitives can also be found in a library called ZeroMQ, which can be used in any language. It's especially powerful and expressive when combined with an evented programming environment like Node.js, EventMachine, or Twisted.

I'll let you in on another secret: there is a better way to do evented async programming than having callbacks all over the place, making your program read backwards and making your mind melt. By combining the reactor pattern of async frameworks with green threads, you get something that, so far, only exists in Python as two libraries called Gevent and Eventlet. These give you the advantages of I/O based evented programming, but without the ugly callbacks. Plus, most Python libraries are likely to work, unlike the completely incompatible and reimplemented world of Twisted or EventMachine in Ruby. Your Python code ends up feeling as powerful as Go code.

By combining these two technologies, you have a powerful toolkit for easily throwing together distributed systems in a "normal" language. Although we'll focus on Gevent and ZeroMQ in Python, the lessons of this talk apply to any language. We'll dive in and further explain these two technologies and then explore them together in action.

Click to view published talk video

Jeff Lindsay

Twilio

Jeff Lindsay is a systems thinking hacker-philosopher. He builds, he explores, and is apparently “wired differently.” Obsessed with esoteric knowledge, he applies modern technology driven by an eclectic model of the bigger picture. He’s heavily involved in hacker culture, startup culture, web technologies and the indie game scene. With dozens of new and ongoing projects, his motto is “always be shipping.” While not hacking on software (currently at Twilio), Jeff speaks on the Evented Web, teaches programming, continues to run the legendary SuperHappyDevHouse hackathons, and co-directs Hacker Dojo, the largest hackerspace in the country.