We use a lot of different server software here to accomplish various tasks. In the case of web servers, there is a great deal of excellent software available to achieve the tasks, for example the majority of our applications make use of the Puma application server, Nginx web server, and HAproxy.

However we sometimes inevitable find that nothing is available to meet a specific requirement, and we need to write our own server. Over the years we have written a number of these, including the SMTP server for Postal, and RPC servers for Codebase and Deploy. In this blog post I will discuss the use of threads to achieve concurrency in servers, and why we have moved away from Threads to a more powerful event driven model for server development.

Threads

The simplest way to create a socket server capable of handling multiple clients simultaneously is to use threads. A simple server using threads might look like this.

require 'threads'
require 'socket'

listener = TCPServer.new(1234)
loop do
  Thread.new(listener.accept) do |client|
    client.puts "Hello World!"
    client.close
  end
end

This is very simple and very effective way to develop a server. One thread is created for each client and these run in parallel. The code is easy to write, easy to understand, and this prevents any one client blocking the others.

There are however a number of disadvantages to threads, particularly with regard to scaling when a large number of clients create a large number of threads. The first and perhaps simplest problem is that each thread costs a significant amount of memory. This is a simple problem but as the number of clients grows memory can become a concern.

The second problem is the performance of threads. With lots of threads running in parallel, it can become very difficult to debug performance issues, or track down an issue that is occurring within a specific thread. Ruby is only able to execute one operation at a time (regardless of thread). It does a good job of switching quickly between threads, however the total performance will never exceed that of a single thread. The more threads are running, the greater the overhead of monitoring and switching between them.

While many applications will run very happily in a threaded model, the issues above mean that sometimes this model does not scale, particularly with regard to servers handing large numbers of simultaneous connections. For this reason, we have found it beneficial to move to an event driven model for many of our new servers.

Events

In contrast to the threaded model, in an event driven server there is only a single thread. This thread runs in a loop waiting for any event to occur, and when it does, it calls an appropriate method. Once this method returns, the loop waits for the next event. This is called an event loop. This highlights the one major disadvantage of event driven development: methods must return as quickly as possible because nothing else in the program will run until they do.

Lets look at what our hello world application above would look like as an event driven server.

class HelloServer
  def on_connect
    send_data "Hello World!"
    close
  end
end

This also seems extremely simple, however it is clear that a lot of code is missing here. We need something to run the event loop, to accept connections, to create an instance of our HelloServer class, and to call our method.

In order to achieve this, we will need a framework, and we have 2 main options available to us:

  • EventMachine is a full framework and will handle absolutely everything, allowing you to focus on your application logic
  • nio4r is a lower level library that will allow you to monitor sockets for events, excellent if you want a greater level of control

A server in EventMachine

First, lets write our example server using EventMachine. This is almost identical to our conceptual server code above. EventMachine handles all all the work of listening, accepting connections, creating an instance of our EchoServer class, and allows us to focus entirely on our application logic. This is an extremely easy and effective way to get started with event driven server programming.

require 'eventmachine'

module EchoServer
  def post_init
    send_data "Hello World!"
    close_connection
  end
end

EventMachine.run {
  EventMachine.start_server "::", 1234, HelloServer
}

nio4r

The nio4r library allows you to monitor for events on IO instances, such as TCPServer, TCPSocket, UDPSocket. Writing a complete event driven framework is beyond the scope of this post, however the following code shows a simple event loop using nio4r that accepts TCP connection:

selector = NIO::Selector.new

server = TCPServer.new(1234)
selector.register(server, :r)

loop do
  selector.select do |monitor|
    case monitor.io
    when TCPServer
      if monitor.readable?
        # This means our TCPServer has clients waiting to connect
        client = monitor.io.accept_nonblock
      end
    end
  end
end

This concept can be built up into a full event driven server by writing 3 main classes:

  • EventLoop - contains the above code and runs the main loop
  • EchoServer - responds to events that occur on the listening server, such as a client waiting to connect
  • EchoClient - responds to events that occur on a connected client, such as incoming data

Conclusion

We can use event driven servers as a powerful alternative to multi-threaded servers. There are a number of advantages to this in terms of performance, however it is important to take care never to write any long running or blocking code in any of our event driven methods. EventMachine makes it very easy to get started with event driven servers so if you're used to using threads, why not give it a try!

Tell us how you feel about this post?