1 server. 22 lakh connections.

• To deliver messages instantly, a chat app keeps an open TCP connection between your phone and its server. Millions of users online means millions of open connections.
• The classic way to serve connections, one operating-system thread per connection, gets expensive fast. In 1999 even 10,000 connections on one server was a famous challenge: the C10K problem.
• WhatsApp's server is written in Erlang, a language built for telephone networks. Erlang processes are tiny (about 2.6 KB when created) and an idle one uses no CPU, so one server can hold a huge number of them.
• The team then measured and removed bottlenecks: past about 425,000 connections, CPU cores were queueing on shared locks. They patched the Erlang VM, tuned FreeBSD, and went from 1 million to 2 million connections per server in about a month.
• By early 2014 WhatsApp had 465 million monthly users on about 550 servers. Facebook bought it for about $19 billion, when it had roughly 35 engineers.

There are two ways a phone can find out it has a new message. It can ask the server every few seconds ("anything for me?"), which wastes battery and data and still arrives late. Or it can keep one connection open and let the server push the message the moment it arrives. Chat apps use the second way.

The catch is on the server. Every connected phone is an open socket the server must keep track of: some kernel memory for its buffers, a file descriptor, and whatever the application keeps per user. Most of those connections are idle most of the time (people aren't typing every second), but they all have to stay open and ready.

In 1999, Dan Kegel's page "The C10K problem" asked how to get one server to handle ten thousand clients at the same time. That year ftp.cdrom.com was serving 10,000 clients at once on a single machine, and it was notable enough to become the page's opening example.

Why the classic way struggles

The straightforward design gives every client its own operating-system thread. It's easy to write: each thread reads, waits and writes as if it were the only client. But each thread needs its own stack, and the operating system has to schedule all of them.

The C10K page's answer was to stop dedicating a thread to each connection and instead let one thread watch many sockets with an efficient event mechanism, such as kqueue on FreeBSD or epoll on Linux. That's fast, but the code turns into callbacks and state machines that are hard to get right. Erlang offers a third path.

Processes, not threads

Erlang code runs in processes that belong to the Erlang virtual machine (the BEAM), not to the operating system. They share no memory and talk only by sending each other messages. Because the VM manages them itself, they're extremely light:

A process runs until it waits for a message in a receive. While it waits, the VM doesn't schedule it at all, so idle connections cost memory but no CPU. The VM runs processes on a small pool of scheduler threads, by default one per CPU core.

WhatsApp's server started from ejabberd, an open-source chat (XMPP) server written in Erlang, and the team later rewrote large parts of it and moved to its own protocol. The general Erlang pattern for a server like this looks like the diagram below. (WhatsApp hasn't published its exact internal design, so treat this as the model Erlang gives you, not a blueprint of their code.)

The pattern in code

A minimal Erlang server that gives every connection its own process (an echo server, for illustration):

%% one Erlang process per connection (illustrative, not WhatsApp's code)
-module(echo).
-export([start/1]).

start(Port) ->
    {ok, Listen} = gen_tcp:listen(Port, [binary, {active, false}, {reuseaddr, true}]),
    accept(Listen).

accept(Listen) ->
    {ok, Socket} = gen_tcp:accept(Listen),
    Pid = spawn(fun() -> receive go -> loop(Socket) end end),
    ok = gen_tcp:controlling_process(Socket, Pid),
    Pid ! go,
    accept(Listen).

loop(Socket) ->
    ok = inet:setopts(Socket, [{active, once}]),
    receive                                  %% asleep here: no CPU
        {tcp, Socket, Data}  -> gen_tcp:send(Socket, Data), loop(Socket);
        {tcp_closed, Socket} -> ok
    end.

Erlang made millions of connections possible. Getting one machine there still took work. Rick Reed described the journey at Erlang Factory in March 2012: their servers started at about 200,000 connections each, and hit their first bottlenecks at about 425,000.

What was wrong: contention

Reed's summary was "contention, contention, contention". The server had 24 logical CPUs, and at high load the cores spent their time waiting in line for the same locks inside the VM and kernel (examples he gave include the timer wheel and the time-of-day lock). Adding connections stopped adding throughput because the cores were queueing instead of working.

What they changed

• Patched the Erlang VM (BEAM) to remove lock contention. Reed said most of the gains from 200k to 2M came from these fixes.
• Tuned FreeBSD: backported kernel improvements (the TSC time counter and the igb network driver) and raised limits with sysctl settings such as socket counts and the TCP hash table size.
• Changed the app where shared data caused contention, for example by partitioning it.

The climb, to scale

Bars to scale against the 2.8M peak. The 10,000 bar is so small it barely shows.