Improving the performance of SSL services in NodeJS_node.js

When browsing the internet, we all know that encryption via SSL is very important. At PayPal, security is our top priority. We use end-to-end encryption, not just for our public website, but for our internal service calls as well. SSL encryption technology will affect the performance of node.js to a great extent. We have taken the time to adapt our external services and make the most of them. Below is a list of some SSL configuration tweaks we have found to significantly improve SSL external performance.

SSL Password

Out of the box, SSL for Node.js uses a very strong set of cryptographic algorithms. In particular, Diffie-Hellman key exchange and elliptic curve algorithms are extremely expensive. And when you use too many outbound SSL calls in the default configuration, the performance of Node.js will be fundamentally weakened. To get a sense of just how slow it is, here is a CPU sample of a service call:

918834.0ms 100.0% 0.0 node (91770)
911376.0ms 99.1% 0.0  start
911376.0ms 99.1% 0.0  node::Start
911363.0ms 99.1% 48.0  uv_run
909839.0ms 99.0% 438.0  uv__io_poll
876570.0ms 95.4% 849.0   uv__stream_io
873590.0ms 95.0% 32.0    node::StreamWrap::OnReadCommon
873373.0ms 95.0% 7.0     node::MakeCallback
873265.0ms 95.0% 15.0     node::MakeDomainCallback
873125.0ms 95.0% 61.0     v8::Function::Call
873049.0ms 95.0% 13364.0    _ZN2v88internalL6InvokeEbNS0
832660.0ms 90.6% 431.0     _ZN2v88internalL21Builtin
821687.0ms 89.4% 39.0      node::crypto::Connection::ClearOut
813884.0ms 88.5% 37.0       ssl23_connect
813562.0ms 88.5% 54.0       ssl3_connect
802651.0ms 87.3% 35.0        ssl3_send_client_key_exchange
417323.0ms 45.4% 7.0         EC_KEY_generate_key
383185.0ms 41.7% 12.0        ecdh_compute_key
1545.0ms 0.1% 4.0          tls1_generate_master_secret
123.0ms 0.0% 4.0           ssl3_do_write
...

Let’s focus on key generation:

802651.0ms 87.3% 35.0 ssl3_send_client_key_exchange
417323.0ms 45.4% 7.0 EC_KEY_generate_key
383185.0ms 41.7% 12.0 ecdh_compute_key

87% of the time in this call is spent generating keys!

These passwords can be changed to make them less computationally intensive. This idea has been implemented with https (or proxy). For example:

var agent = new https.Agent({
  "key": key,
  "cert": cert,
  "ciphers": "AES256-GCM-SHA384"
});

The above key has not been exchanged with the expensive Diffie-Hellman key. After replacing it with something similar, we can see a significant change in the example below:

...
57945.0ms 32.5% 16.0 ssl3_send_client_key_exchange
28958.0ms 16.2% 9.0 generate_key
26827.0ms 15.0% 2.0 compute_key
...

You can learn more about cipher strings through the OpenSSL documentation.

SSL Session Resume

If your server supports SSL session resumption, then you can pass the session over https (or proxy). You can also wrap the proxy's createConnection function:

var createConnection = agent.createConnection;

agent.createConnection = function (options) {
  options.session = session;
  return createConnection.call(agent, options);
};

Session resumption can reduce the number of connections used by adding a short handshake mechanism to the connection.

Keep active

Allowing the proxy to stay alive will ease the SSL handshake. A keep-alive agent such as agentkeepalive can fix the node keep-alive problem, but it is not required in Node 0.12.

Another thing to keep in mind is the proxy’s maxSockets, a high value can have a negative impact on performance. Control your maxSockets value based on the number of outbound connections you create.

Slab size

tls.SLAB_BUFFER_SIZE determines the allocated size of the slab buffer used by the tls client (server). Its size defaults to 10MB.

These allocated ranges will expand your rss and increase garbage collection time. This means that high capacity will impact performance. Adjusting this capacity to a lower value can improve memory and garbage collection performance. In version 0.12, slab allocation has been improved and no further adjustments are required.

Recent changes to SSL in 0.12

Test Fedor's SSL enhanced version.

Test instructions

Run an http service as an SSL service proxy, all running on this machine.

v0.10.22

Running 10s test @ http://127.0.0.1:3000/
20 threads and 20 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 69.38ms 30.43ms 268.56ms 95.24%
Req/Sec 14.95 4.16 20.00 58.65%
3055 requests in 10.01s, 337.12KB read
Requests/sec: 305.28
Transfer/sec: 33.69KB

v0.11.10-pre (Build from main version)

Running 10s test @ http://127.0.0.1:3000/
20 threads and 20 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 75.87ms 7.10ms 102.87ms 71.55%
Req/Sec 12.77 2.43 19.00 64.17%
2620 requests in 10.01s, 276.33KB read
Requests/sec: 261.86
Transfer/sec: 27.62KB

This doesn’t make much of a difference, but that’s due to the default password, so let’s adjust the password’s proxy options. For example:

var agent = new https.Agent({
  "key": key,
  "cert": cert,
  "ciphers": "AES256-GCM-SHA384"
});

v0.10.22

Running 10s test @ http://localhost:3000/
20 threads and 20 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 59.85ms 6.77ms 95.71ms 77.29%
Req/Sec 16.39 2.36 22.00 61.97%
3339 requests in 10.00s, 368.46KB read
Requests/sec: 333.79
Transfer/sec: 36.83KB

v0.11.10-pre (Build from main version)

Running 10s test @ http://localhost:3000/
20 threads and 20 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 38.99ms 5.96ms 71.87ms 86.22%
Req/Sec 25.43 5.70 35.00 63.36%
5160 requests in 10.00s, 569.41KB read
Requests/sec: 515.80
Transfer/sec: 56.92KB

As we can see, after Fedor's modification, there is a huge difference: the performance difference from 0.10 to 0.12 is almost 2 times!

Summary

Some people may ask "Why not just turn off SSL, it will become faster after turning it off", and for some people this is also an option. In fact, this is the typical answer when I ask people how they solve SSL performance problems. However, if enterprise SSL requirements are anything but increased, and even though a lot has been done to improve SSL in Node.js, performance tuning is still needed. I hope some of the above techniques help you tune the performance of your SSL use cases.