tinc 1.1pre19 slower than tinc 1.0, experimentalProtocol even more

[Henning Verbeek]

Hi there,

we're using tinc to mesh together hosts in a public datacenter (instead of
using a private VLAN, sort of). So all hosts are reasonably modern;
connections are low latency with an available bandwith of around 500Mbit/s
or 1Gbit/s (depending on how close they are to each other). Iperf between
two nodes directly reports around 940Mbit/s. The CPUs are Intel(R) Core(TM)
i7-4770 CPU @ 3.40GHz, the hosts are running debian wheezy (kernel
3.2.54-2), the aesni module is loaded. OpenSSL is at 1.0.1e.

On tinc 1.0, with cipher aes-128-cbc and digest sha1, I managed to get
around 900Mbit/s throughput (iperf, simplex, both tinc daemons at 90% CPU
utilisation). Switching to a null-cipher came up with only marginally
better throughput. Note, this is from memory, can't find my notes
unfortunately.

Reading about tinc 1.1, specifically the use of a GCM cipher suite to
reduce the cost of HMAC, we've decided to test it. The results are quite
surprising:
* using the experimental protocol, throughput falls to around 380Mbit/s;
both tincd are at or just above 100% CPU (the host has 8 cores)
* not specificing ECDSA keys, throughput is around 600Mbit/s; both tincd
are at around 90-95% CPU

sptps_speed reports:
Generating keys for 10 seconds:           5200.94 op/s
ECDSA sign for 10 seconds:                3710.72 op/s
ECDSA verify for 10 seconds:              1734.05 op/s
ECDH for 10 seconds:                      1449.40 op/s
SPTPS/TCP authenticate for 10 seconds:     641.37 op/s
SPTPS/TCP transmit for 10 seconds:           8.45 Gbit/s
SPTPS/UDP authenticate for 10 seconds:     640.18 op/s
SPTPS/UDP transmit for 10 seconds:           8.64 Gbit/s

Note: the two nodes are connected via two separate tinc-mesh-networks
(since they participate in different network groups). Two tincds are
running, sharing the same UDP port; one is configured with ECDSA keys, the
other without. Not sure if that is causing a problem.

Are these findings expected? Or are we doing something wrong, missing
something?
Is there any way to get close to raw network throughput?

Thanks a lot! Best regards,
Henning
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.tinc-vpn.org/pipermail/tinc/attachments/20140415/7d8d2ac4/attachment.html>