tinc 1.1pre10 slower than tinc 1.0, experimentalProtocol even more

[Henning Verbeek]

Thanks for your very quick response!

On Tue, Apr 15, 2014 at 2:18 PM, Guus Sliepen <guus at tinc-vpn.org> wrote:
> > * not specificing ECDSA keys, throughput is around 600Mbit/s; both tincd
> > are at around 90-95% CPU
>
> That is cause for concern, since it should then just use the old protocol with
> exactly the same behaviour as tinc 1.0. There are however some other changes in
> tinc 1.1 which might cause throughput to be lowered, I'll have a look at that.

I'll also try to retest with tinc 1.0 and report back.

> > sptps_speed reports:
> > Generating keys for 10 seconds:           5200.94 op/s
> > ECDSA sign for 10 seconds:                3710.72 op/s
> > ECDSA verify for 10 seconds:              1734.05 op/s
> > ECDH for 10 seconds:                      1449.40 op/s
> > SPTPS/TCP authenticate for 10 seconds:     641.37 op/s
> > SPTPS/TCP transmit for 10 seconds:           8.45 Gbit/s
> > SPTPS/UDP authenticate for 10 seconds:     640.18 op/s
> > SPTPS/UDP transmit for 10 seconds:           8.64 Gbit/s
>
> That's the fastest I've seen so far, but not unexpected of one of the fastest
> Haswell processors available. Note that 1.1pre11 will not use AES-GCM anymore,
> but will instead use ChaCha-Poly1305, which will lower the throughput on your
> processor to about 2 Gbit/s (you can try out the latest commit from the 1.1
> branch of tinc's git repository, and run sptps_speed again). However, it will
> be much faster on processors which do not have AESNI.

Here's the output for sptps_speed on ChaCha-Poly1305. As you
predicted, the throughput is lower:
Generating keys for 10 seconds:          24501.85 op/s
ECDSA sign for 10 seconds:               23853.56 op/s
ECDSA verify for 10 seconds:              9375.22 op/s
ECDH for 10 seconds:                      7048.03 op/s
SPTPS/TCP authenticate for 10 seconds:    3314.79 op/s
SPTPS/TCP transmit for 10 seconds:           2.53 Gbit/s
SPTPS/UDP authenticate for 10 seconds:    3252.67 op/s
SPTPS/UDP transmit for 10 seconds:           2.52 Gbit/s


With ChaCha-Poly1305 on SPTPS, we're now seeing between 300Mbit/s and
320Mbit/s only. Again both tincd's are fully CPU bound.
'tinc info <node>' reports:
Node:         riak_ankara
Address:      <external IP> port 656
Online since: 2014-04-16 21:39:08
Status:       validkey visited reachable sptps udp_confirmed
Options:      pmtu_discovery clamp_mss
Protocol:     17.3
Reachability: directly with UDP
PMTU:         1451
Edges:        riak_belfast
Subnets:      <internal subnet>

Everything here as it should be?

> Uh, are you sure they are running on the same UDP port? Normally, only tinc
> daemon can use a given port. If you did manage to run them on the same port,
> then it would indeed cause problems.

Sorry, my bad here. I misread the config. The two tinc daemons run on
separate ports. All good *phew* :)

> I do hope that when all issues have been resolved and tinc 1.1.0 can be
> released, actual throughput is much closer to the throughput measured by
> sptps_speed. Also, at the moment, both tinc and sptps_speed are single-threaded, so
> on a multi-core machine the throughput could in principle be multiplied by the
> number of cores, however that only makes sense if the encryption and
> authentication themselves are the bottleneck.

I'm struggling to comprehend this. If sptps_speed reports one value,
and what I measure through an actual sptps-tunnel is another, and in
both cases only a single core is used, what "ate up" all the
throughput? Is it the tun/tap handling as you suggested? Is it the
network device driver? Is it the latency of the actual packet over the
wire?

Thanks again!
Cheers, Henning