MTU probes fail on reconnect

[Donald Pearson]

I've noticed some inconsistent performance with some of my tunnels and
thought I would take some of the spare free time I have over the holidays to
try to figure out what the cause of that may be.  My environment in this
case is my home LAN.

Please forgive my use of the terms "server" and "client" in this email, I
only use these terms to simply explanation.

I statically assigned my server with an ip of 10.10.10.1, and my client is
set to 10.10.10.2.    The rest of my LAN uses 192.168.2.0/24, so in this
case I am using Tinc to create a tunnel to access the 192.168.2.0/24 network
from my client.  This is all on common switch fabric, no in-between
firewalls of any kind involved, and no firewalls configured on either Server
or Client.

On the Server, Tinc is running on stripped down Centos 5.5 as a virtual
machine and all numbers given here are in this configuration.

I have also tested this on a normal Centos 5.5 install, as well as Ubuntu
9.04, 9.10, 10.04, and 10.10.  All with and without vmware tools installed.
 Although there are performance differences observed between the different
builds, the behavior I describe has been the same on all builds.   The only
thing I haven't tested is a native OS install.

Tinc is configured in switch mode.
The server virtual adapter is bridged to the physical adapter using brctl.
 The client receives an address on the 192.168.2.0/24 network via DHCP from
my internet router.

ifconfig of Tinc "server"

br0       Link encap:Ethernet  HWaddr 00:0C:29:58:B5:6B
          inet addr:192.168.2.4  Bcast:192.168.255.255  Mask:255.255.0.0
          inet6 addr: fe80::20c:29ff:fe58:b56b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21384 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23987 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8452737 (8.0 MiB)  TX bytes:23819155 (22.7 MiB)

br0:0     Link encap:Ethernet  HWaddr 00:0C:29:58:B5:6B
          inet addr:10.10.10.1  Bcast:10.10.10.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0      Link encap:Ethernet  HWaddr 00:0C:29:58:B5:6B
          inet6 addr: fe80::20c:29ff:fe58:b56b/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:196191 errors:0 dropped:0 overruns:0 frame:0
          TX packets:38068 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:145802768 (139.0 MiB)  TX bytes:28914683 (27.5 MiB)
          Interrupt:177 Base address:0x1424

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:222 errors:0 dropped:0 overruns:0 frame:0
          TX packets:222 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:126456 (123.4 KiB)  TX bytes:126456 (123.4 KiB)

vpn       Link encap:Ethernet  HWaddr FE:42:68:39:D9:1F
          inet6 addr: fe80::fc42:68ff:fe39:d91f/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:13890 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22405 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:5055429 (4.8 MiB)  TX bytes:21399229 (20.4 MiB)


[root at localhost ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.000c2958b56b       no              vpn
                                                                       eth0



ipconfig of windows xp "client"

Windows IP Configuration


Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . : local
   IP Address. . . . . . . . . . . . . . . . : 10.10.10.2
   Subnet Mask . . . . . . . . . . . . . . : 255.0.0.0
   Default Gateway . . . . . . . . . . . . :

Ethernet adapter Wireless Network Connection 2:

   Media State . . . . . . . . . . . : Media disconnected

Ethernet adapter Tinc:

   Connection-specific DNS Suffix  . : local
   IP Address. . . . . . . . . . . . . : 192.168.2.246
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 192.168.2.1


What I've discovered using level 5 debugging is that often when a connection
is made, MTU probes from the client are not responded to.

The tell-tail sign I've seen every time is particularly high latency.

I've been able to reproduce the condition not every, but nearly every time,
if I manually start the client (windows xp client) in a command prompt.
Press Ctrl+c to stop the client, and then restart it after approximately 5
seconds.

The client will print the message "No response to MTU probes from Server"

And then basically all traffic from then point on carries the message
"Packet for Server (10.10.10.1 port 8002) larger than minimum MTU,
forwarding via TCP"