<div dir="ltr">We have the same network topology of 6 hub nodes and 30+ leaf nodes.<div>We also suffering from periodic network blockage and very high network usage of ~300mb per node a day (50mb of upload and 250 of download).<br>This high traffic usage happens on idle network.</div><div>We have 1.0.34 version on our nodes.<br><br><div class="gmail_quote"><div dir="ltr">On Tue, Dec 11, 2018 at 12:52 PM Amit Lianson <<a href="mailto:lthmod@gmail.com">lthmod@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello,<br>
We're suffering from sporadic network blockage(read: unable to ping<br>
other nodes) with 1.1-pre17. Before upgrading to the 1.1-pre release,<br>
the same network blockage also manifested itself in a pure 1.0.33<br>
network.<br>
<br>
The log shows that there are a lot of "Got ADD_EDGE from nodeX<br>
(192.168.0.1 port 655) which does not match existing entry" and it<br>
turns out that the mismatches were cuased by different weight received<br>
by add_edge_h().<br>
<br>
This network is consists of ~4 hub nodes and 50+ leaf nodes. Sample<br>
hub config:<br>
Name = hub1<br>
ConnectTo = hub2<br>
ConnectTo = hub3<br>
ConnectTo = hub4<br>
<br>
Leaf looks like:<br>
Name = node1<br>
ConnectTo = hub1<br>
ConnectTo = hub2<br>
ConnectTo = hub3<br>
ConnectTo = hub4<br>
<br>
Back to the days of pure 1.0.33 nodes, if the network suddenly<br>
fails(users will see tincd CPU usage goes 50%+ and unable to get ping<br>
response from the other nodes), we can simply shutdown the hub nodes,<br>
wait for a few minutes and then restart the hub nodes to get the<br>
network back to normal; however, 1.1-pre release seems to autoconnect<br>
to non-hub hosts based on the information found in /etc/tinc/hosts, which<br>
means that the hub-restarting trick won't work. Additionally, apart<br>
from high CPU usage, 1.1-pre tincd also starts hogging memory until<br>
Linux OOM kills the process(memory leakage perhaps?).<br>
<br>
Given that many of our leaf nodes are behind NAT thus there's no<br>
direct connection to them expect tinc tunnel, I'm wondering about if<br>
there's any way to bring the network back to work without shutting<br>
down all nodes? Moreover, is there any better way to pin-point the<br>
offending nodes that introduced this symptom?<br>
<br>
Thanks,<br>
A.<br>
_______________________________________________<br>
tinc mailing list<br>
<a href="mailto:tinc@tinc-vpn.org" target="_blank">tinc@tinc-vpn.org</a><br>
<a href="https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc" rel="noreferrer" target="_blank">https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><b>Vitaly Gorodetsky</b><br><font color="#666666">Infrastructure Lead</font></div><div dir="ltr" style="font-size:12.8px"><font color="#666666"><br></font><font color="#666666" style="color:rgb(136,136,136)">Mobile: <a value="+972546616205" style="color:rgb(17,85,204)">+972-52-6420530</a></font></div><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><a href="mailto:vgorodetsky@augury.com" style="color:rgb(17,85,204)" target="_blank">vgorodetsky@augury.com</a></div><span style="color:rgb(136,136,136);font-size:12.8px"><div dir="ltr"><br></div><div dir="ltr"><img src="http://www.augury.com/assets/images/augury_email_sign_logo.png" width="96" height="49"></div><div dir="ltr"><font size="1" color="#666666">39 Haatzmaut St., 1st Floor, </font></div></span><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><font size="1" color="#666666">Haifa, 3303320, Israel</font></div></div></div></div></div>