#ls2 (irc2p) | #ls2 I2P Protocol Development Discussion - Next Meeting Monday, October 9th 6:30 UTC

R4SAS in mid of december i2pd was added to MSYS2 distribution by user outside of our team. If someone used it from here, there was incorrect configuration with disabling network state verifyer (USE_MESHNET=yes for make)

R4SAS so I router incorrectly set R cap to tlags

orignal my question is

orignal does every i2pd routers fail?

R4SAS so router **

zlatinb orignal: no, only some 0.9.52s, mostly in .ru from my experience

zlatinb Xfr caps

zzz there's still a few "random" causes possible - the router goes down, or it hits connection limits

zzz so the data's never going to be 100%

zzz my last test was expl. tunnels configured for 1-hop outbound, 0-hop inbound, I think 5 tunnels

zzz run for 24 hours

zzz and then add up all routers with at least one test failure

zzz 129 total routers failed. 36 java, 93 i2pd, 24 were in RU

zzz so at this point, I think it's an i2pd problem

zzz what do you mean 'how do you build'?

orignal "some" what's the percentage?

zzz 93 i2pd routers failed at least once, out of about 1450 i2pd routers in my netdb on average

zzz but I have no idea how many routers were built through in 24 hours

orignal I trid to build 3 hops tunnel through my routers

orignal no problems at all

orignal that's whY I'm confused

orignal R4SAS it might another problem

zzz could it be a crypto problem or openssl 3?

orignal build for 21.10

orignal opessl 3 exists in Fedora only

orignal and I tried

orignal no probems with it

R4SAS in archlinux too

R4SAS but not defaulted to it yet

zzz since these are 1-hop, almost all 0.9.52, it's STBM

orignal what was the real problem

orignal broken build for 21.10

orignal that was rebuilt and fixed later

orignal how about VTBM?

orignal can you test?

zlatinb I tested VTBM, same behavior

zlatinb with that hash I found last week

orignal and it works for me wtih VTBM

zzz I can also post a list of router hashes if that helps

zzz or I can test with 0-hop OB, 1-hop IB

zzz orignal, can you test the broken 21.10 build to see if it drops tunnel messages?

orignal no, it just didn't work

orignal for us

orignal but maybe it worked for somebody else

zzz so, in summary, I'm out of ideas, I think it's an i2pd problem, and if you can't figure it out in the next 4 weeks I don't know what I need to do for the release

orignal zzz however the source of problem is not this

zzz ok

orignal the problem is

orignal 1. i2pd doesn't see this problem.

orignal 2. if it's around 100 routers, any advesary can put the network down

orignal by doing it intentionally

zlatinb agreed

zzz can I give you a list of routers to test with?

orignal yes please

orignal I will see

orignal give me few

zlatinb but tunnel testing works around problem 2

zlatinb so maybe there's no need to do anything else?

orignal that's what I meant

zzz tunnel testing makes it better but is not a magic fix

orignal if tunnel testing solves the problem just change profiling

orignal also I can say there is no problem with i2pd in trunk or relese

orignal but there are also many forks

zzz the thing is, we started seeing the problem right after the .52 release, and it steadily got worse

orignal ofc

orignal because you swiches to STBM

orignal in 0.9.52

orignal no?

orignal Tunnel creation success rate: 69%

orignal I can't see it gets worse

zzz yes I switched in 0.9.52, but will use STBM for routers >= 51

orignal that's what happened

orignal you build a lot of STBM since 0.9.52

orignal that's why you see it

zzz I don't follow your reasoning

zzz but I can do a VTBM test if you like. zlatinb said it didn't matter

orignal becuse failed routers works fine with VTBM

orignal and data goes through

orignal please do

orignal because VTBM works for me

zzz so you did testing where STBM failed and VTBM passed?

orignal even with failed routers

orignal yes, I did

orignal that's my pint

zzz so you DO see the problem

orignal only STBM causes that problem

orignal yes, I do

zzz <orignal> 1. i2pd doesn't see this problem.

orignal with that router provided by zlatin

orignal i2pd just doesn't care

orignal tunnel failed

orignal we build another one

orignal and not huge amount of failed tunnels

zzz it still hurts reliability. Let's not say "doesn't care".

orignal everything works as ecpected

zzz this is a major problem

zzz and tunnel testing isn't free

orignal major problem where?

zzz major problem with the reliability of the network

orignal I can't control every single fork

zzz major enough for me to work my ASS off trying to find the root cause for two months

zzz let's not assume it's a fork. maybe it is, maybe not, but you can't assume that and say you're not going to investigate

orignal I can't iverstigate because the code in trunk works properly

orignal what and how should I investiagte?

zzz take all the info, go look for possible problems. I don't know your code, how can I give you advice?

zzz this can't just be that you give me more work to do, more things to test. I need help

orignal I will try few more scenrios

zlatinb I'm happy to help with a mixed i2pd+java LXC testnet if anyone is interested

orignal zlatinb good point

orignal try to reproduce the problem there

orignal because I can't

zlatinb ok, over the coming week I'll build from the (which?) tag

R4SAS 59%, 68%, 83%, 47%, 56%, 52%, 60%, 67%, 48%, 52%, 76% - success rate of my routers

zlatinb and maybe try to reproduce with a broken build too

zzz this has nothing to do with build success R4SAS

zzz it's about tunnels that don't work after being built

orignal zzz it does

zzz why?

orignal when you try to build OB

R4SAS failed tunnels counted too, as I know

orignal you must specify IB

orignal and if IB is bad OB will fail

orignal so we will see tunel build ratio degradation

zlatinb orignal & zzz: there's difference in behavior between java and i2pd which means there's difference in the crypto somewhere

zlatinb let's focus on that

zzz ok

zzz I'll post a list of hashes shortly

zzz anything else on 2) ?

orignal But I also saw the problem with that router

orignal no

zzz 3) ntcp2 clock skew

zzz I implemented all the recommendations from last week

zzz also, I reported that on my ff I see about 180 skewed connections per hour

orignal as we agreed we will adjust clock from SSU

zzz I assume they're mostly i2pd

orignal ofc they are

orignal if time difference more than 15 sec we adjust

zzz so I recommend you enable NTP by default

orignal we don't want to

orignal it's leak

R4SAS We will do it other way

orignal as advesary can regnize i2p by that ntp packets

zzz funny, I thought you told us that using DNSoverHTTPS for NTP server lookup was a waste of time? but maybe misremember

R4SAS on peer test we will check offset with local time, if it more than 15 seconds, we will switch "adjust" flag and apply diff local-external

zzz however you do it, it's a huge number of routers out there that can't work at all because of clock skew, it needs to be fixed

orignal no I didn't suugest it

orignal zzz many routers uses system daemon for it

R4SAS next few peer during peer test will be used to calculate average difference and it will be applied

orignal and mobile clients use network time

orignal as I said, adjust with SSU during initial peer test should eliminate the problem

zzz ok

zzz anything else on 3?

zzz anything else for the meeting?

R4SAS no

orignal basically we use ntp sync explicitly on OpenVZ VPS only

eyedeekay no

orignal where you can't adjust system clock

orignal please gimme the list

orignal and I will test

zzz I'll have the router list in 10 minutes

orignal thanks

orignal I will give you the stats with STBM and VTBM

zzz stats.i2p/docs/tunneltest.txt

orignal downloaded

orignal will let you know in few days what I see

zzz thanks

zzz will start a VTBM test shortly

orignal no mine among them