~dr|z3d
@RN
@StormyCloud
@T3s|4
@T3s|4_
@eyedeekay
@orignal
@postman
@zzz
%Liorar
+Afkaid
+FreefallHeavens
+Unbur
+Xeha
+cumlord
+goose2_
+j6
+poriori
+profetikla
+r00tobo
+snex
+uop23ip
+weko
An0nm0n
Arch
Danny
DeltaOreo
DiCEy1904
Irc2PGuest11229
Irc2PGuest21326
Irc2PGuest48909
Irc2PGuest54864
Irc2PGuest71250
Nausicaa
Onn4l7h
acetone_
anon2
anontor
anu3
bak83
boonst
enoxa
fujifilm
goose2
itsjustme
mareki2pb
onon_1
qend-irc2p
shiver_
tr
u5657
user
veiledwizard
xen_NULL
eyedeekay
weko re: proposal status still undecided. I think it's a good idea but we'll need time to discuss it.
weko
eyedeekay: okay I'll not write anything
dr|z3d
I've been giving snark's single destination limitation some thought recently.
dr|z3d
If snark didn't create tunnels and its own dest, and delegated that, then it's possible multiple dests could be nominated to handle traffic.
dr|z3d
Different dests for inbound and outbound traffic, non-persistent destinations per torrent, dests that automatically cycle, that sort of thing.
weko
I hope final (from me) version
weko
zzz, please edit meta/content if needed and add to proposals.
weko
sorry. final final version:
weko
Markdown, if somebody needed (can be not the same):
dr|z3d
have you got that in esperanto, weko?
weko
dr|z3d: idk esperanto... but want.
weko
If you about esperanto version readme in one of my git repos.... So it's translator as I remember.
weko
I'm already tires of doing this proposal
weko
tired
dr|z3d
I was joking, weko. I have no interest in esperanto :)
dr|z3d
esperanto's about as useful as a marzipan dildo, and much less edible.
weko
dr|z3d: oh missunderstood joke
weko
didn't understand*
dr|z3d
it's ok, weko, obscure joke :)
weko
Jokes direct understand typically for me
not_bob
I second the idea of dynamic non-persistant i2psnark dests per torrent. It would be better for privacy.
not_bob
orignal: I sent you a message a few days ago about having option in the tunnels.conf to allow idle tunnels to auto-close and re-open later if traffic is detected.
not_bob
orignal: Also, clearnet outproxying does not seem to work if I make a second http tunnel in my tunnels.conf. Thoughts?
not_bob
orignal: The goal is to get a new dest when the tunnel re-opens.
dr|z3d
support for multiple outproxies would be handy too
orignal
multiple outproxy? how?
orignal
I mean how do we select one
dr|z3d
you have a list, choose one at random until it fails, choose another thereafter.
dr|z3d
or implement your own selection strategy.
dr|z3d
you might designate one as primary, and the others as secondary, or primary, secondary, tertiary etc, and select in that order.
dr|z3d
extra points if you actually test a clearnet address with the nominated proxy to check that it's working before deciding to use it.
dr|z3d
I was trying to explain to zzz a while back the usefulness of this particular strategy. not good enough to just check that the b32 responds.
orignal
so round-robin?
orignal
another option is balance
dr|z3d
sure round-robin, balance, random.
dr|z3d
if you're opting for balance, then you're talking about using more than one at a time.
dr|z3d
which isn't necessarily a bad idea, but you'd want an option to cluster certain requests/hosts to a single outproxy in some instances, otherwise you'll get cross-site issues and wotnot.
dr|z3d
zzz: those new sidebar notifications for the news feed, are they specific to router updates, or do they trigger for all news feed updates (ie blocklists) ?
zzz
not sure, suggest you look at the diff
dr|z3d
needs a bit of a buff up, either way.
zzz
whats the buff?
dr|z3d
putting something useful up there other than a b32.
dr|z3d
anything other than the b32 would be better.
zzz
we could do a reverse lookup, MRs welcome
dr|z3d
not entirely sure by looking at the diff if non-news updates are causing notifications.
dr|z3d
if (!langChanged()) {
dr|z3d
long lastMod = NewsHelper.lastUpdated(ctx);
dr|z3d
if (lastMod > 0)
dr|z3d
_lastModified = RFC822Date.to822Date(lastMod);
dr|z3d
}
dr|z3d
a reverse lookup to get the hostname you mean?
zzz
yes
dr|z3d
is there even a hostname for the news feed?
zzz
not sure
zzz
the langchanged stuff is just to reset the last-mod after you change languages so you get the new one, which I snuck into the same checkin iirc
dr|z3d
if there is, it should be specified in the configs as the feed, not the b32, and added to the default hosts list. if there isn't, then it should be registered and then used. but I was thinking more about displaying something useful there to notify the user.
dr|z3d
if the blocklists get updated via the feed, then what about telling the user they've been updated. "20 new entries blocklist entries via foo.i2p" or whatever.
dr|z3d
if it's a news item, "new news item by idk downloaded" or whatever.
dr|z3d
right, my bad.
dr|z3d
if (_failMsg != null) {
dr|z3d
// from checkForUpdates()
dr|z3d
_mgr.notifyComplete(this, "<b>" + _failMsg + "</b>");
dr|z3d
} else if (_showStatus) {
dr|z3d
if (status == 200)
dr|z3d
_mgr.notifyComplete(this, "News updated from " + _currentURI.getHost());
dr|z3d
else
dr|z3d
_mgr.notifyComplete(this, "No new news available from " + _currentURI.getHost());
dr|z3d
}
zzz
happy buffing, MRs welcome ))
dr|z3d
I'm not convinced of the utility of it right now. Maybe I'll get it.
zzz
the two news servers are hardcoded b32s, not hostnames, to ensure they work no matter what addressbook shenanigans or downstream packaging things happen
dr|z3d
well here's an idea. why not hardcode them with a reserved hostname, there are a few available.
dr|z3d
news.router.i2p or whatever.
zzz
I thought I did what you guys were asking for, but apparently I missed the mark? Still not sure quite what the complaint is
dr|z3d
this is different. the unsigned update notification is great.
dr|z3d
I thought you were going to replicate that for signed dev updates, but you went off piste :)
zzz
I thought I stayed right on the piste
zzz
I adapted the unsigned stuff over to the news fetcher
dr|z3d
yeah, I see that. but we already have a sidebar notification system, persistent, that indicates the currently available news. if the transient notification indicated *what* was updated (news/blocklists) with a bit more info like the published date, author or number of entries, then it might get a pass. :)
dr|z3d
and we also have persistent info on /configupdate that indicates when the news was last updated.. so just showing that the news was updated in the sidebar is redundant. that's my point.
zzz
my understanding of the request was 'show more info in sidebar on results after clicking manual check-for-update, rather than just 'no new update available'
zzz
in particular, failure info
zzz
did not hear a request for 'tell me exactly what components updated on success'
dr|z3d
yeah, that's broadly correct, but relating specifically to router updates, unsigned and otherwise, at least in terms of mesh's original moan/request.
zzz
so maybe that's the difference between my piste and yours?
dr|z3d
could be :)
zzz
it's also possible that you're disagreeing with whomever made the request (mesh?)
dr|z3d
no, I think it was a valid request to inform the user when connecting to the router update server failed, that was the original request in essence.
dr|z3d
because before that it indicated "no update available" which was potentially misleading.
weko
Clearnet access should be done with different software, as I think. And all features (multiple proxies etc) should be in that software.
weko
Or like plugin...
dr|z3d
weko, the problem with i2pd is that you can only specify a single outproxy.
weko
dr|z3d: i2pd should not support outproxy ideally.
weko
It's can be done with other software
weko
zzz: did you add proposal?
zzz
yes see link above
weko
Nice
weko
Thanks
dr|z3d
weko: you can't use i2p's outproxies with other software.
zzz
np. if you have updates please submit as a diff vs. the file in github, which I did have to tweak some
weko
dr|z3d: I can and I use
dr|z3d
no, not i2p's proxies. you need i2p to access them.
dr|z3d
and ideally i2p would have a huge number to choose from, not none.
weko
acetone outproxy exists for minimum
weko
Not really "none"
dr|z3d
well if that's available via clearnet, great.
weko
I done tunnel for outproxy and connecting to clearnet IRC servers with setup this tunnel address port as proxy in irc client
weko
Don't need really "outproxy" feature in i2p router
weko
really need*
dr|z3d
but that misses the point. and I'm not saying we don't have any, I'm countering your argument that "ideally i2p should not support outproxy".
weko
dr|z3d: I was wrong. Meant "i2p router"
weko
Not i2p at all
weko
Because i2p support any connections. And connections for clearnet proxy also supported anyway
dr|z3d
if I patch, I'll let you know, zzz, but my inclination right now is to remove the news related notification.
dr|z3d
the more you say, weko, the more confused I am.
dr|z3d
java i2p at least supports multiple outproxies, and on a single tunnel.
zzz
would be nice to get feedback from mesh on it; I don't know how to reconcile your piste with his really
dr|z3d
i2pd apparently only supports 1.
weko
dr|z3d: ye I said ideally i2pd should not support outproxy at all
dr|z3d
yeah, that boat has sailed, zzz, I prompted him months ago to offer something by way of feedback for your work, and he didn't.
weko
Because unix philosophy
dr|z3d
more "chatty" notifications for (un)signed dev router updates, great. perfect. that's what he asked for, that's what you delivered. perfect.
zzz
most of the changes only kick in if you click check-f0r-update manually which I doubt people do often, so imho not worth a lot of drama either way
dr|z3d
sure, weko, sure, you can create a bunch of tunnels to different outproxies and knit them all together with a backend, but that's not something every user wants to do. some users just want a one size fits all single port they can proxy all their traffic through, .i2p or clearnet.
dr|z3d
true, zzz. minor.
weko
dr|z3d: can be done with one proxy at userside, one destination in i2pd with software
weko
And with multiple outproxies*
dr|z3d
I only mentioned it because, having not seen it until earlier today, it was brought to my attention by someone who had no idea what it was. but yeah, minor.
dr|z3d
you're adding a huge amount of complexity for something that shouldn't require it, weko.
weko
dr|z3d: you do same with java i2p
dr|z3d
no, in java it's simple. you nominate 1 or more outproxies, and that's it. java randomly chooses one.
dr|z3d
java will check the dest is up or cycle to another.
dr|z3d
if java checked a clearnet endpoint to validate the proxy was up, would be even better. but still, it's streets ahead of i2pd.
zzz
the generic argument that 'java i2p has x feature, i2pd should do it too' rarely if ever gets any traction, ditto vice versa
zzz
it's just not compelling on its own, and it also (if regularly adopted) stomps out product differentiation
dr|z3d
there's nothing generic going on here, it's more a specific deficit that i2pd has that's been noted. whether orignal thinks it's worth considering is another matter.
dr|z3d
as for java, with multiple outproxy support, we can fill in the gaps with software without the same level of complexity required to get i2pd working with multiple outproxies.
dr|z3d
but there's always room for improvement, regardless of the implementation. a high tide lifts all boats..
zzz
sure
dr|z3d
have you thought any more about rate limiting strategy for keep-alive?
zzz
point is that requests from users, rather than from fellow devs pointing out feature deltas, rightly get more attention
zzz
no good ideas yet
dr|z3d
notbob made the request, I just puffed it up :)
zzz
once a feature is put on the list, that's the time to eyeball the other guy to see how he did it
dr|z3d
so, rate limiting. max connections per client is one way to keep things sane.
dr|z3d
concurrency, in other words.
dr|z3d
buffers could be another way to implement some sort of throttling.
zzz
that doesn't do anything if you can make 10k requests per minute on one connection
dr|z3d
you know, instead of max requests per minute, clients are allocated a buffer instead.
zzz
what are we buffering?
dr|z3d
the amount of transferred data. buffers, or buckets..
dr|z3d
and/or the number of requests made.
dr|z3d
not so much requests, hard to peer into the keep-alive connection to identify those. but data transferred in a given period might be something to consider, with a queueing/burst strategy.
zzz
don't start with implementation. start with what do we limit and where in the protocol stack. and/or how/where do we repair the current limiter that will be bypassed via persistence
dr|z3d
throttling would be at the application layer, no?
dr|z3d
ie where we do it now, more or less.
dr|z3d
and repair is what I'm suggesting, instead of throttling by # of requests, we throttle by data transfer instead.
dr|z3d
so instead of 50 requests/m, we throttle at 1MB/m or whatever seems suitable given the server context.
dr|z3d
the problem with either implementation is that they can be trivially bypassed with a bit of effort.
zzz
the layer choice is streaming vs i2ptunnel
zzz
the counters are in streaming now
zzz
but only i2ptunnel knows about add'l requests on same conn
dr|z3d
probably a combination of both. i2ptunnel retains its ability to throttle based on requests, streaming handles data transfer?
zzz
so the options are 1) add counters to i2ptunnel; 2) use accessfilter in i2ptunnel; 3) add something to socketmgr API that i2ptunnel can call, e.g. gotANotherRequest() to increment the counter and accept/rej
zzz
any layer can and does reject
zzz
but i2ptunnel can't get to streaming's counters atm
dr|z3d
ok. so we keep the connection throttler broadly the same.
dr|z3d
it serves a purpose, will carry on serving a basic purpose.
dr|z3d
the filter gets beefed up to handle data rates.
dr|z3d
does that sound vaguely coherent and viable?
zzz
not really ((
zzz
now we're counting "data rates" instead of requests?
dr|z3d
data transfered per defined window.
dr|z3d
which of course in the filter can be much more fine-grained.
zzz
'count something different instead of requests' is orthogonal to 'fix counting requests'
dr|z3d
that's not the premise. the premise is "how do we throttle clients if not by requests?"
dr|z3d
counting requests within a single keep-alive connection, forget it.
dr|z3d
unless you want to implement some sort of java wireshark feature. which you don't. :)
zzz
so far, I'm thinking of 'fix counting requests', and I laid out 3 options above to do that
zzz
per-far-end counters of anything are relatively expensive, bw estimators even more so
dr|z3d
not easy. the problem is that with keep-alive, counting requests is mostly a meaningless metric.
dr|z3d
(as you pointed out yourself)
dr|z3d
so buckets or buffers would be one way to mitigate that. once you've spent your allocated buffer, you're in a holding pattern until the next window.
dr|z3d
or you use the allocated bucket and window size to determine the data rate.
zzz
we already have bw estimation in streaming, but there's no configurable throttle there, and no aggregation of all conns to same dest
dr|z3d
maybe lose the connection throttler altogether and provide a ui an i2ptunnel hookup for the tunnel filter is the way to go, seeing as there's plenty of overlap there.
zzz
sounds like a lot of work
dr|z3d
you mentioned possible url-based filtering using the access filter, so you've already sort of considered it. but yeah, it's not a trivial fix.
zzz
anyway, all theoretical at this point, need two MRs approved before we get there
dr|z3d
how far are we away from the server end of the keep-alive deal?
zzz
client-end merge + a few days
dr|z3d
roger that.
dr|z3d
how's it all looking from your seat?
zzz
+ server end review/merge
zzz
maybe still a little flaky?
dr|z3d
still git.idk issues or those resolved now?
zzz
seems ok