asciilifeform: this ^ is simple means of testing the logger for folx not yet ready to stand up bot. e.g. : zcat dump.gz | psql -U nsabot -d nsalog will eat (assuming you already inited db w/ the provided schema)
asciilifeform: if folx want to mirror, i'd prefer that also happen daily, otherwise no amt of pipe will suffice
asciilifeform: i expect will be roughly 3x that when all the back-archives added.
asciilifeform: it is even possible that the pipe is to blame, i.e. the measurement is 'random' depending on how congested the pipe at piz on acct of e.g. the multiple trb noads sitting there
asciilifeform has nfi why the # given for 'all' vs the 1st 'resource' could differ
asciilifeform: or hm, possibly misread, and it's 0.737s
asciilifeform: aint nginx, apache, etc supposed to ~cure~ impedance mismatches, rather than cause'em...
asciilifeform: if this box weren't also running phuctor, could simply hang the logger on port 80 and get, theoretically, 200-300msec loads, loox like.
asciilifeform will try on various FF later today. other folx welcome to report theirs.
asciilifeform: this thing dun seem to distinguish the phases of the load tho (only distinguished load from rendering, and cuts the latter into phases. which makes a kind of sense, tool is evidently for debug of 'slow cuz hands of who wrote html-shitter grow from arse' scenarios )
asciilifeform: it aint best possible pipe, but pretty good; and , as i pointed out already, effect persists ~without~ it ( i.e. operating curl within dulap shaves only 200msec vs to my chair; and has same peculiar 'burst' pattern and wild variance in total time )
asciilifeform: https://tools.pingdom.com/#5b22d518dfc00000 << this one confirms my init observ., almost whole second in 'connect' + 'wait' ( and 5 in receive , doesn't say in what pattern, but visually curl in the 'slow even on local shell' variant appears 'bursty' )
asciilifeform: iirc chromisms have timer, i have box w/ one
asciilifeform: pretty certain that the pipe itself is blameless, i get consistent realtime work through it, not once long pings much less subjective feel of 'mars probe'
asciilifeform: tried large buffers, no buffers, various tcpism knobs, no obv culprit for the impedance mismatch (what i suspect this is )
asciilifeform: i found no obvious cause, but in process of knob turning combinatorics cut the delay somewhat ( seems to only be detectable in graphic wwwtrons tho, and , annoyingly, cannot actually get useful number from those, unlike curl )
asciilifeform: and in fact accts for all but ~70msec of a heavy pg
asciilifeform: mp_en_viaje: possibly i did not elaborate , but on box's own shell, if viewing through ngx , fully 1-2s moar delay than if direct to port where py proggy
asciilifeform: ( after upped buffer on box and reenabled gzipism )
asciilifeform: something is odd, i sat all evening last night reading vintage #t logz via my displayer and none ate >3s
asciilifeform squeezed out a bit of the delay via knob-turning but this is still ridiculous
asciilifeform: ( and if cures, move dulap entirely to classic apache. back in the old days went w/ 'nginx' because slightly moar miserly re memory )
asciilifeform: i'ma have to set whole orchestra up on a box w/ classic apache, to see whether same thing
asciilifeform: reason why never noticed this effect with phuctor, is that its pages are small
asciilifeform: cachism is apparently red herring, the proggy is actually fast enuff. there is parasitism in the loop.
asciilifeform: approx 0.2s is eaten by transmission on net from BingoBoingostan to my chair; the rest is eaten by the www server somehow
asciilifeform: actual generation of e.g. current /log, takes 0.073s !
asciilifeform: err, nginx there, presently. interestingly, set up as experiment 'wsgi', python's equiv. of php's 'mod-php', and made ~no measurable diff !
asciilifeform: ... 100% of the time diff, apparently, in response req time
asciilifeform: dulap aint currently swamped with loads, either
asciilifeform: soo, in strange findings: on dulap, 'time curl http://logs.nosuchlabs.com/log' yields 2.342s; ( from my chair, same -- 2.455s ); but on identical (3m ago) clone of db , running locally, on substantially slower box, 0.252s !
asciilifeform: dijkstra spent whole life killing trees on subj of deadlocking etc. but the subj is deep enuff for 9000 dijkstras.
asciilifeform: ( and typically sooner than you think )
asciilifeform: this is actually the interesting thing about multiprocesstronic proggies. as alluded to by spyked . the improbable -- is certain to eventually happen.
asciilifeform: i thought this also, when had pre-PG phuctor. turns out ~nothing is measurably > nothing, lol
asciilifeform: mp_en_viaje: re 'delete file', it isn't quite a simple, cuz fs ops are potentially locking. i elided what means 'wipe the cache' for brevity, but there's a knob.
asciilifeform: already this thing is eating surprising amt of hand-cranking
asciilifeform dislikes pyistic libism , it spirals into insanity pretty quickly and makes testing / swaps of proggy on live box complicated
asciilifeform: that way reduce to 1 proggy, and all of the shared routines unduped.
