log☇︎
38200+ entries in 0.334s
asciilifeform: mircea_popescu is seeing it through the naive vertically integrating rockefeller eyes, 'power plant expensive? let's put it right in my mansion'
asciilifeform: mircea_popescu: if you think trb is a hard nut to crack, picture reading, grasping postgres.
a111: Logged on 2016-12-30 16:28 mircea_popescu: but he might be interested to hear about it.
asciilifeform: mircea_popescu: sql doesn't have a bignumatron
asciilifeform: mircea_popescu: algo ~demands~ O(1) random access to the bignums.
asciilifeform: mircea_popescu: it's a screaming nope
mircea_popescu: you can also set bgwriter_lru_maxpages to 0 and disable background writing altogether
mircea_popescu: eg bgwriter_delay you may want to be set low for this reason.
mircea_popescu: and work_mem should prolly be larger than 4mb, but hard to guess how large without a profile, and this is a major resource consumption ticket, so you actually want to do some maffs.
asciilifeform: mircea_popescu: 0 temp tables
asciilifeform: mircea_popescu: i'd first like to know what this'll do to integrity-on-mains-failure
mircea_popescu: 2) huge_pages should probably be on.
mircea_popescu: 1) shared_buffers is to be per spec "25% of available ram" ; but it does diminish returns in the gb. you probably have it as 128mb, make it 2gb say.
asciilifeform: mircea_popescu: probably. i'ma run with new knob settings as soon as it is safe to reset the db.
mircea_popescu: incidentally asciilifeform since we're now doing open source db optimization shared_buffers is probably a larger concern. what is it ? defaults to 125mb but i'd readily see it 1-4gb in your case.
a111: Logged on 2016-12-30 16:06 mircea_popescu: http://btcbase.org/log/2016-12-30#1593220 << depending on your setup about 40 to 60 days in the wild, about half with ben_vulpes recommended method.
a111: Logged on 2016-12-30 14:32 Framedragger: (do note, 'work_mem' is per user / per request. so may be easier to DoS. thought i should mention this for completeness)
mircea_popescu: http://btcbase.org/log/2016-12-30#1593220 << depending on your setup about 40 to 60 days in the wild, about half with ben_vulpes recommended method. ☝︎☟︎
a111: Logged on 2016-12-30 06:58 ben_vulpes: felipelalli, vexare, wyrdmantis, Sinclair6, luke-jr please fix your bouncers
mircea_popescu: (they did decide to move over to unity last year, then they abandoned the plan._
asciilifeform: mircea_popescu: presently i have nfi what part is published, or where
mircea_popescu views with mind's eye diana_coman 's beard growing inches/second in the minds of alf
asciilifeform: mircea_popescu, diana_coman : so you folx wrote own 'boost'-like horror? fwiw most games firms did, in the golden olden days. my brother's co, for instance, did.
mircea_popescu: diana_coman ^
asciilifeform: mircea_popescu: linux by default uses all spare ram as disk readcache
Framedragger: mircea_popescu: because you could tell postgres to flush rows (forcing all caching layers) every 100 rows, not every 1 row as currently specified
Framedragger: mircea_popescu: consider a scenario in which you knew how much data you could lose ("up to 100 last rows"), and you could check if you lost any (last row id == last-id-processed.txt ? false : true). that being said, this way things become more wibbly-wobbly, so probably fuck that. :(
Framedragger: "Note that open_sync writing is buggy on some platforms (such as Linux), and you should (as always) do plenty of tests under a heavy write load to make sure that you haven't made your system less stable with this change. Reliable Writes contains more information on this topic. " oh god. more inserts/sec but zero data loss => probably can't help you much. documentation doesn't encourage me :/
asciilifeform: mircea_popescu: tbh i had a 'what the hell is all this' reaction to reading ben_vulpes and phf vtron problemz
asciilifeform: mircea_popescu: that's a straight (and spam-encrusted) translation of an english article that was posted here ('against hardforks' iirc) a while back.
Framedragger: here's what i'm thinking: disable synchronous_commit , but set 'checkpoints' so that results are flushed to db every $n inserts/updates. i can see however how you may barf from such an idea, "it's either reliable, or isn't".
asciilifeform: mircea_popescu -- switched hosts, i -- slowly, painfully, rewrote the thing...
Framedragger: (do note, 'work_mem' is per user / per request. so may be easier to DoS. thought i should mention this for completeness) ☟︎
Framedragger: ah hm. tbh i'd still change work_mem because it's ridoinculously low by default, but i hear ya.
Framedragger: work_mem (used for in-memory sorts) is 4 MB default. 4 MB. (9.5 anyway). set it to 50 MB as per advisory at least.
Framedragger: mircea_popescu: it's dark here in the northern hemisphere, god it's deperessing :( mornin'..
asciilifeform: guten morgen mircea_popescu !
Framedragger: right. i just thought about checkpoint_completion_target (set to say 0.9) which may help with inserts, but ultimately you're right, physical reality
ben_vulpes: i believe that mod6 has a solid one as well, pete_dushenski's has been blackholed of late
davout: ben_vulpes: any suggestions for such a node?
mod6: https://weechat.org/files/doc/weechat_faq.en.html#filter_irc_join_part_quit
phf: ben_vulpes: you mean like ~parse~ the output of patch?
ben_vulpes: ^^ mircea_popescu asciilifeform mod6 trinque and any other vtronicists pls to opine
phf: ben_vulpes: where'd that 0 come from?
phf: ben_vulpes: well, we're kind of constrained by the hardcode -p1 behavior, but i've no idea if that's an implementation detail or a spec
asciilifeform: mircea_popescu: no, it is specifically 'doesn't work because ~nearly there is a cock-shaped shadow~
trinque: mircea_popescu: yep works now
trinque: mircea_popescu: ah you know what, excluding nicks with no ratings breaks this, because writing the keyfile happens inside that logic.
asciilifeform: ben_vulpes: 'this shape' being 'html as-seen-by-reader as the only storage format'
asciilifeform: trinque: mircea_popescu will probably barf if the 2 genesisen get displayed on same screen. it isn't an eggog in vtronics, but is in his head.
asciilifeform: the mention of pg_notify
trinque: ben_vulpes: and yes, app code needn't be anywhere near www
trinque: there's a listening process consuming these pg_notify events, of the form (gen-nick-page "trinque") which debounces them according to some sane interval, i.e. if a few updates hit the same user in a short span, it will result in one single rebuild
trinque: ben_vulpes and I had an interesting conversation yesterday about how to handle static sites generated from a db. idea we ended up with was that we'd have triggers which emit a pg_notify signal when the "dirty bit" has flipped for any page.
trinque: mircea_popescu: it'll be up there in a sec, let ya know
asciilifeform: ben_vulpes: not each (nixing the win32 #ifdefs did not, for so long as nobody is dumb enough to try to build for win32)
asciilifeform: ben_vulpes: semantics is , more or less, what the proggy ~is~
asciilifeform: ben_vulpes: same as in any other proggy
a111: Logged on 2016-12-29 23:32 ben_vulpes: http://btcbase.org/log/2016-12-28#1591573 << the diff line is distinct from the --- / +++ lines, does one ever see a patch file where the files compared aren't prefixed with a/ or b/ ?
asciilifeform: pete_dushenski: the only long-term answer is full wotnetization of the nodes.
asciilifeform: ben_vulpes: the 'a' and 'b' are historic artifacts from my torture room. but notice, gnudiff ignores the name.
a111: Logged on 2016-12-28 02:39 phf: http://btcbase.org/patches/hashes_and_errors#L118 you don't really want to do this. you're subseq'ing there to strip the a/ b/ but that's not at all a guarantee! i have a vpatch with `diff -ib -ruN /Users/pf/cmucl20d-build/src/hemlock/abbrev.lisp src/abbrev.lisp` in it for example. at the very least you want to abstract it away into its own function. that would correctly operate on a hashed-path datastructure.
asciilifeform: mircea_popescu has been running public nodez longer than i , and iirc has pretty good instrumentation, he might have something to add to this thread .
asciilifeform: 4 is where 1,2,3 can be ruled out with some confidence. could be mircea_popescu's 'magic packet', or just about anything, i have ~0 useful data.
asciilifeform: type2 ( pete_dushenski's ) is the garden variety shitflood. which is sometimes solved by ip ban, but only in the case of 'shrapnel addressed to occupant', i.e. idiot prb nodes wildly spamming crapolade, and not in the 'bullet with your name on it' case, where somebody actually has a sybil constellation drowning your trb node in liquishit, with no SINGLE ip misbehaving in any way ☟︎
asciilifeform: mircea_popescu: ultimately for so long as peers are unauthenticated and speak unauthenticated plaintext , there will be type4 blackhole.
pete_dushenski: mircea_popescu: have done some, will try more banning.
asciilifeform: mircea_popescu: blackhole, in my current understanding, is at least 4 distinct things
mircea_popescu: pete_dushenski looks like stock blackholing. ipban the offenders, see what happens.
jurov: ben_vulpes: prb 0.11 and newer insist on using improved protocol
asciilifeform: pete_dushenski: this'd be the genuine article
asciilifeform: ben_vulpes: the thing phf refers to , is in use still, whenever i make (yes) xp box! < 400MB! (lighter weight than, e.g., africa-linux)
asciilifeform: and can you get a core dump out of the thing pete_dushenski
asciilifeform: pete_dushenski: from your telling, it seems that there was no 'after'. so let's have the 'before' and 'during'
ben_vulpes: pete_dushenski: kill -9 may help
pete_dushenski: ben_vulpes: rpc unresponsive so... ya.
a111: Logged on 2016-12-29 22:50 ben_vulpes: run moar macos or how does it go
asciilifeform: pete_dushenski: be so kind as to post plox some logs from your node during and immediately prior to and after the blackhole
ben_vulpes: pete_dushenski: you have to reboot the whole machine?
pete_dushenski: mircea_popescu: i don't disagree from a philosophical standpoint but nor can i tolerate having dead fucking trb nodes. that i should have to reboot a machine ~daily~ is the death of bitcoin. yukoners never had it so bad.
asciilifeform: ^ is probably what mircea_popescu was thinking of ^
asciilifeform: in particular, this bit of nonsense: http://btc.yt/lxr/satoshi/source/src/main.cpp?v=asciilifeform_add_verifyall_option#2339
asciilifeform: ben_vulpes: http://btc.yt/lxr/satoshi/ident?v=asciilifeform_add_verifyall_option&_i=vInventoryToSend
mircea_popescu: pete_dushenski there's really no good reason to not serve arbitrart txn data. the fact that "modern" prb nodes can't support this is entirely their doom.
pete_dushenski: in other inequalities, latest taleb : https://medium.com/@nntaleb/inequality-and-skin-in-the-game-d8f00bc0cb46#_edn3.14k1h647w
asciilifeform: ben_vulpes: nah, full of the previous installment of same
a111: Logged on 2016-12-09 02:38 ben_vulpes: asciilifeform should probably poop more often
asciilifeform: ben_vulpes: lel, 1 single 'yara' sig
ben_vulpes: in other unholy file formats: https://www.us-cert.gov/sites/default/files/publications/JAR_16-20296.pdf
asciilifeform: in other lulz, https://media.ccc.de/v/33c3-8099-how_do_we_know_our_prngs_work_properly
davout: ben_vulpes: it's even worse here, it's to *exit* a position
davout hands ben_vulpes some wine
ben_vulpes: pete_dushenski: asking or being asked?
a111: Logged on 2015-07-02 20:44 ascii_field: 'getdata is used in response to inv... ...t can be used to retrieve transactions, but only if they are in the memory pool or relay set - arbitrary access to transactions in the chain is not allowed to avoid having clients start to depend on nodes having full transaction indexes (which modern nodes do not).'
a111: Logged on 2016-12-29 21:04 ben_vulpes: mod6: moreover the general case is looking to be "project-genesis.vpatch"
pete_dushenski: mircea_popescu: i have sincerely nfi why my node is asking for these. it's at ~full height.
mircea_popescu: pete_dushenski why is your node asking for these ? or is it ?
shinohai: mircea_popescu will be saddened to know the inventor of the red Solo cup has died.
mod6: <+mircea_popescu> http://btcbase.org/log/2016-12-29#1592756 << that is historical nonsense that got hammered out in a large discussion when it was actually brought to the forum. << ok ☝︎
ben_vulpes: mircea_popescu: have a link handy?