log☇︎
233000+ entries in 0.146s
asciilifeform: afterwards, anything that contains only (A) can be made lockless
asciilifeform: if any (C) is found, it must be sliced apart with a knife until there is no (C)
asciilifeform: mircea_popescu, ben_vulpes, et al : the pill that would be needed to cure the locks retardation once and for all (and enable, e.g., queueing) while preserving semantics, would be to go through each and every function call in trb and determine if it A) Reads state B) Modifies state C) both D) neither
asciilifeform: you will find that it is never less time than it takes to actually verify the block.
ben_vulpes: all of a sudden i want to collect data on how long trb takes to return from a simple getinfo call, per your above protocol
asciilifeform: as described on the ml (and linked again last night)
asciilifeform: which is, that simply getting a valid block takes your node out of action for 5-15 min !
asciilifeform: you can grep your log for 'SetBestChain' also, and likely will find the same thing.
asciilifeform: and yes i end up with multi-GB logs. but they are quite informative.
asciilifeform: on my nodes, just about every 'received block...' is followed by a bunch of telltale 'socket closed' from barfing peers
ben_vulpes: i may also need to adjust my log rotator, 1 gig of bitcoind logs is...not actually that much history.
asciilifeform: ben_vulpes: in so far as it consists of the block processing delay, as described earlier, it does.
asciilifeform: initially idea was that it would be a kind of load balance arrangement, if one were in blackhole state, connections would rout to other. but in the end i did not bother with this, and simply let one hang behind the other, and used the setup simply to observe how 'blackholing' propagates.
asciilifeform: at one time i ran a barbaric experiment where same box would run ~two separate~ instances of trb
asciilifeform: well that'd be the definition of 'preserve semantics'
mircea_popescu: just reallocate the respective pointers mwahahaha.
mircea_popescu: i suspect it can be done by surgery without the two parts ~even knowing~ they're not satoshi-full-bitcoin.
asciilifeform: the one caveat is that this is probably not doable while preserving trb semantics.
mircea_popescu: and you know, sanity suddenly flowers everywhere - do you send too much in bytes/s ? b.peer can can your. do you send too much crap ? b.blockchain can tell b.peer it dun wanna hear from you no mo.
asciilifeform: if unsure, eat tx, can always drop it on the floor later.
asciilifeform: but yes, mircea_popescu's algo is The Right Thing
mircea_popescu: anyway. the divorce is required and continuing in the current monolith sheer nonsense.
asciilifeform: and never need other mempool tx.
asciilifeform: i'll add to this that in trb as we have it, you ONLY need blockchain to verify a potential mempool tx
mircea_popescu: rather than "do it now!"
mircea_popescu: if queried, b.peer loads mempool-txn as it is and uses that ; so what if it's stale, fuck you.
mircea_popescu: b.blockchain picks up items from received-blocks and verifies them, adding to blockchain if necessary ; picks up items from received-txn, and verifies them, adding to mempool-txn if necessary.
mircea_popescu: b.peer receives items from peers and puts them in received-txn and received-blocks ; both of which are queues.
mircea_popescu: asciilifeform it answers immediately. let's formalize this.
tb0t: Project: trb, ID: 35, Type: I, Subject: Investigate blackhole, Antecedents: , Notes: Investigate what might be occuring with the so-called black-hole, described here: btcbase.org/log/2016-12-20#1586635
mod6: !%p trb 35
asciilifeform: queueing is sane but i will point out that from the other side, a node that doesn't answer immediately because it queued, is not distinguishable from one that doesn't answer because it is spinning wheels on block verification.
mod6: !%a trb I "Investigate blackhole" "Investigate what might be occuring with the so-called black-hole, described here: btcbase.org/log/2016-12-20#1586635"
mircea_popescu: business has a secretary and an actual worker ; fast food has a front office and a back office. there's reasons for things wtf.
mircea_popescu: the fact that you need the blockchain part to tell you what tx are to be relayed does not reduce to, you must make it to listen to all comers.
asciilifeform: you need the entire orchestra to decide whether to accept a tx for relay.
mircea_popescu: nope. blockchain part will get to it when it gets to it, and tell you. until then, peer part builds queues. ☟︎
asciilifeform: ditto tx
asciilifeform: and to do this, you need the entire spittoon.
asciilifeform: this doesn't add up to a working btctron. in order to evaluate a peer, it is necessary to be able to distinguish block from rubbish.
mircea_popescu: mixing these into a single frail monolith is very much satoshism, but we're not supposed to stick to stupid.
mircea_popescu: the process that talks to peers has ~no business~ knowing anything about the blocks ; the process which maintains the blockchain, verifies blocks etc has ~no business~ talking to peers.
asciilifeform: plox to elaborate
mircea_popescu: at the very least block digestion and peering must be cleaved in trb ☟︎
mircea_popescu: the miners are pushing block complexity to the maximum possible because hey, more fees ; the cost this imposes on the nodes is not to be discussed because hey, fatso thinks he should matter and stuff.
asciilifeform: what's more, this is a rinse-and-repeat
asciilifeform: (they -- elementarily -- time out)
asciilifeform: while it verifies, the thing sits around being very sad, and ends up losing most of its peers
asciilifeform: we get a block (dulap already saw same block; it was valid.) which simply takes ages to verify.
mircea_popescu: whole story is, "fatso is angry at not being as relevant as he thinks he should be"
asciilifeform: http://wotpaste.cascadianhacker.com/pastes/SlwGQ/?raw=true << this is fresh from zoolag. and very typical.
asciilifeform: 'To Read the Full Story, Subscribe or Sign In' << lel
asciilifeform looks at his nodes, and discovers that, sure enough, both are blackholed at this very moment.
mircea_popescu: practically speaking, the exact equivalent of "putin did wtc"
mircea_popescu: continuing the floundering sv-tech lulz for a moment, there's https://archive.is/Wdiyj
asciilifeform: or conceivably the described bear trap will actually contain a bear at the end of the day.
mircea_popescu: this butressed on the observation that satoshi code is large and stupidly wrought.
mircea_popescu: asciilifeform the suspicion is you'll be seeing too much variance to be able to say much definitively
mircea_popescu: myeah. i have no problem with the baruch spinoza approach, esp when eg, dealing with viri etc. but it's not a universal wrench.
asciilifeform: but we would have the culprit, yes.
asciilifeform: whether it is waste of time, is separate question.
mircea_popescu: imo this is a total waste of time ; we won't have the culprit even if we drink the gigabyte swamp. which we needn't be drinking in any case.
asciilifeform: after a few days of this, we have the culprit.
ben_vulpes: (impossible to differentiate!)
asciilifeform: so we get a snapshot of the, say, 20 minutes of tcpdump prior to node entering blackhole state.
asciilifeform: at the same time, tcpdump is running
asciilifeform: if it times out !
asciilifeform: this will consist of a shell script that makes, e.g., getinfo api request, AND
mircea_popescu: just keeping track of the binary star is already a huge thing.
asciilifeform: the necessary experiment is i think quite obvious and i do not need to describe it in detail.
asciilifeform: something that comes down the wire. precisely what, i do not yet know.
mod6: oh that's not it then asciilifeform? this is helpful, please continue to describe.
asciilifeform: what happens is that it makes ~your~ node sit retardedly instead of talking into socket at appropriate time.
asciilifeform: ben_vulpes: not what happens. those get banned quite normally.
ben_vulpes: during which the node is ~entirely unresponsive
ben_vulpes: mod6: no, the thing where either a shitty network client or some joker opens sockets and lets them expire, eating 60 seconds of the loop through each peer to service
asciilifeform: mod6: trb isn't multithreaded in the, e.g., apache, sense. it services clients round-robin style, and if it gets stuck at any point, this is what you get
ben_vulpes: i don't expect to be able to make an rpc call while the node is looping through its list of peers and get 'the right' peer shat out
mod6: im not sure that is helpful... are you discussing how the client just seems to have no peers after some period of time?
ben_vulpes: asciilifeform: i mean when the node is running through 'accepted connection' 'socket no message in first 60 seconds'
asciilifeform: recall, whole thing deadlocks, won't accept any interactive command at all.
asciilifeform: ben_vulpes: it'd have to be continuously dumped
ben_vulpes: might be interesting to patch trb to dump relevant connection's self-identification string
mircea_popescu: to think the dork actually claimed he is "slightly more productive on linux than windows". satoshi never fucking as much as saw a posix compliant box.
asciilifeform: it is atrocious design, as is the whole thing.
mircea_popescu: nevertheless, bitcoin 2.0 has no room for this serial nonsense.
asciilifeform: i.e. is the very worst kind of mutilation of 'grandfather's pistol' conceivable.
asciilifeform: which potentially changes the semantics of EVERYTHING ☟︎
asciilifeform: the only boojum is that parallelizing would require removing the locks.
mircea_popescu: anyway. like you said, seems pretty credible parallelizing will fix this./
mircea_popescu: yes but this doesn't answer if the idiocy is naturally ocurring so much
asciilifeform: (it is idiotically serial and the entire thing locks, won't respond to rpc commands, etc)
a111: Logged on 2016-12-20 19:04 mircea_popescu: we have no definitive answer on this.
asciilifeform: http://btcbase.org/log/2016-12-20#1586638 << i was able to reproduce a very similar effect. simply throw blocks at a node that don't fail verification until the very end. ☝︎
ben_vulpes: but i mean blackholing as artifact of some other poorly written client, instead of script opening sockets to trb nodes
ben_vulpes: sure, trb shoulda been aborted
mircea_popescu: but in general there is no such thing as an unwanted pregnancy. for the attack to exist, the client has to be "retarded".
ben_vulpes: didnt think so
mircea_popescu: we have no definitive answer on this. ☟︎
ben_vulpes: is there intuition about whether blackholes are attacks or retarded clients?