log☇︎
▁▂▁▁▁▁▁▁▁▁▁▁⏐︎▁▁▁▁▁▁▁█▄▃▁▂ 5041
asciilifeform: danielpbarron: O(n^2) -- with orphanages etc. - tx verification-- suxx.
asciilifeform: antecedents-in-existing-block or bust.
deedbot: http://trilema.com/2017/the-story-of-the-bulb-that-was/ << Trilema - The story of the bulb that was
asciilifeform: http://btcbase.org/log/2017-02-25#1618589 << was spot on ☝︎
a111: Logged on 2017-02-25 23:22 mircea_popescu: basically nodes are the digital equivalent of women : men fuck them so the state can have babies. hurr durr, pill plox.
asciilifeform: and 'nodes charge' probably isn't practical
asciilifeform: easy to undercut
asciilifeform: 'race to the bottom'
mircea_popescu: myeah.
mircea_popescu: that's the deep lesson in http://btcbase.org/log/2017-02-25#1618597 ☝︎
a111: Logged on 2017-02-25 23:39 mircea_popescu: you could, in theory.
mircea_popescu: provisioning nodes on the basis of "the minimum number needed" is insanity.
deedbot: http://phuctor.nosuchlabs.com/gpgkey/BCB341DDDEF343017F32DED678F07753D22B62F19B274955AC2AD739FC55E6D2 << Recent Phuctorings. - Phuctored: 1095...1183 divides RSA Moduli belonging to '82.185.178.65 (ssh-rsa key from 82.185.178.65 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (host65-178-static.185-82-b.business.telecomitalia.it. IT)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/F5531B421CF54E13A5413500A2CAA7EC23E3D2267C57196B08C217AC2A4EF4B7 << Recent Phuctorings. - Phuctored: 1095...1183 divides RSA Moduli belonging to '62.86.29.254 (ssh-rsa key from 62.86.29.254 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (host254-29-static.86-62-b.business.telecomitalia.it. IT)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/575A15CF2DA998E5E78663389111EFA3BF1F80A47EEB005ED795EA946D20253F << Recent Phuctorings. - Phuctored: 1095...1183 divides RSA Moduli belonging to '85.47.20.107 (ssh-rsa key from 85.47.20.107 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (host107-20-static.47-85-b.business.telecomitalia.it. IT)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/F7EE9D7418360EBF5222D566B0A5B811023AC29616EBC02A9EE52BDB5A00038A << Recent Phuctorings. - Phuctored: 1095...1183 divides RSA Moduli belonging to '2.116.209.1 (ssh-rsa key from 2.116.209.1 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (host1-209-static.116-2-b.business.telecomitalia.it. IT)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/A558B10A1CCC09194358E0502CAFA4415F57F1944DD9FE618E2A2E2D29EFAC0F << Recent Phuctorings. - Phuctored: 1095...1183 divides RSA Moduli belonging to '217.57.196.177 (ssh-rsa key from 217.57.196.177 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (host177-196-static.57-217-b.business.telecomitalia.it. IT)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/24698AD72475BB28F25AD63CC626A01EC41AF82BC51BC4CB8118FC5B15BDF42E << Recent Phuctorings. - Phuctored: 1095...1183 divides RSA Moduli belonging to '217.57.196.179 (ssh-rsa key from 217.57.196.179 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (host179-196-static.57-217-b.business.telecomitalia.it. IT)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/BCB341DDDEF343017F32DED678F07753D22B62F19B274955AC2AD739FC55E6D2 << Recent Phuctorings. - Phuctored: 9692...0219 divides RSA Moduli belonging to '82.185.178.65 (ssh-rsa key from 82.185.178.65 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (host65-178-static.185-82-b.business.telecomitalia.it. IT)
mod6: <+asciilifeform> mircea_popescu, ben_vulpes , mod6 , et al: http://wotpaste.cascadianhacker.com/pastes/FEYhA/?raw=true << thus far on dulap. << very cool alf. wow @ 'ProcessBlock (res == 1) took : 218863ms; db write wait: 175065ms'
BingoBoingo: In other recent classics http://trilema.com/2016/the-fagspital/
deedbot: http://qntra.net/2017/02/major-us-socialist-party-experiences-first-contested-chair-contest-in-decades-as-internal-divisions-grow/ << Qntra - Major US Socialist Party Experiences First Contested Chair Contest In Decades As Internal Divisions Grow
shinohai: https://github.com/sparky005/constbot <<< Moar losing party tears
mircea_popescu: kikzy
deedbot: http://qntra.net/2017/02/automotive-insurance-premiums-surging-in-united-states-texting-blamed/ << Qntra - Automotive Insurance Premiums Surging In United States, Texting Blamed
BingoBoingo: !~ticker --market all
jhvh1: BingoBoingo: Bitstamp BTCUSD last: 1142.22, vol: 6191.70566313 | BTC-E BTCUSD last: 1128.554, vol: 5174.1881 | Bitfinex BTCUSD last: 1142.1, vol: 17915.48912693 | BTCChina BTCUSD last: 1092.001456, vol: 4756.02690000 | Kraken BTCUSD last: 1140.23, vol: 2321.4936139 | Volume-weighted last average: 1133.52004477
deedbot: http://thewhet.net/2017/yankee-doodle-henry-part-iv/ << The Whet - Yankee Doodle Henry, Part IV
mircea_popescu: !!up sdfsdfasdf
deedbot: sdfsdfasdf voiced for 30 minutes.
mircea_popescu: in a shocking development...
BingoBoingo: kek
BingoBoingo: http://qntra.net/2017/02/automotive-insurance-premiums-surging-in-united-states-texting-blamed/#comment-88212
mircea_popescu: !!up mark_00123
deedbot: mark_00123 voiced for 30 minutes.
mark_00123: hi mircea_popescu
mircea_popescu: ello
mircea_popescu: !!up btcreal
deedbot: btcreal voiced for 30 minutes.
deedbot: http://trilema.com/2017/walk-hard-the-dewey-cox-story/ << Trilema - Walk Hard: The Dewey Cox Story
mircea_popescu: "A 17-year-old transgender boy won a Texas state girls wrestling title on Saturday".
mircea_popescu: also cnnleaks.com if anyone still somehow gives a shit about the fake media orgs.
asciilifeform: 'Rules to receive the up-to-$10,000 award from Project Veritas - Project Veritas only offers awards for valuable video or other media types which was legally obtained. It is important for the submitter to follow all local, state and federal laws while obtaining video or other media for submission.' << snoar
mircea_popescu: myeah
mircea_popescu: also, apparently trump will not invite un security council to wh gagglemeets either. nor the correspondents dinner.
mircea_popescu: and in other holy shit 8chan, petslut.com! https://media.8ch.net/zoo/src/1414820994001.webm
phf: http://btcbase.org/log/2017-02-21#1616415 << http://btcbase.org/log/2016-12-03#1577072 ☝︎☝︎
a111: Logged on 2017-02-21 22:51 asciilifeform: phf's logger swallows without burping
a111: Logged on 2016-12-03 01:35 phf: http://btcbase.org/log/2016-09-28#1549612 << so normal action is ^AACTION ... ^A, turns that when the line is too long it gets cut off (which is normal behavior) but in case of action none of client seem to do the regular split, meanwhile the irc server cutsoff terminating ^A, which breaks most parsers (including mine)
phf: since then fixed on btcbase
asciilifeform: ACHTUNG, PANZERS! ☟︎
asciilifeform: mircea_popescu, ben_vulpes , mod6 , et al :
asciilifeform: http://therealbitcoin.org/ml/btc-dev/2017-February/000256.html
asciilifeform: [BTC-dev] (EXPERIMENTAL) Blackhole Read Timings, and the Verdict.
asciilifeform: in other lulz: https://arstechnica.com/security/2017/02/watershed-sha1-collision-just-broke-the-webkit-repository-others-may-follow
asciilifeform: 'Thursday's watershed attack on the widely used SHA1 hashing function has claimed its first casualty: the version control system used by the WebKit browser engine, which became completely corrupted after someone uploaded two proof-of-concept PDF files that have identical message digests.'
asciilifeform: ^ guess what the 'fix' was.
asciilifeform: ^ ... hardcoded check for the published example !!
asciilifeform: in very other lulz, http://www.metzdowd.com/pipermail/cryptography/2017-February/031615.html ( https://archive.is/jLUGT ) << 'Bruce Schneier has recently published an impassioned plea for a United States Federal Internet Security Agency, which would likely gain control of civilian cryptography, among many other munitions.'
asciilifeform: lol block 454862 , 91.61kB ! >> ProcessBlock (res == 1) took : 11081ms; db write wait: 2402ms; db read wait: 597ms
asciilifeform: in VERY other noose,
asciilifeform: http://nosuchlabs.com/pub/nonblockverifying_blackhole.txt
asciilifeform: behold!
asciilifeform: finally caught this beast in the wild, in real time
asciilifeform: a blackhole certifiably NOT connected with block verification.
asciilifeform: we start with where 454862 was received, and (in record time, as it was tiny) accepted.
asciilifeform: grep for 'received block' -- it does not appear again in this log. because we are blackholed OUTSIDE of the verification delays.
asciilifeform: instead we see what appears to be a node simply pecked to death by queued-up getdata-for-block flood
asciilifeform: (grep for 'received getdata for: block')
asciilifeform: this is the mega-prize, folx, the blackhole that can carry on unabated for hours, days, weeks.
asciilifeform: for as long as the enemy is able to keep up the 'gimme, gimme, gimme' flood of 'ahahaha, you're giving something to allcomers! well where's mine'
asciilifeform: incidentally, rationing by ip is a nonstarter, notice that the requests come from a multitude of 'nodes'.
mircea_popescu: "ProcessBlock (res == 1) took : 167839ms; db write wait: 130117ms; db read wait: 21201ms" lelz
asciilifeform: the 'type 2' (non-verification) blackhole goes right back to the fundamental question of 'something to all comers', how much disk thrashing does a derp get to invoke simply by coming up with a not-yet-banned ip and a pseudonode. ☟︎
mircea_popescu: so we're past http://btcbase.org/log/2017-02-24#1617823 ? ☝︎
a111: Logged on 2017-02-24 22:24 asciilifeform: there are no substantial 'other large parts', detectably
asciilifeform: mircea_popescu: that was in re the verification blackhole (the most common type observed on dulap)
mircea_popescu: ahem.
asciilifeform: seems to consist ~entirely of db.
mircea_popescu: the problem with prb is that it's run by a specific sort of retard.
mircea_popescu: that specific sort of retard is specified as follows : "i heard about bitcoin yesterday, and i have a solution!".
asciilifeform: sadly i have 0 solutions, only moar problemz
mircea_popescu: "i observed something on three blocks on one machine and here's the 100% conclusion ; tune in tomorrow for another one that a) fails to reference how i was wrong yesterday or address why and b) offer another 100% plus measures to be taken" is entirely undistinguishable.
asciilifeform: mircea_popescu: if you, using my patch or similar, determined that type1 (verification) blackhole consists of any substantial portion of something other than db wait -- i'm all ears
mircea_popescu: you just said this ; yourself ; above. in discussing "type 1" you are engaging in pure nytimes-ism.
mircea_popescu: where the FUCK was your "type 1" on the 24th ?
asciilifeform: mno, we had a thread where i cut'em up into classes
mircea_popescu: and this isn't the first time we run into the problem of ... let's call them sloppy run "experiments". but the second, this week.
mircea_popescu: it IS a problem.
mircea_popescu: link me to this thread ?
asciilifeform: http://btcbase.org/log/2016-12-29#1592898 ☝︎
a111: Logged on 2016-12-29 23:20 asciilifeform: type2 ( pete_dushenski's ) is the garden variety shitflood. which is sometimes solved by ip ban, but only in the case of 'shrapnel addressed to occupant', i.e. idiot prb nodes wildly spamming crapolade, and not in the 'bullet with your name on it' case, where somebody actually has a sybil constellation drowning your trb node in liquishit, with no SINGLE ip misbehaving in any way
mircea_popescu: we're discussing the discussion on the 24th.
mircea_popescu: which can be briefly summarized as "alf : omg all blackhole is disk wait for db ; mp : thatr's a factor, there's more" repeated a dozen times.
asciilifeform: and i will point out, none of my findings so far contradict mircea_popescu's original 'it sits and waits for the disk.' only question was, where.
mircea_popescu: and i'll point out that the problem here isn't the work, or the thought, but the fucking packaging. you get overexcited and oversignal. it detracts from very valuable stuff.
asciilifeform: all i got is a stopwatch. the idea is, mod6 et al can run same stopwatch, on other boxes, with other types of disk
asciilifeform: i also have a test going where :
asciilifeform: dbenv.set_cachesize(4, 0, 1); // 4GB of contiguous cache ☟︎
asciilifeform: but it had 0 measurable effect
asciilifeform: and i did not bother to vpatchify it.
mircea_popescu: now then : a fix for the db would significantly improve a few classes of block verification delays ; and it would alleviate blackhole-like behaviour due to that, node's frozen checking a new block. there's at least 3 different dos vectors for other nodes, and a) the foregoing wouldn't help ; b) if it helped the enemy could easily upregulate the crapflood to compensate. ☟︎
asciilifeform: (b) is where i ended up earlier.
mircea_popescu: yeah.
mircea_popescu: the other problem is that a good db fix is a very large project, because bitcoin is written insanely. and our fs db isn't moving, last i heard a month ago someone was going to try and profile an extx ☟︎
asciilifeform: i suspect fs-as-db will have same problem as the ancient shitdb
asciilifeform: it doesn't aggregate accesses
asciilifeform: nor has any means for bolting on such a thing
mircea_popescu: it doesn't ?
asciilifeform: nope.
mircea_popescu: i thought modern raid controllers did
asciilifeform: a good raid card will aggregate BLOCK accesses
asciilifeform: but it has nfi re files !
mircea_popescu: yes, but we do file=blocks
mircea_popescu: i thought that was the idea anyway
asciilifeform: that only takes care of block fetches
asciilifeform: but not tx indices
mircea_popescu: there is that. perhaps a better indexing scheme could be had. hence the fucking symlinks
asciilifeform: symlink gets you a file containing desired block, but there is no way on any unix fs that i know of, to symlink to ~an offset inside a file~
mircea_popescu: but your file = block (in the fs sense) = block (in bitcoin sense)
mircea_popescu: we will have 1mb blocks on the filesystem.
asciilifeform: so this'd be a new fs.
asciilifeform: (a trb-i item )
mircea_popescu: no, it'll be a mildly configured ext4 i guess
mircea_popescu: with blocks set to 1mb
mircea_popescu: but i have nfi whether this is even feasible, because this'd be step 2, after the "hey, what happens if you fill a disk with symlinks" EXPERIMENT returns some fucking results.
asciilifeform: has anyone began to do this ?
mircea_popescu: i dunno. coupla people sorta-promised to sorta-try.
mircea_popescu: but the reason it's mired in "first, experiment, profile" is because this is EXACTLY the sort of thing which should theoretically work out of the box on a modern nix, and ABSOLUTELY never does, at all. central lizard fodder.
mircea_popescu: you know, like linking.
asciilifeform: general-purpose fs is likely to be extremely wasteful of space, because it carries the assumption of rewritability
asciilifeform: something that bitcoin has very little use for
mircea_popescu: if it works on ext4 it can be implemented on fucking tape.
asciilifeform: how do you randomaccessfully fetch blocks from tape ?
mircea_popescu: there is that. i'm just saying, you can have nonrewritable media, whatevs.
asciilifeform: bitcoin as we have it, is almost the antithesis of 'tape problem', the ultimate in random access, to the point that it makes disk cache ~useless, every tx can depend on literally any block from 0 to current
mircea_popescu: yes, but blocks once written don't change. that's the non-rewritable part.
asciilifeform: aha
asciilifeform: (blocks really oughta live in antifuse rom. we had a thread..)
mircea_popescu: and there's actually some benefit for the network not even physically being capable to accept reorgs deeper than x.
asciilifeform: iirc we also had this thread
mircea_popescu: yea.
asciilifeform: http://btcbase.org/log/2016-12-19#1585884 << possibly this ☝︎
a111: Logged on 2016-12-19 18:18 ben_vulpes: asciilifeform: if martians produce longest chain with greatest difficulty i think by the rules of the game they own bitcoin
mircea_popescu: there's a difference between extension and reorg, however.
mircea_popescu: one's like "you were in a coma for the past thirty years, here's what happened that you don't remember" ; the other's like "you had a hallucinatory episode, your history for the past x period is bad and you'll have to rewrite it".
asciilifeform: extensions work entirely fine on otp rom
asciilifeform: mircea_popescu: what's the longest reorg you've personally observed, to date ?
asciilifeform: i have this notion, that there was a massive one some time before i tuned in.
mircea_popescu: the one during the original split with the idiots who were there baptised as power rangers
mircea_popescu: 60ish or so deep if i recall.
mircea_popescu: !#s "power rangers"
a111: 155 results for "\"power rangers\"", http://btcbase.org/log-search?q=%22power%20rangers%22
asciilifeform: was this the one where everybody used same buggy client nonsense ?
mircea_popescu: http://btcbase.org/log/2013-05-13#28327 << exciting times. ☝︎
a111: Logged on 2013-05-13 22:39 mircea_popescu: it's screwed wtf.
mircea_popescu: asciilifeform no, this is the one where the idiot miners "updated" and then the update failed and then usgavin gave them some of the bitcoin usg stole to compensate for the lost mining.
asciilifeform: aah
mircea_popescu: and then the usual usgtards were all over "oh, wow, you broke it and then you were "right on it and omg fixed it you're like power rangers" and so...
mircea_popescu: http://trilema.com/wp-content/uploads/2013/03/in-re-bitcoin-devs-are-idiots.htm << the actual event, march 11th
asciilifeform: the unfortunate bit that i keep coming back to in my head, is that http://btcbase.org/log/2017-02-26#1618702 + http://btcbase.org/log/2017-02-26#1618674 add up to a fundamental boojum, that is not a matter of simply 'make a faster db, buy a faster disk' ☝︎☝︎
a111: Logged on 2017-02-26 19:26 mircea_popescu: now then : a fix for the db would significantly improve a few classes of block verification delays ; and it would alleviate blackhole-like behaviour due to that, node's frozen checking a new block. there's at least 3 different dos vectors for other nodes, and a) the foregoing wouldn't help ; b) if it helped the enemy could easily upregulate the crapflood to compensate.
a111: Logged on 2017-02-26 19:14 asciilifeform: the 'type 2' (non-verification) blackhole goes right back to the fundamental question of 'something to all comers', how much disk thrashing does a derp get to invoke simply by coming up with a not-yet-banned ip and a pseudonode.
mircea_popescu: we aren't actually following a purpose here, like "have a good bitcoin". we're merely proceeding from cause : db is broken and THEREFORE must be fixed. not BECAUSE it would bla bla ; but therefore.
asciilifeform: certainly is broken.
asciilifeform: however a node that can be brought to its knees by randos, regardless of how, is also likewise broken design.
mircea_popescu: in this sense your house is broken design, randos could tp it every hour.
asciilifeform: my house is a poor example, i suspect that squirrels could tip it.
mircea_popescu: you live in it dont you ?
asciilifeform: sorta.
mircea_popescu: well then.
asciilifeform: very concretely: all enemy has to do, to gum up a trb node, is to repeatedly request blocks from 0 .. current, randomly, and switch ips as needed
mircea_popescu: if the node allows non-ssh-tunnel access.
asciilifeform: correct.
mircea_popescu: which is something the operator can decide for self.
asciilifeform: 'decide'
mircea_popescu: what ?
asciilifeform: i could cut off public ip access right now -- and never see a block again.
mircea_popescu: depends, anyone talking through your ssh pipes yet ?
asciilifeform: no takers.
mircea_popescu: so give it some time, it just got released like two days ago
asciilifeform: so far it's dulap <--> zoolag.
mircea_popescu: patience my dear alf.
asciilifeform: let's picture that whole deedbot l1 subscribes. the folx who run public/exposed node, so as to soak up blocks from heathen miners -- will be the ones to blackhole.
asciilifeform: leading to very similar picture as today's.
asciilifeform: if mining were not an intrinsically heathen activity, 'closed network' would solve the 'allcomers' problem. but as i currently understand, it'd simply ~move~ it.
jurov: asciilifeform: have you tried to merely neuter fsync?
asciilifeform: jurov: elaborate plox
jurov: fsync waits on data to be written to disk. and with all modern filesystems, this means flushing all kinds of metadata. too
asciilifeform: the journal?
asciilifeform: why would i want to turn off the journal ?
asciilifeform: may as well run the entire node from ramdisk
jurov: fsync is only remotely related to journal
asciilifeform: also flushes write cache , neh
asciilifeform: just what i need, in my node!11!!!! random corruption.
jurov: random corruption happens only if power is yanked
asciilifeform: which it WILL BE
asciilifeform: omfg this is elementary.
asciilifeform: i'm not willingly building another phuctor-1.0, nothx.
jurov: you can stop the daemon every day, call sync and backup the db, and start it. max 24hrs of work lost.
asciilifeform: FUCK that.
asciilifeform: the thread today where mircea_popescu sees something with even the superficial silhouette of a cheap prbistic hack, and barf mightily, taught you nothing, jurov ?
jurov: using spinning rust is not a cheap hack?
asciilifeform: spinning rust is the baseline.
asciilifeform: it is definitionally not 'a hack'.
asciilifeform: nonsense like 'let's flush write cache once in a blue moon instead of normally', 'let's let the db corrupt and SURE I PROMISE WE CAN RECOVER' is what winblowz world is built from.
asciilifeform: and also is how you get http://btcbase.org/log/2017-02-19#1615434 ☝︎
a111: Logged on 2017-02-19 03:54 asciilifeform: (iirc we had a thread where i described how corporate ameritards, if given a problem like phuctor, would happily soak up a few $mil and megawatt of iron)
asciilifeform: ( does jurov volunteer to buy the double, triple, quadruple sets of disk, for all of my nodes, for these backups ? and what is the node to do while copying over a db ? say 'sorry , we're closed ' ? )
jurov: You run some service that needs five-nines availability? I'm fine with some of my nodes being down for few minutes to have known-good state backed up.
asciilifeform: copying 400GB takes 'few minutes' on jurov's disk ? where can i buy one ?
jurov: ebay?
asciilifeform: link to the GB/sec disk plox ?
jurov: what 400 GB? who said you have to copy whole blockchain every time?
asciilifeform: if the entire fs is subject to random rot, as it would be without synced writes, then yes.
asciilifeform: entire .bitcoin dir.
asciilifeform: because random rot is NOT acceptable omfg why is this hard.
jurov: "entire fs is subject to random rot if my app won't checkpoint its files by calling fsync all the time" is NOT true. where did you get such silly notion?
asciilifeform: well not entire fs, but every file touched by trb
asciilifeform: which includes the blocks.
asciilifeform: and the indices.
asciilifeform: (and, on a hotwallet box, if you have one -- the wallet.)
jurov: waitwhat, it *writes* to old blk_... files?
asciilifeform: jurov: depends what you mean by 'old'.
asciilifeform: reorgs are a thing, neh.
jurov: if enemy can reorg half of your blk_files, some sync latency os least of your problems
asciilifeform: and how do you propose to guarantee detection of corruption in the tx indices ?
jurov: *is least
jurov: no detection. unclean shutdown -> restore backup
asciilifeform: what if block files are not in perfect sync with the backed-up index ?
asciilifeform: e.g. 1 block longer.
asciilifeform: now you need extra logic in trb per se, and this is no longer 'cheap perl hack'.
asciilifeform: now it is ~expensive~ shit hack.
mircea_popescu: there ~may~ be some optimizations that can be applied as-is to turn a jfs into something more appropriate for bitcoining than the "middle of the road" setting it ships with.
jurov: Suppose i shutdown bitcoind and backup .bitcoin using rsync (so that all files with recent mtime are backed up).. you say this won't work?
mircea_popescu: not necessarily in the sense of turning it off or altogether
asciilifeform: jurov: the most sane variant of your proposal i can think of would be to run the entire tx index in memory. this is just barely practical on dulap, a massive box. and would mean multi-hour regeneration of the index on warmup. but is at least theoretically an improvement, in that it does not threaten to randomly lose bits.
mircea_popescu: index-in-ram is actually how you run a large, infrastructural like node.
mircea_popescu: jurov shutdown of node can readily take > 15 minutes ; and can't even be initiated if the node is, eg, in db lock because block eating.
asciilifeform: mircea_popescu: plz consider publishing your patch where you made this happen; it is not a trivial change
asciilifeform: (re index-in-ram)
mircea_popescu: asciilifeform i don't really do blockchain.info style "public support". i think i have the stuff somewhere, i'll have to dig for it. basically it's, blk* live on /sda ; blkindex.dat and friends live on /sdb which happens to be a ramdisk.
mircea_popescu: the paths are hardcoded to .bitcoin/ in stock satoshib, not the end of the world to change them
mircea_popescu: i will note that this is an expensive and unrewarding activity that i personally discontinued cca 2014.
asciilifeform: mircea_popescu: as per yesterday's thread -- running node per se is 'expensive and unrewarding activity' aha!
mircea_popescu: myeah
mircea_popescu: no but i mean, a 30gb ramdisk node to support the general public's mistaken notions and unwarranted pretensions... meh.
asciilifeform: mircea_popescu: what did you do with the index if power failed (or node -- crashed) or whatever disruption ?
mircea_popescu: you rebuild it.
mircea_popescu: this is why i say expensive, it's not a case of "random vps hur durr".
mircea_popescu: it needs very specific complex things.
asciilifeform: trb does not have a knob for 'reindex and make sure blkxxxx matches the index'.
asciilifeform: that's what's in mircea_popescu's patch presumably.
asciilifeform: so it isn't simply a matter of 'make a ramdisk', no.
asciilifeform: (unless you're willing to live with random bitflippage, as jurov apparently is)
asciilifeform: i suppose one way to rebuild the index using existing mainline trb would be to nuke the .bitcoin dir entirely, and refeed the node via 'eatblock'.
asciilifeform: (and this is probably not measurably slower, in practice, than what mircea_popescu did)
asciilifeform: expect a warmup time of 2-3 days...
jurov: random bitfippage its purely your hallucination
asciilifeform: jurov: people flush write caches purely from hallucination ? maybe i don't need a disk at all then ?
asciilifeform: understand, if my proggy writes an x to disk, at some point, and then later i read a y, and y != x, this IS CORRUPTION
asciilifeform: whatever you want to call it. it is not and will not be an invited guest anywhere i am answerable for.
asciilifeform: if jurov wants to run his node off ramdisk -- more power to him. but don't try to spin the resulting bitrot as 'hallucination'.
jurov: but this will happen if, and only if, power is cut. if i sync once per day and backup the x, the backup will keep the x forever
asciilifeform: and if power is cut during backup ?
jurov: this is unsolvable problem?
asciilifeform: it very much is.
jurov: i said i do sync, then backup
asciilifeform: especially when it becomes known that your box fails catastrophically on power loss.
jurov: this means x is safely on disk
asciilifeform: and i'm still waiting for an answer, what is the node to do during backup and during restore of same ?
asciilifeform: sit like a brick ?
jurov: i'll use another node. or you're supposed to have a precious one and only one?
jurov: yes, it will sit as a brick and i'm fine with it.
asciilifeform: so now jurov invites me to see a node with double-digit % of down time as something acceptable ?
jurov: i'm still waiting for explanation how did you came to double digit downtimes.
asciilifeform: say i can finally buy jurov's magical disk, where backup AND restore of 30GB (that's just the tx index) takes only 1 hr.
asciilifeform: so now if i need 24/7 access to the network, i gotta keep... 24 nodes
asciilifeform: each with different scheduled hour of downtime.
asciilifeform: jurov: it can't run during backup. or during restore. so -- downtime.
jurov: you mean your disks are capable only 350kB/s read rate?
jurov: er. read write
asciilifeform: if i'm backing up across the net -- then yes, pretty close to that
asciilifeform: but take limit of t-->infinity. today it is 30GB, next year - 100
asciilifeform: in n years - 1TB
asciilifeform: also daily backup then ?
asciilifeform: or what, martians will land by then, and give us for free 1TB/sec 1PB disks ?
asciilifeform: like gavin pronounced ?
jurov: if enemy can cause 1TB of incrememntal data daily, then we failed miserably
asciilifeform: yearly, jurov
jurov: lolwut
asciilifeform: the tx index ain't ever getting ~smaller~.
asciilifeform: eventually - and in probably not very long time -- it will exceed what your box can copy over and back in one day.
asciilifeform: then what will jurov say -- 'oh, no prob, just sync weekly' ??!@
jurov: and can trb live with terabyte tx index at all?
asciilifeform: jurov: this is an open question. for all i know, bdb will reach some idiot hardcoded limit long before this.
asciilifeform: but i have zero interest in designing a bdb-like abortion. for any purpose.
asciilifeform: i already once built a system where the reward for enemy for unplugging seeped into days, weeks, then months of lost cpu time. until the unplugging became an inevitable thing. never again.
jurov: looky, i am just saying that right now it eats, say, a minute per block, which is 144 minute unreachability daily anyway. and am proposing short term tradeoff of having the unreachability shorted once per day.
jurov: i don't get why that bothers you so much
jurov: *shorter once per day
asciilifeform: because reliance on duct tape .
asciilifeform: it is the kind of thinking that gave us prb. as mircea_popescu described earlier today.
mircea_popescu: asciilifeform yes trb does reindex.
mircea_popescu: anyway, no the ramdisk thing doesn't work indefinitely, soon enough the block index will exceed the commodously available ram
jurov: asciilifeform et al.: @ -> ' at ' mailinglist mangling fixed (it was only HTML output, nothing else was mangled)
asciilifeform: neato, ty jurov
asciilifeform: !~later tell mircea_popescu http://btcbase.org/log/2017-02-26#1618699 >> possibly not wholly useless: >> http://wotpaste.cascadianhacker.com/pastes/g97IR/?raw=true ☝︎
a111: Logged on 2017-02-26 19:26 asciilifeform: dbenv.set_cachesize(4, 0, 1); // 4GB of contiguous cache
asciilifeform: ^ very puzzling result, and now finally similar to mircea_popescu's 'disk and Other Factors'
asciilifeform: it appeared to have 0 effect, but after cache had a chance to fill -- now this.
mircea_popescu: myeah.
asciilifeform: mircea_popescu: the 40-80 secs. verifications are still 15x slower than on zoolag (ssd box). i wonder, possibly , if the remaining delay still consists of disk -- but of block fetches, rather than tx index.
mircea_popescu: certainly a possibility.
mircea_popescu: this needs to run over many, as in thousands, of blocks.
asciilifeform: it is running on dulap, and will run there until further notice.
asciilifeform: will be interesting to see if this abolishes '30min blox'
asciilifeform: ( i would try a larger cache, but 4GB is the idiot hardcoded max in bdb , they used a 32bit width )
mircea_popescu: aha.
mircea_popescu: no way out of it, can't allocate more.
asciilifeform: 64bit linuxen on x64 will happily allocate moar. it is bdb that is retarded.
mircea_popescu: well it's a 32 bit item
asciilifeform: aha
mircea_popescu: and in today's useless-website-of-the-week, https://www.bitrated.com/explore
asciilifeform: 'Most reputable users' 'coblee350 Director of Engineering at Coinbase, Litecoin'
asciilifeform: lel
asciilifeform: and wtf is a 'trust agent'
asciilifeform: which he is also 'top' of.
mircea_popescu: it's been dead for about a year or so, but anyway, "oh i know, i'll make yet another fake wot website. because i'm a jew and we're fucking stupid congenitally." or some shit.
mircea_popescu: asciilifeform it's better because no pgp see.
asciilifeform: this gotta be the 'special-needs' corps of the zog
mircea_popescu: some "nadav ivgi" ytard
mircea_popescu: anyway, there's an old time bitcoin scammer that keeps pushing these
mircea_popescu: !#s rosenfeld
a111: 42 results for "rosenfeld", http://btcbase.org/log-search?q=rosenfeld