log☇︎
102100+ entries in 0.02s
asciilifeform: then what will jurov say -- 'oh, no prob, just sync weekly' ??!@
asciilifeform: eventually - and in probably not very long time -- it will exceed what your box can copy over and back in one day.
asciilifeform: the tx index ain't ever getting ~smaller~.
asciilifeform: yearly, jurov
asciilifeform: like gavin pronounced ?
asciilifeform: or what, martians will land by then, and give us for free 1TB/sec 1PB disks ?
asciilifeform: also daily backup then ?
asciilifeform: in n years - 1TB
asciilifeform: but take limit of t-->infinity. today it is 30GB, next year - 100
asciilifeform: if i'm backing up across the net -- then yes, pretty close to that
asciilifeform: jurov: it can't run during backup. or during restore. so -- downtime.
asciilifeform: each with different scheduled hour of downtime.
asciilifeform: so now if i need 24/7 access to the network, i gotta keep... 24 nodes
asciilifeform: say i can finally buy jurov's magical disk, where backup AND restore of 30GB (that's just the tx index) takes only 1 hr.
asciilifeform: so now jurov invites me to see a node with double-digit % of down time as something acceptable ?
asciilifeform: sit like a brick ?
asciilifeform: and i'm still waiting for an answer, what is the node to do during backup and during restore of same ?
asciilifeform: especially when it becomes known that your box fails catastrophically on power loss.
asciilifeform: it very much is.
asciilifeform: and if power is cut during backup ?
asciilifeform: if jurov wants to run his node off ramdisk -- more power to him. but don't try to spin the resulting bitrot as 'hallucination'.
asciilifeform: whatever you want to call it. it is not and will not be an invited guest anywhere i am answerable for.
asciilifeform: understand, if my proggy writes an x to disk, at some point, and then later i read a y, and y != x, this IS CORRUPTION
asciilifeform: jurov: people flush write caches purely from hallucination ? maybe i don't need a disk at all then ?
asciilifeform: expect a warmup time of 2-3 days...
asciilifeform: (and this is probably not measurably slower, in practice, than what mircea_popescu did)
asciilifeform: i suppose one way to rebuild the index using existing mainline trb would be to nuke the .bitcoin dir entirely, and refeed the node via 'eatblock'.
asciilifeform: (unless you're willing to live with random bitflippage, as jurov apparently is)
asciilifeform: so it isn't simply a matter of 'make a ramdisk', no.
asciilifeform: that's what's in mircea_popescu's patch presumably.
asciilifeform: trb does not have a knob for 'reindex and make sure blkxxxx matches the index'.
asciilifeform: mircea_popescu: what did you do with the index if power failed (or node -- crashed) or whatever disruption ?
asciilifeform: mircea_popescu: as per yesterday's thread -- running node per se is 'expensive and unrewarding activity' aha!
asciilifeform: (re index-in-ram)
asciilifeform: mircea_popescu: plz consider publishing your patch where you made this happen; it is not a trivial change
asciilifeform: jurov: the most sane variant of your proposal i can think of would be to run the entire tx index in memory. this is just barely practical on dulap, a massive box. and would mean multi-hour regeneration of the index on warmup. but is at least theoretically an improvement, in that it does not threaten to randomly lose bits.
asciilifeform: now it is ~expensive~ shit hack.
asciilifeform: now you need extra logic in trb per se, and this is no longer 'cheap perl hack'.
asciilifeform: e.g. 1 block longer.
asciilifeform: what if block files are not in perfect sync with the backed-up index ?
asciilifeform: and how do you propose to guarantee detection of corruption in the tx indices ?
asciilifeform: reorgs are a thing, neh.
asciilifeform: jurov: depends what you mean by 'old'.
asciilifeform: (and, on a hotwallet box, if you have one -- the wallet.)
asciilifeform: and the indices.
asciilifeform: which includes the blocks.
asciilifeform: well not entire fs, but every file touched by trb
asciilifeform: because random rot is NOT acceptable omfg why is this hard.
asciilifeform: entire .bitcoin dir.
asciilifeform: if the entire fs is subject to random rot, as it would be without synced writes, then yes.
asciilifeform: link to the GB/sec disk plox ?
asciilifeform: copying 400GB takes 'few minutes' on jurov's disk ? where can i buy one ?
asciilifeform: ( does jurov volunteer to buy the double, triple, quadruple sets of disk, for all of my nodes, for these backups ? and what is the node to do while copying over a db ? say 'sorry , we're closed ' ? )
asciilifeform: and also is how you get http://btcbase.org/log/2017-02-19#1615434 ☝︎
asciilifeform: nonsense like 'let's flush write cache once in a blue moon instead of normally', 'let's let the db corrupt and SURE I PROMISE WE CAN RECOVER' is what winblowz world is built from.
asciilifeform: it is definitionally not 'a hack'.
asciilifeform: spinning rust is the baseline.
asciilifeform: the thread today where mircea_popescu sees something with even the superficial silhouette of a cheap prbistic hack, and barf mightily, taught you nothing, jurov ?
asciilifeform: FUCK that.
asciilifeform: i'm not willingly building another phuctor-1.0, nothx.
asciilifeform: omfg this is elementary.
asciilifeform: which it WILL BE
asciilifeform: just what i need, in my node!11!!!! random corruption.
asciilifeform: also flushes write cache , neh
asciilifeform: may as well run the entire node from ramdisk
asciilifeform: why would i want to turn off the journal ?
asciilifeform: the journal?
asciilifeform: jurov: elaborate plox
asciilifeform: if mining were not an intrinsically heathen activity, 'closed network' would solve the 'allcomers' problem. but as i currently understand, it'd simply ~move~ it.
asciilifeform: leading to very similar picture as today's.
asciilifeform: let's picture that whole deedbot l1 subscribes. the folx who run public/exposed node, so as to soak up blocks from heathen miners -- will be the ones to blackhole.
asciilifeform: so far it's dulap <--> zoolag.
asciilifeform: no takers.
asciilifeform: i could cut off public ip access right now -- and never see a block again.
asciilifeform: 'decide'
asciilifeform: correct.
asciilifeform: very concretely: all enemy has to do, to gum up a trb node, is to repeatedly request blocks from 0 .. current, randomly, and switch ips as needed
asciilifeform: sorta.
asciilifeform: my house is a poor example, i suspect that squirrels could tip it.
asciilifeform: however a node that can be brought to its knees by randos, regardless of how, is also likewise broken design.
asciilifeform: certainly is broken.
asciilifeform: the unfortunate bit that i keep coming back to in my head, is that http://btcbase.org/log/2017-02-26#1618702 + http://btcbase.org/log/2017-02-26#1618674 add up to a fundamental boojum, that is not a matter of simply 'make a faster db, buy a faster disk' ☝︎☝︎
asciilifeform: aah
asciilifeform: was this the one where everybody used same buggy client nonsense ?
asciilifeform: i have this notion, that there was a massive one some time before i tuned in.
asciilifeform: mircea_popescu: what's the longest reorg you've personally observed, to date ?
asciilifeform: extensions work entirely fine on otp rom
asciilifeform: http://btcbase.org/log/2016-12-19#1585884 << possibly this ☝︎
asciilifeform: iirc we also had this thread
asciilifeform: (blocks really oughta live in antifuse rom. we had a thread..)
asciilifeform: aha
asciilifeform: bitcoin as we have it, is almost the antithesis of 'tape problem', the ultimate in random access, to the point that it makes disk cache ~useless, every tx can depend on literally any block from 0 to current
asciilifeform: how do you randomaccessfully fetch blocks from tape ?
asciilifeform: something that bitcoin has very little use for
asciilifeform: general-purpose fs is likely to be extremely wasteful of space, because it carries the assumption of rewritability
asciilifeform: has anyone began to do this ?
asciilifeform: (a trb-i item )
asciilifeform: so this'd be a new fs.
asciilifeform: symlink gets you a file containing desired block, but there is no way on any unix fs that i know of, to symlink to ~an offset inside a file~
asciilifeform: but not tx indices