log☇︎
213600+ entries in 0.128s
asciilifeform: what else to do, the original lure of vacuuming up coin is , i picture, ~gone by now
mircea_popescu: anyway, there's an old time bitcoin scammer that keeps pushing these
asciilifeform: this gotta be the 'special-needs' corps of the zog
mircea_popescu: and in today's useless-website-of-the-week, https://www.bitrated.com/explore
asciilifeform: 64bit linuxen on x64 will happily allocate moar. it is bdb that is retarded.
asciilifeform: ( i would try a larger cache, but 4GB is the idiot hardcoded max in bdb , they used a 32bit width )
asciilifeform: will be interesting to see if this abolishes '30min blox'
asciilifeform: it is running on dulap, and will run there until further notice.
mircea_popescu: this needs to run over many, as in thousands, of blocks.
asciilifeform: mircea_popescu: the 40-80 secs. verifications are still 15x slower than on zoolag (ssd box). i wonder, possibly , if the remaining delay still consists of disk -- but of block fetches, rather than tx index.
asciilifeform: it appeared to have 0 effect, but after cache had a chance to fill -- now this.
asciilifeform: ^ very puzzling result, and now finally similar to mircea_popescu's 'disk and Other Factors'
asciilifeform: !~later tell mircea_popescu http://btcbase.org/log/2017-02-26#1618699 >> possibly not wholly useless: >> http://wotpaste.cascadianhacker.com/pastes/g97IR/?raw=true ☝︎
asciilifeform: neato, ty jurov
mircea_popescu: anyway, no the ramdisk thing doesn't work indefinitely, soon enough the block index will exceed the commodously available ram
mircea_popescu: asciilifeform yes trb does reindex.
asciilifeform: it is the kind of thinking that gave us prb. as mircea_popescu described earlier today.
asciilifeform: because reliance on duct tape .
jurov: i don't get why that bothers you so much
jurov: looky, i am just saying that right now it eats, say, a minute per block, which is 144 minute unreachability daily anyway. and am proposing short term tradeoff of having the unreachability shorted once per day.
asciilifeform: i already once built a system where the reward for enemy for unplugging seeped into days, weeks, then months of lost cpu time. until the unplugging became an inevitable thing. never again.
asciilifeform: jurov: this is an open question. for all i know, bdb will reach some idiot hardcoded limit long before this.
jurov: and can trb live with terabyte tx index at all?
asciilifeform: eventually - and in probably not very long time -- it will exceed what your box can copy over and back in one day.
asciilifeform: the tx index ain't ever getting ~smaller~.
jurov: if enemy can cause 1TB of incrememntal data daily, then we failed miserably
asciilifeform: or what, martians will land by then, and give us for free 1TB/sec 1PB disks ?
asciilifeform: also daily backup then ?
asciilifeform: but take limit of t-->infinity. today it is 30GB, next year - 100
asciilifeform: if i'm backing up across the net -- then yes, pretty close to that
asciilifeform: so now if i need 24/7 access to the network, i gotta keep... 24 nodes
asciilifeform: say i can finally buy jurov's magical disk, where backup AND restore of 30GB (that's just the tx index) takes only 1 hr.
jurov: i'm still waiting for explanation how did you came to double digit downtimes.
asciilifeform: so now jurov invites me to see a node with double-digit % of down time as something acceptable ?
jurov: i'll use another node. or you're supposed to have a precious one and only one?
asciilifeform: and i'm still waiting for an answer, what is the node to do during backup and during restore of same ?
asciilifeform: especially when it becomes known that your box fails catastrophically on power loss.
jurov: i said i do sync, then backup
jurov: but this will happen if, and only if, power is cut. if i sync once per day and backup the x, the backup will keep the x forever
asciilifeform: if jurov wants to run his node off ramdisk -- more power to him. but don't try to spin the resulting bitrot as 'hallucination'.
asciilifeform: whatever you want to call it. it is not and will not be an invited guest anywhere i am answerable for.
asciilifeform: understand, if my proggy writes an x to disk, at some point, and then later i read a y, and y != x, this IS CORRUPTION
asciilifeform: jurov: people flush write caches purely from hallucination ? maybe i don't need a disk at all then ?
asciilifeform: expect a warmup time of 2-3 days...
asciilifeform: (and this is probably not measurably slower, in practice, than what mircea_popescu did)
asciilifeform: i suppose one way to rebuild the index using existing mainline trb would be to nuke the .bitcoin dir entirely, and refeed the node via 'eatblock'.
asciilifeform: (unless you're willing to live with random bitflippage, as jurov apparently is)
asciilifeform: trb does not have a knob for 'reindex and make sure blkxxxx matches the index'.
mircea_popescu: it needs very specific complex things.
asciilifeform: mircea_popescu: what did you do with the index if power failed (or node -- crashed) or whatever disruption ?
mircea_popescu: no but i mean, a 30gb ramdisk node to support the general public's mistaken notions and unwarranted pretensions... meh.
asciilifeform: mircea_popescu: as per yesterday's thread -- running node per se is 'expensive and unrewarding activity' aha!
mircea_popescu: i will note that this is an expensive and unrewarding activity that i personally discontinued cca 2014.
mircea_popescu: the paths are hardcoded to .bitcoin/ in stock satoshib, not the end of the world to change them
mircea_popescu: asciilifeform i don't really do blockchain.info style "public support". i think i have the stuff somewhere, i'll have to dig for it. basically it's, blk* live on /sda ; blkindex.dat and friends live on /sdb which happens to be a ramdisk.
asciilifeform: mircea_popescu: plz consider publishing your patch where you made this happen; it is not a trivial change
mircea_popescu: jurov shutdown of node can readily take > 15 minutes ; and can't even be initiated if the node is, eg, in db lock because block eating.
asciilifeform: jurov: the most sane variant of your proposal i can think of would be to run the entire tx index in memory. this is just barely practical on dulap, a massive box. and would mean multi-hour regeneration of the index on warmup. but is at least theoretically an improvement, in that it does not threaten to randomly lose bits.
mircea_popescu: not necessarily in the sense of turning it off or altogether
jurov: Suppose i shutdown bitcoind and backup .bitcoin using rsync (so that all files with recent mtime are backed up).. you say this won't work?
mircea_popescu: there ~may~ be some optimizations that can be applied as-is to turn a jfs into something more appropriate for bitcoining than the "middle of the road" setting it ships with.
asciilifeform: now you need extra logic in trb per se, and this is no longer 'cheap perl hack'.
asciilifeform: what if block files are not in perfect sync with the backed-up index ?
asciilifeform: and how do you propose to guarantee detection of corruption in the tx indices ?
asciilifeform: reorgs are a thing, neh.
jurov: waitwhat, it *writes* to old blk_... files?
asciilifeform: (and, on a hotwallet box, if you have one -- the wallet.)
asciilifeform: and the indices.
asciilifeform: which includes the blocks.
asciilifeform: well not entire fs, but every file touched by trb
jurov: "entire fs is subject to random rot if my app won't checkpoint its files by calling fsync all the time" is NOT true. where did you get such silly notion?
asciilifeform: because random rot is NOT acceptable omfg why is this hard.
asciilifeform: if the entire fs is subject to random rot, as it would be without synced writes, then yes.
jurov: what 400 GB? who said you have to copy whole blockchain every time?
asciilifeform: link to the GB/sec disk plox ?
asciilifeform: copying 400GB takes 'few minutes' on jurov's disk ? where can i buy one ?
jurov: You run some service that needs five-nines availability? I'm fine with some of my nodes being down for few minutes to have known-good state backed up.
asciilifeform: ( does jurov volunteer to buy the double, triple, quadruple sets of disk, for all of my nodes, for these backups ? and what is the node to do while copying over a db ? say 'sorry , we're closed ' ? )
a111: Logged on 2017-02-19 03:54 asciilifeform: (iirc we had a thread where i described how corporate ameritards, if given a problem like phuctor, would happily soak up a few $mil and megawatt of iron)
asciilifeform: nonsense like 'let's flush write cache once in a blue moon instead of normally', 'let's let the db corrupt and SURE I PROMISE WE CAN RECOVER' is what winblowz world is built from.
asciilifeform: spinning rust is the baseline.
asciilifeform: the thread today where mircea_popescu sees something with even the superficial silhouette of a cheap prbistic hack, and barf mightily, taught you nothing, jurov ?
asciilifeform: FUCK that.
jurov: you can stop the daemon every day, call sync and backup the db, and start it. max 24hrs of work lost.
asciilifeform: omfg this is elementary.
jurov: fsync is only remotely related to journal
asciilifeform: may as well run the entire node from ramdisk
asciilifeform: why would i want to turn off the journal ?
jurov: fsync waits on data to be written to disk. and with all modern filesystems, this means flushing all kinds of metadata. too
jurov: asciilifeform: have you tried to merely neuter fsync?
asciilifeform: if mining were not an intrinsically heathen activity, 'closed network' would solve the 'allcomers' problem. but as i currently understand, it'd simply ~move~ it.
asciilifeform: leading to very similar picture as today's.
asciilifeform: let's picture that whole deedbot l1 subscribes. the folx who run public/exposed node, so as to soak up blocks from heathen miners -- will be the ones to blackhole.
mircea_popescu: so give it some time, it just got released like two days ago
asciilifeform: no takers.
mircea_popescu: depends, anyone talking through your ssh pipes yet ?
mircea_popescu: which is something the operator can decide for self.
mircea_popescu: if the node allows non-ssh-tunnel access.
asciilifeform: very concretely: all enemy has to do, to gum up a trb node, is to repeatedly request blocks from 0 .. current, randomly, and switch ips as needed
mircea_popescu: well then.