log
▁▁▁⏐
deedbot: http://phuctor.nosuchlabs.com/gpgkey/A81D208D3F586D37BB5B01618701760E7ECB09D0E3EE181083CEB4E8C0A52C75 << Recent Phuctorings. - Phuctored: 1489...1973 divides RSA Moduli belonging to '162.217.146.236 (ssh-rsa key from 162.217.146.236 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown US NY)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/A81D208D3F586D37BB5B01618701760E7ECB09D0E3EE181083CEB4E8C0A52C75 << Recent Phuctorings. - Phuctored: 1421...9607 divides RSA Moduli belonging to '162.217.146.236 (ssh-rsa key from 162.217.146.236 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown US NY)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/B38595546DF746890308952213DCBF7C001A148E9135B0D939C136F490B9A052 << Recent Phuctorings. - Phuctored: 1512...7289 divides RSA Moduli belonging to '147.102.194.35 (ssh-rsa key from 147.102.194.35 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (het25.physics.ntua.gr. GR I)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/B38595546DF746890308952213DCBF7C001A148E9135B0D939C136F490B9A052 << Recent Phuctorings. - Phuctored: 1489...2027 divides RSA Moduli belonging to '147.102.194.35 (ssh-rsa key from 147.102.194.35 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (het25.physics.ntua.gr. GR I)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/A18F3D16E5DE757C10A2285BF02B0FFE865B3ABBB5B403352A60E6B74963AC4E << Recent Phuctorings. - Phuctored: 1619...8093 divides RSA Moduli belonging to '77.253.213.229 (ssh-rsa key from 77.253.213.229 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (77-253-213-229.static.ip.netia.com.pl. PL MZ)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/A18F3D16E5DE757C10A2285BF02B0FFE865B3ABBB5B403352A60E6B74963AC4E << Recent Phuctorings. - Phuctored: 1491...4003 divides RSA Moduli belonging to '77.253.213.229 (ssh-rsa key from 77.253.213.229 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (77-253-213-229.static.ip.netia.com.pl. PL MZ)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/0DA4CE74B76C9C061A2B19304702C27363CEC8E2D03C0DE8F263A311ABE07BA0 << Recent Phuctorings. - Phuctored: 1766...1429 divides RSA Moduli belonging to '92.243.14.42 (ssh-rsa key from 92.243.14.42 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (www.docteurbeaute.com. FR)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/0DA4CE74B76C9C061A2B19304702C27363CEC8E2D03C0DE8F263A311ABE07BA0 << Recent Phuctorings. - Phuctored: 1451...4147 divides RSA Moduli belonging to '92.243.14.42 (ssh-rsa key from 92.243.14.42 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (www.docteurbeaute.com. FR)
shinohai: pobrecito gribble disconnects again
asciilifeform: !~later tell phf https://github.com/hanshuebner/vlm << sent in by a reader. somehow this has been just sitting there since '09, without my noticing
jhvh1: asciilifeform: The operation succeeded.
asciilifeform: phf: https://github.com/hanshuebner/vlm/blob/master/c-emulator/emulator.c looks like enough to build that fpga ivory...
Framedragger: re. fs nodes, couldn't sleep + not sure if this makes sense, so just throwing these out - barebones super simplistic (function is `n_objects_to_store ^ 1 / folder_depth`) plots showing expected average number of nodes per folder (assumptions are no bias in hashspace and also equal share of hash bits per folder level) - it may not be intuitive how low the averages are until you look:
Framedragger: 1) http://fd.mkj.lt/stuff/fsgraph1.png - up till 100bn objects (to compare, current number of bitcoin transactions ~= 0.2bn)
Framedragger: 2) http://fd.mkj.lt/stuff/fsgraph1.png - up till 10**24 (which is when avg number of nodes per folder reaches 1000 for total depth of 8)
Framedragger: (really kindergarten level simple but wanted to see this myself, could be useful for reference - unless it's incorrect..)
asciilifeform: same link??
Framedragger: d'oh! thanks.
Framedragger: 2) http://fd.mkj.lt/stuff/fsgraph2.png
Framedragger: << (obviously these'd be more useful with actual empirical numbers of average/median seek times, writes, seek/write as things get congested, etc.)
Framedragger: (deeper path => slower traversal but fewer nodes per folder, up to the point where e.g. 'fast symlinks' can be used by ext4 (http://lxr.free-electrons.com/source/fs/ext4/inode.c#L148) - maybe; etc.)
Framedragger: ..and so she goes, http://www.reuters.com/article/us-southkorea-politics-idUSKBN16H066
BingoBoingo: lulz http://www.returnofkings.com/116375/le-favori-de-lelection-presidentielle-francaise-cette-couille-molle-marie-a-une-femme-de-25-ans-son-ainee
asciilifeform: https://github.com/hanshuebner/vlm/blob/master/admin/Beta-test-customers.text << lenat!!!
trinque: BingoBoingo: the guy looks miserable
trinque: sandpaper on the cock will do that.
BingoBoingo: Happens
trinque: Le mariage de Macron est la publicité parfaite pour un taux de natalité non existant en France << l0l
asciilifeform: from 'beta' list -- r.v. guha, turns out, is http://www.guha.com/cv.html , also cyc
asciilifeform: (now absorbed, apparently, into google)
asciilifeform: and apparently responsible singlehandedly , or so himself claims, for one of the more infamous fake wotrons.
asciilifeform: 'epinions'
asciilifeform: and for the existence of rdf. motherfucker.
trinque suffered under an RDF believer for about 4 years
asciilifeform: https://groups.google.com/forum/#!topic/comp.lang.lisp/BebxJD27sao << ancient lulz found when trying to find what the fuck 'mcc' was
asciilifeform: apparently, long-gone 'Microelectronics and Computer Technology Corporation' in austin, tx.
mircea_popescu: is there such a thing as an indian who isn't a total shitbag ?
asciilifeform: mircea_popescu: i know a handful
asciilifeform: holy fuck how can something the size of mcc vanish without ANY trace
asciilifeform: not so much as a broken shard of babylonian clay pot.
mircea_popescu: it helps if it exists in the first place.
asciilifeform: 'MCC was part of the Artificial Intelligence boom of the 1980s, reportedly the single largest customer of both Symbolics and Lisp Machines, Inc. (and like Symbolics, was one of the first companies to register a .com domain). ' << aaaah apparently THAT's how.
mircea_popescu: heh
mircea_popescu: Framedragger i don't get it, you graphed some functions ? or ?
asciilifeform: apparently was a consortium of mainframe makers, serious r&d corps, etc., worked on cad. utterly thermonuked by lisp winter and the microshitization of computing.
Framedragger: mircea_popescu: basically, and that's strictly it - because i couldn't intuitively wrap my head around the fact that average number of nodes per specific folder would be _really_ low if depth is say more than 3. still weird in my head, but yeah.
asciilifeform: http://wotpaste.cascadianhacker.com/pastes/6zrqi/?raw=true << re mcc, of archaeological interest.
mircea_popescu: a ok.
asciilifeform: 'For a long while I was doing research in software productivity. We began asking, ‘What is wrong? And why is it so difficult? Why is it so costly? What is complexity?’ It led me to interesting research, but nothing happened. We did not discover how to formulate or mathematically express the idea of program complexity. Not program complexity in the sense of algorithmic complexity—NP-complete problems and all that jazz—but the
asciilifeform: complexity of programming.'
mircea_popescu: dude these women presidents aren't doing so well after all ? brazil, korea... who has one left ? impeach the queen ?
mircea_popescu: obviously argentina's ex whore is going to jail as well...
asciilifeform: https://stallman.org/articles/texas.html << further lulz re rms guest lecture at mcc.
asciilifeform: (historical)
asciilifeform: featuring 'nasal plant sex'
asciilifeform: http://btcbase.org/log/2017-03-10#1624012 >> 'La Socité Francaise de Médecine Morphologique et Anti-Age ' << lelzz☝︎
a111: Logged on 2017-03-10 01:44 deedbot: http://phuctor.nosuchlabs.com/gpgkey/0DA4CE74B76C9C061A2B19304702C27363CEC8E2D03C0DE8F263A311ABE07BA0 << Recent Phuctorings. - Phuctored: 1451...4147 divides RSA Moduli belonging to '92.243.14.42 (ssh-rsa key from 92.243.14.42 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (www.docteurbeaute.com. FR)
mircea_popescu: you'd be surprised how many people are fixated on this age business.
asciilifeform: http://btcbase.org/log/2017-03-10#1624006 >> 'Sports Memorabilia & Promotional Products' >> 'Sport It, Inc.', some nowhere, usa. spamola, and diddleddebian.☝︎
a111: Logged on 2017-03-10 01:26 deedbot: http://phuctor.nosuchlabs.com/gpgkey/A81D208D3F586D37BB5B01618701760E7ECB09D0E3EE181083CEB4E8C0A52C75 << Recent Phuctorings. - Phuctored: 1421...9607 divides RSA Moduli belonging to '162.217.146.236 (ssh-rsa key from 162.217.146.236 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown US NY)
deedbot: http://qntra.net/2017/03/us-sportscaster-tout-south-korean-victory-over-china-the-republic-of-in-world-baseball-classic-ignores-actual-losses-by-peoples-republic/ << Qntra - US Sportscaster Tout South Korean Victory Over China (The Republic of) in World Baseball Classic, Ignores Actual Losses By People's Republic
BingoBoingo: ^ Some idiot said Qntra was a "Pro-China" rag
mircea_popescu: oh it isn't ?
mircea_popescu: (he means republic of china, yes, not the communist fake state ?)
asciilifeform: y'mean usg's 'unsinkable carrier' ?
mircea_popescu: lel.
asciilifeform: ( their actual sales pitch from the period!11 )
mircea_popescu: http://btcbase.org/log/2017-03-10#1624042 << i don't mean, privately. i mean a public indian.☝︎
a111: Logged on 2017-03-10 03:43 asciilifeform: mircea_popescu: i know a handful
asciilifeform: 0
mircea_popescu: sad testament for the cockroach race.
BingoBoingo: <mircea_popescu> (he means republic of china, yes, not the communist fake state ?) << Well they both lost
mod6: alf was it mentioned that some of these recent submission need regrinding?
mod6: *submission(s)
asciilifeform: mod6: anything at all might need regrinding, depending on your particular tree
asciilifeform: mod6: had something more specific in mind ?
mod6: asciilifeform_blackhole_odometer.vpatch, asciilifeform_blocktimer.vpatch, and asciilifeform_goodbye_pingers_fixed.vpatch all have the same input hash.
mircea_popescu: meanwhile in ai news, http://68.media.tumblr.com/4277bdf9e99091bd0846649df6201385/tumblr_o6dkfiaXG61umjwnoo1_400.gif
asciilifeform: goodbye_pingers is to be shitburied
mod6: which is fine, if you only use one of the three above, but not good if you try to use >1 of them.
asciilifeform: imho.
asciilifeform: it achieves 0.
mod6: ah, ok.
asciilifeform: (at most 0 !)
asciilifeform: blackhole odometer is probably of ~very~ limited interest to folx who aren't asciilifeform
asciilifeform: 'block timer' is obsoleted by blackhole odometer.
asciilifeform: so we're left with null set.
mod6: no worries. shinohai was building, asked me to double check just to be sure. thought it was worth the mention to future spelunkers.
asciilifeform: mod6: you can safely consign these to 'experimental tree' or wherever.
asciilifeform: same place shiva lives.
asciilifeform: 'poorly conceived crackpotteries' or what it was mircea_popescu called'em.
mod6: no sweat. just putting up a signpost for future spelunkers.
mod6: in the spirit of experimentation, it makes sense that one experiment would not necessairly contain the same changes as a different experiment.
mod6: and would have the same input hashes.
asciilifeform: aha
mircea_popescu: aaand in other kid-and-his-chemiferous-garage news, http://68.media.tumblr.com/7c9c5e795cd0594b0b4ebb887243e652/tumblr_oetvs7QpQd1u4w44io1_500.gif
mircea_popescu: http://68.media.tumblr.com/5fc1176a864c9f578d1c17d01d6fb52e/tumblr_oetvs7QpQd1u4w44io4_500.gif etc
shinohai: ghehehehe
mod6: a cautionary note to anyone who is going to use my V to press with wires_rev1 (http://therealbitcoin.org/ml/btc-dev/2017-February/000251.html), be sure to name the seal as such or it won't get picked up in the flow (in the new, forthcoming version 99994) as such: asciilifeform_wires_rev1.vpatch.asciilifeform.sig
trinque: yar, had to rename it
trinque: mod6: the thing should really have one of those google AIs to figure out which goes with which
trinque: I'm sure they have a cloud API for that
shinohai: lel
mod6: hah
BingoBoingo: AV - Artificial Veh
deedbot: http://phuctor.nosuchlabs.com/gpgkey/BD8C9C1ADBE5ED9416A31D88FB9C1A1090230FACD87D55B9363507BAB92E0559 << Recent Phuctorings. - Phuctored: 1497...5393 divides RSA Moduli belonging to '78.137.160.9 (ssh-rsa key from 78.137.160.9 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (ip-78-137-160-9.dedi.digiweb.ie. IE L)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/BD8C9C1ADBE5ED9416A31D88FB9C1A1090230FACD87D55B9363507BAB92E0559 << Recent Phuctorings. - Phuctored: 1709...1097 divides RSA Moduli belonging to '78.137.160.9 (ssh-rsa key from 78.137.160.9 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (ip-78-137-160-9.dedi.digiweb.ie. IE L)
lobbes: http://btcbase.org/log/2017-03-09#1623875 << I agree and will admit that my bot and log-o-tron are/were largely forged from used dildos I found laying in the forest. I also don't pretend that they are anything but☝︎
a111: Logged on 2017-03-09 18:23 trinque: I said to him "students may learn swordplay but may not tell us the fork in their hands is a sword"
lobbes: I'm still in the process of climbing out of the primordial soup, so to speak. 'had to start somewhere, etc'; Though I realize I will need to flamethrow my own creations soon enough and rebuild once I 'evolve' a bit more. But all in due time
trinque: cool, later in the thread I'm convinced the whole thing is reasonable
trinque: :p
lobbes: sure, but from what I can observe, your own deedbot (and others' log-o-trons) appear to operate in a more sane manner. I guess I'm just saying I have much to learn
trinque: happy to help when you feel like trying it out.
trinque: in fact, I'll go ahead and get the logbot-service thing released now, since it's been running the wot & voicing for maybe a month now
ben_vulpes: > forged from used dildos
lobbes: trinque: ah thank you!
ben_vulpes: HAVE YE VISITED MOUNT DILDO FORGE, NEWCOMER?
lobbes: l0l
davout: http://btcbase.org/log/2017-03-10#1624027 <<< bwhaha "couille molle"☝︎
a111: Logged on 2017-03-10 03:18 BingoBoingo: lulz http://www.returnofkings.com/116375/le-favori-de-lelection-presidentielle-francaise-cette-couille-molle-marie-a-une-femme-de-25-ans-son-ainee
phf: asciilifeform: there's several sources of vlm floating around, some of them improved upon by independent hackers (in the direct meaning of the word improve)
phf: asciilifeform: if you spent time with it though, you'll realize that it's an assembly dump of the original emulator written for alpha, and the "emulator" part is actually a combination of c and lisp necessary to ~translate original alpha assembly into portable-ish c~
phf: i've spent some time with it before, improving mac os support, and working out crash issues
phf: i don't quite remember, what goes where, but i'm pretty sure the emulator.c that you linked is a frankenstein that primarily emulates alpha, though it has some signifcant knowledge about what the alpha emulator does, so it's not quite qemu, but something a lot scarrier
deedbot: http://qntra.net/2017/03/park-geun-hye-impeachment-upheld-by-south-korean-court-shes-out-of-there/ << Qntra - Park Geun-hye Impeachment Upheld By South Korean Court – She's Out Of There
phf: https://github.com/hanshuebner/vlm/blob/master/alpha-emulator/ifunlist.as#L52 e.g. lisp machine instruction DoAssoc, but the code below, all those BIS, SUBL, SUBQ are alpha instruction set
phf: there's a translation code (in lisp) that takes that alpha body and spits out c equilvalent of the BIS ... SUBL ... etc. sequence
phf: lasciate ogne speranza, voi ch'intrate
davout: in other women torture news: https://www.youtube.com/watch?v=aR-fA6OG21w
mircea_popescu: meanwhile in "totally not torture she's evidently enjoying herself your honor", http://68.media.tumblr.com/b017ae134ec78e38392c2fe15680847c/tumblr_ocdzn3JHCz1ric730o2_400.gif
mircea_popescu: davout est ce qu'elle jouit un peu?
davout: i know i would
mircea_popescu: o shit did i actually get the verb righ for once ?
davout: almost perfect
davout: just missed a hyphen
davout: est-ce
davout: other than that nailed it
mircea_popescu: nuts.
mircea_popescu: I was with ben_vulpes at dildo forge, knee deep in girl spunk
mircea_popescu: i said how come the men here suffer like they do ?
mircea_popescu: phf the story of alpha is perhaps one of the best illustrations of the fundamentally anti-intellectual stance and calling of the female state. 1980s true 64 bit architecture, well supported (openvms started there, ffs, go revolutionize shit in 2015 with the remnants of 1980s tech). "sold" to fiorina's company, never heard from again, because gotta prop up intel.
mircea_popescu: phf it's ogni/ch'entrate not ogne / ch'intrate, btw :D
mircea_popescu: you totally gotta to eat more varied cunt, i can tell by your vowels your gyneceum's monocultural!
deedbot: http://phuctor.nosuchlabs.com/gpgkey/83747C8EA3613D6D3DEEEDE006197672FCB135ABA18188DB332E25098C44C765 << Recent Phuctorings. - Phuctored: 1650...4657 divides RSA Moduli belonging to '202.130.103.22 (ssh-rsa key from 202.130.103.22 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (wtt22.smartinfo.com.hk. HK)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/83747C8EA3613D6D3DEEEDE006197672FCB135ABA18188DB332E25098C44C765 << Recent Phuctorings. - Phuctored: 1622...1153 divides RSA Moduli belonging to '202.130.103.22 (ssh-rsa key from 202.130.103.22 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (wtt22.smartinfo.com.hk. HK)
asciilifeform: http://btcbase.org/log/2017-03-10#1624127 << the classic alpha ver you described was in one of the subdirs, aha. and a recent g5 one (that dks mentioned to me in 2010 when he delivered the long-awaited genera alpha to my rupturefarm)☝︎
a111: Logged on 2017-03-10 10:39 phf: asciilifeform: if you spent time with it though, you'll realize that it's an assembly dump of the original emulator written for alpha, and the "emulator" part is actually a combination of c and lisp necessary to ~translate original alpha assembly into portable-ish c~
asciilifeform: http://btcbase.org/log/2017-03-10#1624127 << the c src i linked to does not appear to match this description, seems to be honest emulator☝︎
a111: Logged on 2017-03-10 10:39 phf: asciilifeform: if you spent time with it though, you'll realize that it's an assembly dump of the original emulator written for alpha, and the "emulator" part is actually a combination of c and lisp necessary to ~translate original alpha assembly into portable-ish c~
asciilifeform: http://btcbase.org/log/2017-03-10#1624129 << mno☝︎
a111: Logged on 2017-03-10 10:43 phf: i don't quite remember, what goes where, but i'm pretty sure the emulator.c that you linked is a frankenstein that primarily emulates alpha, though it has some signifcant knowledge about what the alpha emulator does, so it's not quite qemu, but something a lot scarrier
asciilifeform: phf: link to yours some time ?
asciilifeform: http://btcbase.org/log/2017-03-10#1624146 << alpha was already half-dead when hp swallowed compaq, in '02. iirc compaq shitburied it in '01 and sold all rights to intel, who proclaimed the arch now known as 'itanic' as its replacement.☝︎
a111: Logged on 2017-03-10 12:25 mircea_popescu: phf the story of alpha is perhaps one of the best illustrations of the fundamentally anti-intellectual stance and calling of the female state. 1980s true 64 bit architecture, well supported (openvms started there, ffs, go revolutionize shit in 2015 with the remnants of 1980s tech). "sold" to fiorina's company, never heard from again, because gotta prop up intel.
mircea_popescu: iirc it was late 90s
asciilifeform: amd hired the designers away
mircea_popescu: yeah it was ishitsomething, itembel, i dun recall.
asciilifeform: and turned from a sort of 'cyrix' , piss clones maker, into actual cpu house
mircea_popescu: aha
mircea_popescu: lasted about as their professional lives did, also.
mircea_popescu: amd couldn't hire a new set.
asciilifeform: mircea_popescu: 'itanium', vliw monster for which nobody was ever able to make a useful compiler
mircea_popescu: yeah, i recall it wasn't ever actually deployed in any meaningful sense. ielbrus.
asciilifeform: afaik only known working examples are in usg heathen pits
mircea_popescu: same as elbrus yes.
mircea_popescu: "a technology so powerful, so secret, it needn't even exist!"
mircea_popescu: the nuke totally fucked up their brains.
asciilifeform had a prof in uni who taught systems architecture (cpu design) with itanic as 'what not to do'
asciilifeform: re alphas, i am rather fond of the alpha, i have several here, possibly mentioned before
asciilifeform: not quite 'fits in head' machine, but 'fits in book'
asciilifeform: one perverse fact, is that microshit had a winblowz ecosystem for it, ready to go, it could have been he 'designated winner' in intel's place. except -- no, because gotta target the konsoomer chump junk ...
asciilifeform: ( the sane folx, who had alphas with tru64, openvms, etc, went 'wtf why would i even think of winblowz' -- which wouldn't do. whole edifice 'musted die' )
mircea_popescu: ah i recall that, windows nt
mircea_popescu: well, anyway, i suppose it is a good thing the intelligent people involved could you know, pursue their own self-determined will and etcetera. who knows how many glorious flights of amateur rc airplanes this obviously reasonable arrangement gave the world!
mircea_popescu: not to mention of all the truly priceless fishing and whatnot else, history hasn't recorded.
Framedragger: i like my rc airplanes. "the will of history necessitates you to X" has a marx'ified hegelian vibe :p
mod6: mornin'
Framedragger: o/
mircea_popescu: oyesitdoes.
mircea_popescu: adolescence is the perceiving of boundless possibilities and maturity comes with the comprehension of the bounds.
Framedragger: myeah #trilema is basically valgrind.
mircea_popescu: but, as the eternally mentioned bottle well informs : the above has any merit only inasmuch it rankles the subject himself. otherwise it's nothing ; and besides there's many ways to, in brick pollitt's words, "hear that little click"
mircea_popescu: so all is not lost! stupor abounds and thanks to the ever advancing technologee is now cheaper than ever!
mircea_popescu: meanwhile at spiked ball ranch, http://68.media.tumblr.com/b9d67860081dce95976c346981a1bc71/tumblr_obxnfzrDdZ1s4jw4io1_1280.jpg
asciilifeform: http://btcbase.org/log/2017-03-10#1624188 << i built myself a 'games box' , not long ago, 'to unwind', but turned out that the part of my head that was able to enjoy this, had atrophied, found myself constantly 'if i'm this awake, oughta be running $experiment, finishing up $unfinisheds', etc. ended up giving the (princely) fuckofftron to brother to play with.☝︎
a111: Logged on 2017-03-10 15:02 mircea_popescu: so all is not lost! stupor abounds and thanks to the ever advancing technologee is now cheaper than ever!
asciilifeform: somehow, long ago internalized mircea_popescu's above point re the model airplanes.
asciilifeform: 'if not you, then who' (tm) (r)
mircea_popescu: yeah, it's not lost on me that i'm the loudest preacher of continence in a cloister of monks muchly more restrained and disciplined than myself.
mircea_popescu: i suppose that's why they have that "excluding present company" device.
trinque: asciilifeform: getting rejected by dulap with that ssh key I gave ya.
asciilifeform: trinque: you gotta log in as shown in the recipe
trinque: oops, k.
asciilifeform: trinque: i.e. your login is 'tunnel', not 'deedbot'
trinque: ding
asciilifeform: worx?
trinque: ayep
trinque: connecting trb to it now
deedbot: http://phuctor.nosuchlabs.com/gpgkey/0376A22D7C6DDA3C6BCF95FADF960352C32B7EC823CB87C4197DEE78BA0E4D9F << Recent Phuctorings. - Phuctored: 1489...1137 divides RSA Moduli belonging to '213.182.76.82 (ssh-rsa key from 213.182.76.82 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (213-182-76-82.ip.welcomeitalia.it. IT)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/0376A22D7C6DDA3C6BCF95FADF960352C32B7EC823CB87C4197DEE78BA0E4D9F << Recent Phuctorings. - Phuctored: 1580...0613 divides RSA Moduli belonging to '213.182.76.82 (ssh-rsa key from 213.182.76.82 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (213-182-76-82.ip.welcomeitalia.it. IT)
Framedragger: http://btcbase.org/log/2017-03-10#1624048 << feltbad, so wrote that stupid symlink and fs profiling tool that no-one wanted to do. results later. while at it: anyone knows if CLOCK_MONOTONIC has sufficient resolution for profiling? asciilifeform? allegedly - yes.☝︎
a111: Logged on 2017-03-10 03:46 mircea_popescu: Framedragger i don't get it, you graphed some functions ? or ?
Framedragger: getting ~4000-7000ns for symlink resolution to real path for a 1mn symlink dir structure, e.g.
asciilifeform: that's slightly slower, looks like, than the old db..
Framedragger spent longer than wants to admit sorting out his heap and valgrind'ing. too much python is bad for a person
Framedragger: orly? this is *ns* (10^-9), mind you. hm. and this is just resolution of path with single symlink in it
asciilifeform: ok backtracking : what means 'resolution'
Framedragger: sorry - yeah
asciilifeform: does this include the actual read ?
Framedragger: no
asciilifeform: aah
asciilifeform: also, on what
asciilifeform: ssd ?
Framedragger: asciilifeform: call to `readlink()`.
Framedragger: yes, ssd
asciilifeform: and on top of what (ext4 ? )
Framedragger: ssd under docker fs, later - real disk
Framedragger: ext4, yes
asciilifeform: and connected how ? ( sata2 ? 3 ? )
Framedragger: uhh
asciilifeform: wtf is 'dockerfs'
Framedragger: note, it's just some additional syscalls, re docker
Framedragger: will get a way to test real disk soon, didn't want to run on personal trashy PC, hence shitty server
asciilifeform: to rephrase, this would tell something useful if we had with what to directly compare it with -- what operation in ye olde bdb would this measure correspond to ?
Framedragger: aha, right. i haven't even looked at bdb.
Framedragger: what i want to do later when i find time is, actually read file, too, of course.
Framedragger: for now, just generated 1mn symlinks with names corresponding to transaction hash hex.
asciilifeform: let's put down in the log, what exactly ye olde bdb does, that eats 99+% of trb's wall clock:
asciilifeform: it exists (wallet idiocy aside) in trb for 1 purpose and 1 purpose only
asciilifeform: ... so that a 256-bit turd, e.g., 3ec455a2a84e978687a3990cec73f36b324fbd28e03603c6d9fc52018b001558, can be taken and matched to a block # where said tx lives.
asciilifeform: and likewise when a new block comes, all of the tx in it, get indexed this way.
Framedragger: so the 'matching' (index lookup) is the 99% here, right?
asciilifeform: lookup+store
Framedragger: aha, right.
asciilifeform: btw i will also put down in the log, one very simple possible algorithm for a 'txidx-fs' :
asciilifeform: picture a 16GB contiguous block of ssd. you cut it up so that it corresponds to 2^32 x 32bits.
Framedragger: (yeah btw, just ftr, symlink *creation* under populated dir structure (`ln -s files_f1/block35461.txt dc/dc89c1f2b58909d3814b250a731a9b9b791b092759553e3ba6579ffaad3a7565`) is slow. however, the creation was done using shellscript, need to move to c to be able to actually profile with precision.)
asciilifeform: now you store the table as follows: the top 32 bits (e.g., 3ec455a2 from above) are an array index into this table
asciilifeform: if there are no collisions (i.e. there is no other tx on the disk whose id begins with 3ec455a2) then that entry in the array IS the block index where the tx lives.
asciilifeform: after which finding the actual tx is an O(N) operation in the block, with small N (the tx in the block)
Framedragger: aha, right! so it's basically a (small) hashtable.
Framedragger: (right?)
asciilifeform: lemme finish plox
Framedragger: sorry.
asciilifeform: BUT if there are collisions, the upper bit of the 32bit entry, is set. and then we know that the lower 31, are an index into another contiguous array
asciilifeform: and there we find ~lists of block indices~, so the machine might have to try 2 or 3 blocks before it finds tx.
asciilifeform: how to ~write~ to this data structure is, i think, obvious to the reader, i do not need to describe.
asciilifeform: anyway this is the simplest physically-possible scheme.
trinque: oh jesus docker Framedragger
asciilifeform: but, thing is, it won't work any better on mechanical disk than the old bdb. and imho it is quite impossible to make variable-length tx bitcoin work well on mechanical disk, with modern spamola level.
trinque gets his knotted rope
asciilifeform: however it does have the advantage of not using 10,000,001 lines of ext4 open sores crapolade for anything.
asciilifeform: this whole thing would probably fit in ~1,000 lines of c.
asciilifeform: and in-head.
Framedragger: trinque: it's just a kindergarten way of wrapping up some syscalls. will obviously benchmark outside it later. i wasn't completely certain that my tool wouldn't trash the host fs. :)
Framedragger: asciilifeform: hmm, very nice. i suppose it's as close to fixed-length as is possible given current bitcoin
asciilifeform: Framedragger: it is literally the smallest number of moving parts i can think of , that'd do the job.
Framedragger: efficient seeking.
asciilifeform: also on a reasonable box you can keep the l1 table in ram
asciilifeform: so then you have 1 disk seek per tx, rather than 2.
asciilifeform: ( or, in the case of ext4 or other konsoomer fs, fuck-knows-how-many ! )
Framedragger: asciilifeform: wait, what is "block index"? just the integer denoting block number?
asciilifeform: Framedragger: correct
asciilifeform: block where tx lives.
Framedragger: why the need for "the machine might have to try 2 or 3 blocks before it finds tx" then? and if so, then no guarantee of only 1 seek?
asciilifeform: Framedragger: it depends how many collisions per 32bit head
deedbot: http://phuctor.nosuchlabs.com/gpgkey/7FF10FFD5E7F7EE6C2497D20452061FB8713C20F904A7A86B21EE093A5E7D254 << Recent Phuctorings. - Phuctored: 1391...6403 divides RSA Moduli belonging to '79.98.16.61 (ssh-rsa key from 79.98.16.61 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (a61.ip.network-consulting.fr. FR)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/7FF10FFD5E7F7EE6C2497D20452061FB8713C20F904A7A86B21EE093A5E7D254 << Recent Phuctorings. - Phuctored: 1797...3053 divides RSA Moduli belonging to '79.98.16.61 (ssh-rsa key from 79.98.16.61 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (a61.ip.network-consulting.fr. FR)
asciilifeform: one seek, on a standard disk, gives you at least 4kB (blocks are 512b, but no disk made in past 20 years actually ~stops at~ 512)
asciilifeform: remember, we are contemplating contiguous blocks on disk
Framedragger: oh i finally understood, literally all there is when one seeks to location 3ec455a2 is a list of block numbers. (or single block number.)
asciilifeform: aha.
asciilifeform: if you're willing to blow another 16GB, you can have an l2 table
Framedragger: this is quite nice, and as you say, seek operation already gives a small chunk which should cover most/all tx for current state of affairs (total number of transactions)...
asciilifeform: on same principle
asciilifeform: Framedragger: it's good for many, many times the current cumulative-tx
asciilifeform: you'll have somewhere between 0 and 2 , statistically, tx per 32bit table entry.
Framedragger: is this the first time you articulated this approach here? i think that's the best on can have for fs-tx-db
asciilifeform: yes, first time, i kept waiting for someone to open a schoolbook and describe this very elementary algo
asciilifeform: it has obvious minus, in that you need a kernel driver
asciilifeform: if you want to actually have the contiguous 16GB
asciilifeform: however mircea_popescu's 'let's patch ext4' has same.
Framedragger: at least i have the excuse of not having looked at the bdb problem / staying away from trb for the time being :p
asciilifeform: i dare say the tabular algo is simple enough to actually be implementable.
Framedragger: i'm not sure if you do need driver
asciilifeform: Framedragger: if you want your own fs, and to write to contiguous disk, you do
Framedragger: here i have an ssd seek profiler which just needs root
asciilifeform: who the fuck wants trb running as root
asciilifeform: (you may as well stuff it in the kernel, then)
Framedragger: it uses `lseek64()`
Framedragger: have separate service taking care of that? i mean, kernel driver is this kind of 'externality', too (and also ring0)
asciilifeform: the Right Thing would probably be to have a very simple kernel driver that takes a specially-marked disk partition and gives userland trb linear use of it, as plain array
asciilifeform: ...rather than to have any serious number crunching in kernelspace
Framedragger: makes sense.
asciilifeform: also this is probably obvious, but you want separate disk for this
asciilifeform: physically
asciilifeform: from the one you are running from
asciilifeform: if you want serious speed.
Framedragger: sure
asciilifeform: ideally you would even have separate disks for blocks and tx indices.
asciilifeform: ok, now question for ben_vulpes , trinque , mircea_popescu , et al : anybody got a quick c proggy that will eat blkcut's blocks and produce a linear list of tx ? i'd like to actually calculate the current 32bit tabular collision rate.
asciilifeform: it is statistically possible that we don't even have ONE collision yet
asciilifeform: (and thereby a 16GB l1 table would give you wholly in-memory index for ~all tx lookups)
asciilifeform: btw Framedragger , we can do 1 better than 'block index'
ben_vulpes: asciilifeform: not a c proggy, no
asciilifeform: say we put down blocks on 1024 byte alignments.
Framedragger: as in, disk/partition alignments?
asciilifeform: nono listne
asciilifeform: then a 31bit (remember, highest bit means 'go to l2 table') offset gives you a 2^^41 bit space in which your table entry can point to the beginning of a block.
asciilifeform: or, rather, to where-to-start-looking-for-tx
asciilifeform: ^
asciilifeform: so we dispense with the O(N) dumb search inside-block
deedbot: http://phuctor.nosuchlabs.com/gpgkey/1B44E66975AB5EE7B22A779D4B31F07CE8CE336A45DE02B74BAD206055C62130 << Recent Phuctorings. - Phuctored: 1594...7037 divides RSA Moduli belonging to '93.90.180.151 (ssh-rsa key from 93.90.180.151 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown DE)
deedbot: http://phuctor.nosuchlabs.com/gpgkey/1B44E66975AB5EE7B22A779D4B31F07CE8CE336A45DE02B74BAD206055C62130 << Recent Phuctorings. - Phuctored: 1711...0703 divides RSA Moduli belonging to '93.90.180.151 (ssh-rsa key from 93.90.180.151 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown DE)
asciilifeform: Framedragger: alignment in this case means that we can point ~inside~ blocks, supposing they are stored linearly on disk
asciilifeform: so long as we are ok with having to chug through 1023 bytes of 'not here yet!'
asciilifeform: which costs us 0 because no disk made in last 20 yrs reads any less than 4kB on 1 seek.
asciilifeform: point being, to point at a tx sitting on a multi-TB disk, you don't need to be able to address it ~bytewise~
asciilifeform: thinkaboutit.
Framedragger: bear with my slowness, can you clarify how it looks like if there's a collision in the initial lookup?
Framedragger: in this case.
asciilifeform: it goes to l2 table
asciilifeform: which can take 2 basic forms, depending on your storage budget
asciilifeform: if you're 'rich', it can be a table entirely like l1
asciilifeform: where if top bit of the entry is not set, you have your where-to-go-to-find-tx-on-disk result
asciilifeform: and if set, you gotta look at the next machine word (each entry has a certain amount of space for collisions)
asciilifeform: 'look at next' can also mean 'go to l3', if you're willing to budget another 16G.
asciilifeform: and so forth.
asciilifeform: the most basic setup has an l1 which immediately proceeds to the 'slow' table (where you have to look at multiple offsets in case of collision, to find your tx)
Framedragger: right! ahh that's nice. (so just to clarify, the 1024 byte block trick wouldn't work if there's a collision (unless additional budget / w/e))
asciilifeform: this form will almost certainly suffice for another decade of trb
asciilifeform: considering that we have, what, 200 mil tx; and l1 table gives you 4billion slots.
Framedragger: yeah, given actual tx amounts.. 250mn vs. 2^32
asciilifeform: aha.
Framedragger: yeah.
Framedragger: well *that sounds like a very decent idea*. :)
asciilifeform: Framedragger: the 1024bt thing works in all cases, it simply means that you don't have to point to beginnings of blocks, but to a place very near the tx
asciilifeform: see, we store the blocks linearly
asciilifeform: this also abolishes the cost of 'budget 1MB per block'
asciilifeform: now you can store them back-to-back
asciilifeform: because you can point inside'em in the index.
Framedragger: yeah i forget sometimes. fixed block length is nice for this...
asciilifeform: fixed 1MB would give you O(1) block seek.
Framedragger: ahh, yeah okay, back-to-back you mean exactly that, not having to allocate 1MB per block.
asciilifeform: however block index is not large -- say we have 500,000 blocks, and you want to know where on disk lives its beginning, and your disk takes 64bit lba addressor. so you need 500,000 * 8bytes == 4,000,000 bytes ! 3 floppy's worth
asciilifeform: except that you don't actually need byte-addressing !
asciilifeform: that is, you don't need 8 bytes to say which 4kB slice has the beginning
asciilifeform: so in practice, you can have O(1) seeks ~while~ storing the blocks back-to-back in a classical trb.
Framedragger: (i see how good it is to be aware of how actual disks read data here. some theoretician would propose a pointer-exact-location scheme instead...)
asciilifeform: if you want to consider 'what would give best performance on the iron we have, with minimal moving parts, with the trb we have' this is what comes out.
asciilifeform: i was recently considering implementing a similar scheme for phuctor, and realized today that it would work entirely well in trb.
Framedragger: it's a really Good Thing that the hashing function which spits out transaction hashes gives *uniform distribution*. no congestion / too many collisions expected, and this scheme leverages that.
asciilifeform: btw mircea_popescu's suggestion from last night, to dispense with 'blocks' as separate class of object, and simply store sequence of tx, with 'this was in block B' field added -- would work quite well with this scheme.
asciilifeform: now store each tx on a 4096bt 'cylinder' boundary, and you're golden
asciilifeform: the table can index individual tx.
Framedragger: i guess one can imagine a single sequence of tx then, simply.
asciilifeform: aha.
Framedragger: aha right.
Framedragger: decent ideas, what can i say
asciilifeform: imho the 'hard part' is not even to implement this table, it is freshman homework, but to unravel the liquishit in trb and learn where to even put the lookup/write !
Framedragger: yeah that's one reason i'm not too attracted to trb, tbh, the amount of sewage gruntwork required to decouple shit from the monolith.
asciilifeform: http://btc.yt/lxr/satoshi/source/src/db.cpp?v=makefiles#0329 << starting here.
asciilifeform: the gnarly part is to dissect all of shitoshi's idiot special cases (e.g. 'ReadOwnerTxes')
asciilifeform: and reduce all of the bdbisms to calls to table ops.
asciilifeform: or, for instance, what is ReadDiskTx .
asciilifeform: (as far as i can see, it is unused code)
Framedragger: hmh, at least functions make up not the *worst* interface seen, but still lotsa work and weird mutable shit sprayed all around, i imagine
trinque: on the subj, still waiting for my node to shut down to connect the wire to dulap
trinque: god forbid the disk always be in a coherent state, eh?
asciilifeform: trinque: i've found that 'kill', which syncs the db, followed by kill -9, which nukes shitoshi's pointless wait idiocy, worx 100%.
asciilifeform: with 0 loss.
trinque: that's right, it's doing a sleep or two of several minutes
asciilifeform: 0 reason to let it
asciilifeform: winblowz culture.
trinque: what, thing needs to cool?!
asciilifeform: (some time grep for the sleeps, it is instructive)
asciilifeform: idiot put them ~everywhere
asciilifeform: i am nearly convinced that he was, at one time, a microshit employee.
asciilifeform: nobody else does this!11
asciilifeform: Framedragger, ben_vulpes : i predict that there will be few enough collisions that you can shunt ~all~ collision cases off to an O(N) lookup table, for next decade, with ~0 measurable loss of avg.case performance.
asciilifeform: (lookup table consisting of, literally, rows of full txid followed by where-on-disk, that you'd chug through whenever collisionbit is set)
Framedragger: yeah, makes sense to me (on average, current likelihood of particular 32 bit entry being populated is ~ < 6%)
asciilifeform: so, literally,
asciilifeform: if (table[txidtop32] & 0x80000000) return linearlookup(txidtop32);
asciilifeform: else return disklookup(table[txidtop32]) ;
asciilifeform: .
asciilifeform: 'linearlookup' is the slow-table from earlier
asciilifeform: disklookup is the 'go to nth cylinder and before you get to n+1st cylinder, THERE's yer tx' from earlier still.
asciilifeform: and you have here ~whole thing.
asciilifeform: ( i can already picture mircea_popescu spitting out his breakfast, 'modern hdd dun have cylinder, you nut' -- except, it in effect DOES, fetches massive blox , whether mechanical or ssd, by design, for ages now )
Framedragger: there's an assumption as max num of collisions here, of course, but obvs in practical terms it's a very safe assumption...
Framedragger: as to*
asciilifeform: there is not.
Framedragger: oh
asciilifeform: if you get moar collisions, you simply end up in the slowtable moar often.
Framedragger: and the slow-table.. hm
asciilifeform: if you start seeing them ~regularly~ (say, trb makes it to 50 yrs from nao) you put in a l2 table.
asciilifeform: then you get another century.
asciilifeform: (i would hope that l1 will last until trbi..)
Framedragger: right right, not disputing that.
asciilifeform: do the arithmetic, it isn't as if anyone can cancel the 'block per 10min' thing.
Framedragger: yeah. hmh - i guess that's it. literally. nice scheme
asciilifeform: there is 1 potential nuance,
asciilifeform: enemy might start to 'mine' txids
asciilifeform: so that they collide
asciilifeform: but this'd cost'em moar than simply to mine.
asciilifeform: (you could in fact use a repurposed miner, for this, as i understand)
asciilifeform: at any rate, working around this idiocy would be very cheap -- say we index by top32 of keccak(txid) instead of plain txid. in the event of.
asciilifeform: much cheaper to reindex a trb node, what, 1 night, than for enemy to bake new asics.
asciilifeform: so this scenario, imho, is out.
asciilifeform: (enemy is welcome to bash his skull to a pancake against the cement wall)
Framedragger: while at it, and i'm guessing it's in the logs.., i wonder when was the last time some tried to come up with an approximation what the costs to get to >50% network hashing power would be
Framedragger: i'm guessing that an interested state could totally do that
Framedragger: and yeah, this 'generate collisions' scenario seems like a harder version of 'just mine'
asciilifeform: Framedragger: there was old thread with mircea_popescu , where he stated that usg and china attempted it at same time, and perma-deadlocked
asciilifeform: iirc in 2014.
Framedragger: attempted.. citing internal intelligence of course, or something
asciilifeform: aha
Framedragger: hard to believe, but then, i should read logs at least
asciilifeform: Framedragger: the other thing is that it is quite impossible to come up with meaningful cost figure
asciilifeform: because the cost, to someone who ALREADY has fab capability , is miniscule
asciilifeform: whereas to someone without (e.g. the folx here) it is ~infinite.
trinque: http://btcbase.org/log/2017-03-10#1624424 << what, it contradicts yours?☝︎
a111: Logged on 2017-03-10 18:25 Framedragger: hard to believe, but then, i should read logs at least
Framedragger: :p
Framedragger: asciilifeform: yeah, i see what you mean
asciilifeform: Framedragger: http://preshing.com/20110504/hash-collision-probabilities << re earlier. quick likbez on collision probability calculation using 'birthday theorem'
asciilifeform: http://preshing.com/20160314/leapfrog-probing << from same www, also notbad.jpg illustration of schoolbook hash tables.
Framedragger: aha, so there's high probability that there will *be* a collision across the entire space.
asciilifeform: Framedragger: just as you do not need 366 people in a room for a birthday collision.
asciilifeform: (~guaranteed at 80 or so)
Framedragger: sure, yeah! nice illustrations there
asciilifeform: ought to help those folx for whom kindergarten happened long, long ago, to picture the subj of today's thread.
asciilifeform: i'll round this out by observing that potentially you might be able to get away with less than 32bits of table (i.e. less than 16GB of space)
asciilifeform: depending on the collision rate.
asciilifeform: i like 32 (recall, 31, we need 1 bit for collision marker) because it translates cleanly to our physical machine.
asciilifeform: but can use, e.g., 30, 29...
asciilifeform: if 'poor'.
asciilifeform: the scheme 'degrades gracefully'.
asciilifeform: the ~other~ thing i'll point out, is that you do not in fact need 16GB of ram, to implement that table , if you can do it in the kernel
asciilifeform: you can use x64's page table to 'cheat' and store a sparse form !
asciilifeform: but THIS i will leave as an exercise for the reader.
asciilifeform: ( however it will be intuitively obvious, to anyone familiar with x86/x64 and similar archs, that this is physically possible. i.e. you never actually need to store a contiguous empty GB in memory, if you can manipulate the page table )
Framedragger: heh you can even use things like nx bit if not wanting to reorganize table, i would guess!
asciilifeform: why would you ever not have the nx bit set for a motherfucking table
Framedragger: that's a horrible idea for sure, but hey, free space to cheat in
asciilifeform: you don't need any moar kludges, the scheme described, will work.
asciilifeform: with hard guarantees, even.
Framedragger: no, of course, you just mentioned possible cheats, but you probably meant something more 'robust'.
asciilifeform: i meant 'cheats' that use the machine as-prescribed and save multiple GB.
asciilifeform: rather than 'let's put cock into gas tank because it fits'
deedbot: http://trinque.org/2017/03/10/deedbot-wired-to-dulap/ << trinque - deedbot.org wired to dulap
asciilifeform: congrats trinque !!
trinque: thanks to you sir
trinque: if you want a reciprocal wire lemme know
asciilifeform: i very well might, shortly.
asciilifeform: (would like to test 2way 'wiring' first on a non-infrastructural set of boxen )
asciilifeform: it'd suck if dulap ~and~ deedbot choked in same day, say.
asciilifeform: 'if one synchronized swimmer drowns, must the others?' (tm) (unknown)
trinque: aha, lol
asciilifeform: trinque: fwiw i still know of at least 1 unsolved tickle re 'wires' -- how to avoid multiple redundant connections
asciilifeform: (right now, e.g., zoolag and dulap happily also find each other over ordinary peer table and connect plaintext)
asciilifeform: i'd like to think of an elegant solution to this
asciilifeform: (can, say, ban them from talking on 8333 to each other in iptables, but this is stupid because then they will not share the perfectly-good ip with OTHER peers)
asciilifeform: and yes i could put in a '--noplaintextpeer=x,y,z...' flag, BUT
asciilifeform: BUT this would make it very easy for the enemy to determine who is 'wired' to whom
asciilifeform: by looking at who never plaintcp's to whom on 8333 !
asciilifeform: so we have dilemma.
asciilifeform: maybe mircea_popescu can cut it, with his knife.
asciilifeform: ( and ~yes~ right now it is very easy to see 'who wired to whom' by looking at ssh, but in the future we will presumably route multi-hop, via a gossiptron, and it will be less obvious )
asciilifeform: at any rate this is a 'theoretical' problem, imho 'not burning'.
asciilifeform: !~later tell Framedragger the table system can be easily prototyped , without writing any kernel code, by using 16G file on ordinary disk. (at the obvious speed penalty.)
jhvh1: asciilifeform: The operation succeeded.
asciilifeform: (on ordinary userland fs, which you have now.)
Framedragger: asciilifeform: yeah! i may have time / want to do this in ~april. just to be clear, there's really no way to have underlying userland fs allocate a contiguous block, right?
asciilifeform: ( i wonder if there are any traditional fs that will actually give you 16G of contiguous blocks, if asked nicely )
Framedragger: ah, that's the q i guess
asciilifeform: Framedragger: you asked at same time as i .. lol
asciilifeform: afaik answer is 0
asciilifeform: but worth investigating imho.
Framedragger: also, wonder if there could be a relevant linux cap'ability for allowing raw access to given /dev/block. but maybe not.
asciilifeform: Framedragger: there is a udev trick, iirc, for giving particular users access to a given raw block dev
asciilifeform: but you'll need to dedicate an entire disk
asciilifeform: at any rate imho the flat-file variant oughta be tried 1st.
Framedragger: well, 'disk' could just be partition which isn't a bad thing anyway, right?
asciilifeform: then it can be readily mapped to physical disk.
Framedragger: sure!
asciilifeform: in fact it can be simply a file descriptor given on command line.
asciilifeform: trb will not even need to know whether it was given a real disk, or flat file.
asciilifeform: for the ultimate variant, where we don't even store 'blocks' as such, but merely sequence of specially-marked tx, you ~will~ need whole disk.
asciilifeform: for completeness, i will note that you can model any 'whole disk' variant with an enormous flat file, in userland, at some speed penalty.
asciilifeform: (i can make a 400GB file on my disk, right now, and it will cost me a ~constant factor vs 'raw disk' i/o)
Framedragger: ah yeah, iirc some applications even use this 'descriptor passing' technique so minimise high permissions level exposure (but don't recall). quite robust
asciilifeform: to borrow from mircea_popescu et al, 'it works if you work it'
asciilifeform: now, 1 more observation, the table scheme described here, can be safely parallelized, for so long as you ensure that the readers are never writing THE same place being elsewhere written.
asciilifeform: unlike idiot db
asciilifeform: or civilian file systems.
asciilifeform: *the readers are never reading THE
asciilifeform: lol.
Framedragger: yes, but how to cheaply ensure the latter. you won't have a shitload of locks now will you
asciilifeform: the reason for this ability, is that you never have 'tree rebalances' or whatever other crapolade general-purpose db, and konsoomer file systems , occupy themselves with !
Framedragger: (i guess depends on nature of parallel work. e.g. if it's some kind of sequential re-confirm, launch threads with different segments, etc.)
asciilifeform: Framedragger: all you need is to check a read candidate against the current list of active writes.
Framedragger: tree rebalance is a feature of a balance binary tree which is sometimes the right tool for the job, cmon :)
asciilifeform: (in hardware this would be O(1) op, in our reality -- slightly slower)
Framedragger: i guess i can see that
asciilifeform: Framedragger: but not right tool for ~this~ job
Framedragger: s/balance binary/balanced binary/
asciilifeform: the sha256 is balancing for us.
Framedragger: of course, i agree, i'm just saying it's not "inherent problem of all db omg"
Framedragger: yeah.
asciilifeform: Framedragger: i am increasingly finding that 'general purpose db' is, like the infamous 'vise-grip', The Wrong Tool For Every Job
asciilifeform: ('visegrip' was/is an american gadget, looks rather like cross between pliers, locking forceps, and plumber's wrench, that was pitches as 'The Right Tool For Every Job' when introduced, some time mid-lastcentury)
asciilifeform: using general-purpose db where performance is a critical concern, inevitably gets you http://btcbase.org/log/2017-02-19#1615434 .☝︎
a111: Logged on 2017-02-19 03:54 asciilifeform: (iirc we had a thread where i described how corporate ameritards, if given a problem like phuctor, would happily soak up a few $mil and megawatt of iron)
Framedragger: those few $mil would be oracle
Framedragger: heh.
asciilifeform: observe, gavin, hearn, et al, have specific orders for past 5 or so yrs : 'make it so that bitcoin REQUIRES oracle cluster'
Framedragger: yeah, i see what you mean, and everything. i'm not convinced that phuctor db is using the most it can from postgres, but neither you nor me have time to investigate this presently, i guess (and i may not be able to do a great job there anyway)
asciilifeform: Framedragger: i did a good measure of tweaking, and exhausted current time budget for that chore
asciilifeform: will come back to it at some point
trinque: I would, in fact, at this point drop postgresql for a persistence layer for CLOS that did not use the thing under the hood.
asciilifeform: trinque: then i'll need a cl pgp parser
asciilifeform: which i currently don't have.
asciilifeform: and a new everything-else
asciilifeform: (right now it uses an old python lib for wwwtronics)
trinque: nah nah I have no prescription for phuctor
asciilifeform: at some point, yes, i'll rewrite it again.
trinque: we'll make a CL lib for p
trinque: gpg can go die in obscurity
asciilifeform: trinque: phuctor is not simply about gpg, recall
asciilifeform: it is a general-purpose shitrng telescope
asciilifeform: largely yet-unused in the proper way.
asciilifeform: we will be busting ssl keys, for instance, at some point
trinque: aha
asciilifeform: and who knows, what else.
asciilifeform: maybe, who the fuck knows, collect enough data re winblows rng defects, via a future 'uci', to break satoshi's keys.
asciilifeform: ('martian' example, but typifies the yet-unknown)
trinque: there's what, a million or so in the stash? plenty to build asciilifeform's fab.
asciilifeform: i did say, 'martian example', not immediate plan to action.
asciilifeform: trinque: incidentally it is not clear, imho, that satoshi's keys are 'edible', they may not be good in practice for getting 'spendable' coin, the popularity of bitcoin per se may well drop catastrophically if you were to move satoshi's coinz
asciilifeform: i contemplate it as more of a 'demolition charge', when trbi exists.
trinque: occurs to me that the man might've seen the coins the same way.
asciilifeform: ( http://www.loper-os.org/?p=1009 << my old crackpot piece re subj )
asciilifeform: and whether d00d burned his hdd, or not, has 0 bearing on whether his keys are, on account of his idiocy, breakable.
Framedragger: some (very initial) symlink stats - more stuff will have to wait - given a "here are 1mn 'transactions' which symlink to files, resolve links and read from linked files in random order for 100mn times" setup, with one-folder-deep structure, like so: "simple_f1/e5/e5edc34c57d5ea2ea99cfe16d04655aa000c3d7f268022d2b21f95928fa34674 -> /files_f1/99997.txt", most basic stats are:
Framedragger: Processed 1000000 lines (min/max target numbers seen: 1, 100000) - beginning benchmarks now... [i.e., 'target block' numbers are in this range. 1m symlinks point to 100k files randomly.]
Framedragger: Uniform randomness report: min hits seen = 871, max hits seen = 1160, avg = 1000.00, sum = 100000000.0, minloc @ 85798, maxloc @ 88837
Framedragger: Symlink resolve (readlink()) times given 100000000 iterations: min = 949 ns, max = 1353644 ns, avg = 1688.99 ns, stddev = 2502.42 ns, sum = 168899426822.0 ns, minloc @ 41772957, maxloc @ 20095033
Framedragger: fopen() times given 100000000 iterations: min = 1761 ns, max = 1506074 ns, avg = 2326.51 ns, stddev = 3003.91 ns, sum = 232650578514.0 ns, minloc @ 72539767, maxloc @ 19225446
Framedragger: fread() times given 100000000 iterations: min = 1189 ns, max = 53879016 ns, avg = 1552.73 ns, stddev = 6917.04 ns, sum = 155273497933.0 ns, minloc @ 2462746, maxloc @ 126806
Framedragger: fclose() times given 100000000 iterations: min = 645 ns, max = 739202 ns, avg = 741.62 ns, stddev = 1647.59 ns, sum = 74161953610.0 ns, minloc @ 19564589, maxloc @ 67051179
Framedragger: (fread() was 4k. files are short, with just the number of of 'block' for cross-confirmation and list of 'transactions' as contents.)
Framedragger: (5ms read on an ssd ain't too pretty, but would have to check distribution of these kinds of tails, i guess.)
Framedragger: (bbl)
Framedragger: (ftr: sata 3gbps, ext4, intel SSDSA2CW30, mirror raid.)
asciilifeform: Framedragger: your figures are probably quite 'optimistic' you have not only ssd, but cache in play
Framedragger: asciilifeform: you're right, i was meaning to reset caches (`echo 3 >/proc/sys/vm/drop_caches` should be enough i think), but then forgot
Framedragger: but could be other stuff, too, etc etc.
asciilifeform: at this point ftr i'm quite firmly convinced that baking in massive open sores shitbucket, e.g. ext4, is The Wrong Thing
asciilifeform: and not even because of speed.
Framedragger: no disagreement tbh, i probably know your reasoning
asciilifeform: i originally favoured it on account of simplicity
Framedragger: given time, i was also meaning to run lots of strace to see what is in actuality happening with multiple symlinks etc, but maybe sisyphus
asciilifeform: but in table system, you don't even need to munge indices into strings to make paths with
asciilifeform: much less to fopen() !
Framedragger: (anyway, will share tool i wrote if only because couldn't find anything fitting the task out there, but need to polish a bit first, etc.)
Framedragger: yeah
jurov: lmao https://github.com/robertfisk/USG/
jurov: "The USG is Good, not Bad"
asciilifeform: lol!!
mircea_popescu waves
mircea_popescu: http://btcbase.org/log/2017-03-10#1624206 << lol, isn't he endearing ? why do you feel so guilty, yo! about to get married or what.☝︎
a111: Logged on 2017-03-10 16:49 Framedragger: http://btcbase.org/log/2017-03-10#1624048 << feltbad, so wrote that stupid symlink and fs profiling tool that no-one wanted to do. results later. while at it: anyone knows if CLOCK_MONOTONIC has sufficient resolution for profiling? asciilifeform? allegedly - yes.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624226 << you know, you have the most amusing penchant for asking questions you really don't want answered.☝︎
a111: Logged on 2017-03-10 16:53 asciilifeform: wtf is 'dockerfs'
mircea_popescu: http://btcbase.org/log/2017-03-10#1624209 << more importantly, is it blocking ?☝︎
a111: Logged on 2017-03-10 16:50 asciilifeform: that's slightly slower, looks like, than the old db..
mircea_popescu: can i run 64 in parallel ?
Framedragger: in terms of thread-safety, yes (but not in benchmarking tool as it's not using thread-safe posix functions, but then it's not writing anything, either). however, parallel performance not at all certain.
Framedragger: oh wait, i was wrong re thread-safe functions. kernel takes care of that. has internal locking mechanism when dealing with fread/fwrite etc.
Framedragger: that said, i'm pessimistic about this whole ext4 thing just like asciilifeform. no reason not to direct time into his latest proposal
mircea_popescu: it ~should~ work, i ask because man who burned tongue on soup
Framedragger: apparently calling fclose() while syscall is running on that descriptor in another thread is 'not a good idea' (outcome maybe platform/implementation specific), but that's an avoidable corner case.
Framedragger: may be*
mircea_popescu: interesting what people looking to push the whore learn about its substance.
deedbot: http://trilema.com/2017/zuleika-dobson-or-an-proper-love-story/ << Trilema - Zuleika Dobson, or An Proper Love Story
mircea_popescu: ^ me heartily recommends, this fundamental novel of the republic, to the critical eye of all.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624229 << at some point you'll have to decide if you care or don't care about the old harlot.☝︎
a111: Logged on 2017-03-10 16:54 asciilifeform: to rephrase, this would tell something useful if we had with what to directly compare it with -- what operation in ye olde bdb would this measure correspond to ?
mod6: Framedragger: hey man, thanks! keep up the good work.
Framedragger: mod6: thanks - was supposed to be busy with other stuff but this cute insane asylum is concerningly quite attracting :)
asciilifeform: woah, fresh megatonne of mircea_popescu
asciilifeform loads it into bed machine
mod6: ah nice mircea_popescu!
Framedragger: http://btcbase.org/log/2017-03-10#1624229 << for posterity, re. those last stats, relevant time-measurement c snippet for reference: http://wotpaste.cascadianhacker.com/pastes/NxLwl/?raw=true☝︎
a111: Logged on 2017-03-10 16:54 asciilifeform: to rephrase, this would tell something useful if we had with what to directly compare it with -- what operation in ye olde bdb would this measure correspond to ?
asciilifeform: neato Framedragger
mircea_popescu: asciilifeform i did foreshadow.
asciilifeform: aha, i recall
asciilifeform: 'expect massive b00k later this week'
mircea_popescu: the people who actually have to work for their accomplishments a la beerbohm would no doubt be quite upset.
mircea_popescu: how goes mod6
mircea_popescu: http://btcbase.org/log/2017-03-10#1624235 << i thought this was entirely obvious, but yes.☝︎
a111: Logged on 2017-03-10 16:55 asciilifeform: ... so that a 256-bit turd, e.g., 3ec455a2a84e978687a3990cec73f36b324fbd28e03603c6d9fc52018b001558, can be taken and matched to a block # where said tx lives.
asciilifeform: all algos begin with restatement of the obvious (e.g. 'this adds two integers')
mircea_popescu: http://btcbase.org/log/2017-03-10#1624242 << this doesn't sound right. need more directory structure than just that wtf.☝︎
a111: Logged on 2017-03-10 16:58 Framedragger: (yeah btw, just ftr, symlink *creation* under populated dir structure (`ln -s files_f1/block35461.txt dc/dc89c1f2b58909d3814b250a731a9b9b791b092759553e3ba6579ffaad3a7565`) is slow. however, the creation was done using shellscript, need to move to c to be able to actually profile with precision.)
mircea_popescu: dc89/c1f2/b589/09d3/814b/250a/731a/9b9b/791b/0927/5955/3e3b/a657/9ffa/ad3a/7565.symlink
Framedragger: so here's the thing, if folder depth is increased, the actual numbers of symlinks per directory will be basically ~0, as per those graphs from earlier.
mircea_popescu: aha.
asciilifeform: btw creation of just about anything is 'pessimized for' by konsoomer fs -- because it is the rarest case on 'typical system'
Framedragger: but on the other hand this'd benchmark/test directory traversal more.
mircea_popescu: aha!
mircea_popescu: Framedragger i didn't intend any distinctiuon betrween "symlink resolution" "directory traversal" et al.
Framedragger: kk as long as we're clear - i've seen those "but what would happen with 1000s of symlinks in single folder!1" comments so thought that should be tested, etoo.
Framedragger: ah right right.
mircea_popescu: btw, the usg is out in force advertising its ersatz bitcoin. "etoro" taking out google advertising on my own name (no doubt along million other strings) and so on.
Framedragger: well, *in principle* deeper traversal should really not be a problem at all. in practice - guess we shall see
mircea_popescu: quite.
mircea_popescu: no doubt in my mind the selva selvaggia be deep and readily surprising.
asciilifeform: mircea_popescu: etoro?
mircea_popescu: some usg-approved "investment" scam.
asciilifeform: denominated in, i can only guess, ethertardium?
asciilifeform: or 0 relation..?
mircea_popescu: "social trading brokerage". doing its best to eat the third world naivite.
mircea_popescu: no, the bitcoin "etf" vehicle. give us dollars so we buy bitcoin "For you" thing.
asciilifeform: iirc human names are not trademarkeable in usgdom, they can take out ads on you, me, mod6 , et al. whateverses.
asciilifeform: they can always say 'ooh we meant ~this other~ mp'
mircea_popescu: ah, i don;t particularly care. anyone googling me who ends up on etoro deserves his punishment.
mircea_popescu: just pointing out, they've been spending by the million/day in hitler notes for the past week with this nonsense.
asciilifeform: lulzy
mircea_popescu: quite
mircea_popescu: not unlike tv advertising to my chained girl, "a better life". really ? i think if she wanted that "better" she'd have found her way by now.
asciilifeform: i guess some bean counter decided that this gambit has higher ev ~in btc~ than , e.g., nsa mining
mircea_popescu: that is certain. the nsa is fast losing the tech battle.
asciilifeform: 'let's run our own pirate'
mircea_popescu: +/-
asciilifeform: kinda disgraceful, not like there isn't a usg 15nm fab or two still standing in upstate new york
asciilifeform: could make miners, instead of whatever arse-penetrating radar controller golden toilet rubbish. for just one or two shifts.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624243 << does the lady know your only true love is strange addressing schemes ?☝︎
a111: Logged on 2017-03-10 16:59 asciilifeform: now you store the table as follows: the top 32 bits (e.g., 3ec455a2 from above) are an array index into this table
asciilifeform: mircea_popescu: it's literally the simplest physically possible scheme, read whole thread. (it is also NOT married to the 'magic numbers.')
mircea_popescu: asciilifeform usg really can't make anything. you're basically saying "siliconed botoxed cali chick could cook a decent meal." of course she could. of course she won't.
mircea_popescu: asciilifeform i have to protest re the scheme. i'm merely needling you in my spare time.
asciilifeform: mircea_popescu: i am not a psychiatrist, do not specialize in 'patient has arms, legs, nerves are fine, but he won't move'em'
mircea_popescu: myeah.
mircea_popescu: yet move - he won't.
asciilifeform: this, if it was a mystery in 2014, is plainly visible today imho
mircea_popescu: i said years back the race is won. but meanwhile the usg utterly lost it, ot little concern of anyone, itself included.,
mircea_popescu: the race is on*
asciilifeform: ( speaking of golden toilet, factory sent me a gold-plated ruler! complete with drill guide. -- yes, made of finest pcb . for lulz. )
mircea_popescu: http://btcbase.org/log/2017-03-10#1624254 << there are habits that die easy and habits that die hard.☝︎
a111: Logged on 2017-03-10 17:02 trinque: oh jesus docker Framedragger
mircea_popescu: https://www.youtube.com/watch?v=kQFKtI6gn9Y << spare time.
asciilifeform: lul
mircea_popescu: possibly their best.
asciilifeform: http://btcbase.org/log/2017-03-10#1624149 << this is a hosting co in hk, and domain peddler. betcha ~all~ of their shithosting offerings, are debianized.☝︎
a111: Logged on 2017-03-10 13:58 deedbot: http://phuctor.nosuchlabs.com/gpgkey/83747C8EA3613D6D3DEEEDE006197672FCB135ABA18188DB332E25098C44C765 << Recent Phuctorings. - Phuctored: 1650...4657 divides RSA Moduli belonging to '202.130.103.22 (ssh-rsa key from 202.130.103.22 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (wtt22.smartinfo.com.hk. HK)
mircea_popescu: prolly.
mircea_popescu: and in other news, https://www.youtube.com/watch?v=ZtnTaUcMLjA
mircea_popescu: http://btcbase.org/log/2017-03-10#1624278 << this blowing seems altogether a foregone conclusion.☝︎
a111: Logged on 2017-03-10 17:10 asciilifeform: if you're willing to blow another 16GB, you can have an l2 table
mircea_popescu: http://btcbase.org/log/2017-03-10#1624290 << contiguity is the big problem. perhaps it can be resolved through "alternative starvation', as in, running naught else. but that is liable to run into the genius of "dwim" "modern" linuxhit☝︎
a111: Logged on 2017-03-10 17:14 Framedragger: i'm not sure if you do need driver
mircea_popescu: http://btcbase.org/log/2017-03-10#1624297 << this sounds like a quite elegant solution, yes.☝︎
a111: Logged on 2017-03-10 17:16 asciilifeform: the Right Thing would probably be to have a very simple kernel driver that takes a specially-marked disk partition and gives userland trb linear use of it, as plain array
Framedragger: mircea_popescu: but if you read further down, we were saying that it may be possible to just access a raw block device without kernel module :)
mircea_popescu: meh.
Framedragger: not that it's necessarily the way to go. but consider what asciilifeform was saying - one could just pass an already-opened file handle. handle to *whatever*. no ring0 driver, no root permissions.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624307 << it is the case, one collision a year sort of deal./☝︎
a111: Logged on 2017-03-10 17:19 asciilifeform: it is statistically possible that we don't even have ONE collision yet
Framedragger: (or, use linux cap to allow raw access, but that's maybe meh)
mircea_popescu: Framedragger it sounds great on paper and then crashes irrecoverably six weeks in.
Framedragger: if you have shitty code, you have shitty code. shitty code in kernel wreaks more havoc.
mircea_popescu: the noion that hdd is usable or useful is a cute pipe-dream of the web generation, unsupported in cold reality.
Framedragger: but yeh, maybe more exposure to userspace shite if not driver - maybe
Framedragger hears wind swoosh as goalposts shifted at c velocity
mircea_popescu: it's more the case that those 10k lines of ??? actually contain the mystery bullshit that makes the whole pile of crap evern work.
Framedragger: mircea_popescu: (i thought with "sounds great on paper" you were responding to possible kernel module workarounds. but if you're against the *whole* idea, fair enough.)
mircea_popescu: i don't think anyone correctly represents the tower of accidental good luck that stands between a disk seek and your desired pron.
Framedragger: for schizzle.
mircea_popescu: im not against the idea in the slightest. i'm just very unpersuaded by the theory hard drives work, to any spec, in any sanemanner.
mircea_popescu: dispensing with the kernel's accumulated gunk comes at no cost to the wizard, but at disability to anyone else.
Framedragger: kernel driver would work around userland fs specific stuff. but certainly not against disk seek, disk cache, etc. just to clarify
Framedragger: (and yah i'm sure there's lots of crud in the former worth dropping..)
mircea_popescu: afaik it bundles a lot fo that in
mircea_popescu: ,m
mircea_popescu has seen architerctures where there were de facto 3 different disk caches
Framedragger: ah, you're probably right. lots of fs-specific cache madness, too
Framedragger: the wonder!
mircea_popescu: yea
mircea_popescu: http://btcbase.org/log/2017-03-10#1624352 << this is a major part of the savings in fsdb☝︎
a111: Logged on 2017-03-10 17:36 asciilifeform: that is, you don't need 8 bytes to say which 4kB slice has the beginning
mircea_popescu: because yes, alf is entirely correct, the hdds piss out 4kb or more per call
mircea_popescu: http://btcbase.org/log/2017-03-10#1624365 << quite so.☝︎
a111: Logged on 2017-03-10 17:43 asciilifeform: imho the 'hard part' is not even to implement this table, it is freshman homework, but to unravel the liquishit in trb and learn where to even put the lookup/write !
mod6: <+mircea_popescu> how goes mod6 << eh, it goes. getting ready to release V 99994 here soon. mom was diagnosed with stage 4 small cell cancer about 6 weeks ago -- so that's been pretty intense.
mircea_popescu: o.o
mod6: ikr. :/
mircea_popescu: that's not good.
mircea_popescu: 1 in 10 shot or what.
mod6: 3-6mo.
mircea_popescu: some do survive it. but myeah.
mod6: with chemo... could be 8-12. we don't think we'll do chemo tho. don't see much of a point.
mircea_popescu: did she smoke ?
mod6: like a chimney. still does.
mircea_popescu: yeah. well...
mod6: 'tis what happens.
mircea_popescu: sorry to hear.
mod6: Thanks Sir. Me too.
mod6: On a brighter note, about to setup a new trb node this weekend outside of aws. So that's awesome.
mircea_popescu: yeah srs.
mod6: <+mircea_popescu> http://btcbase.org/log/2017-03-10#1624297 << this sounds like a quite elegant solution, yes. << yeah, nice idea here. all of this is very exciting.☝︎
a111: Logged on 2017-03-10 17:16 asciilifeform: the Right Thing would probably be to have a very simple kernel driver that takes a specially-marked disk partition and gives userland trb linear use of it, as plain array
mircea_popescu: http://btcbase.org/log/2017-03-10#1624370 << god help us.☝︎
a111: Logged on 2017-03-10 17:46 asciilifeform: or, for instance, what is ReadDiskTx .
diana_coman: mod6, so sorry about that
Framedragger: mod6: damn man, very sorry to hear that.
Framedragger: bitcoin-out-of-the-cloud ftw.
mod6: diana_coman, Framedragger: thanks
mod6: <3
mircea_popescu: http://btcbase.org/log/2017-03-10#1624374 < heh. though this be a taller order than meets the eye.☝︎
a111: Logged on 2017-03-10 18:01 trinque: god forbid the disk always be in a coherent state, eh?
mircea_popescu: http://btcbase.org/log/2017-03-10#1624375 << confirmed, this is safe. just don't issue the -0 before it hd a chance to settle down.☝︎
a111: Logged on 2017-03-10 18:01 asciilifeform: trinque: i've found that 'kill', which syncs the db, followed by kill -9, which nukes shitoshi's pointless wait idiocy, worx 100%.
mircea_popescu: -9*
mircea_popescu: http://btcbase.org/log/2017-03-10#1624381 << very much so. reminiscent of ye olde http://btcbase.org/log/2016-01-21#1379603☝︎☝︎
a111: Logged on 2017-03-10 18:02 asciilifeform: (some time grep for the sleeps, it is instructive)
a111: Logged on 2016-01-21 13:29 asciilifeform: 'if i make it what i think is the right size, it crashes!111'
mircea_popescu: http://btcbase.org/log/2017-03-10#1624395 << your emulator stil lsucks what can i tell you. ai winter!☝︎
a111: Logged on 2017-03-10 18:12 asciilifeform: ( i can already picture mircea_popescu spitting out his breakfast, 'modern hdd dun have cylinder, you nut' -- except, it in effect DOES, fetches massive blox , whether mechanical or ssd, by design, for ages now )
mod6: heheh
mircea_popescu: http://btcbase.org/log/2017-03-10#1624406 << they try; but yeah hehe.☝︎
a111: Logged on 2017-03-10 18:17 asciilifeform: do the arithmetic, it isn't as if anyone can cancel the 'block per 10min' thing.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624413 << this truly isn't a concern.☝︎
a111: Logged on 2017-03-10 18:20 asciilifeform: at any rate, working around this idiocy would be very cheap -- say we index by top32 of keccak(txid) instead of plain txid. in the event of.
mod6: perhaps not. i like that we're thinking about things like this though.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624417 << this is not accountable in dollars.☝︎
a111: Logged on 2017-03-10 18:24 Framedragger: while at it, and i'm guessing it's in the logs.., i wonder when was the last time some tried to come up with an approximation what the costs to get to >50% network hashing power would be
mircea_popescu: http://btcbase.org/log/2017-03-10#1624420 <<->> http://trilema.com/2013/things-that-matter-these-days-things-that-dont-matter-these-days/#footnote_3_50058 2013.☝︎
a111: Logged on 2017-03-10 18:24 asciilifeform: Framedragger: there was old thread with mircea_popescu , where he stated that usg and china attempted it at same time, and perma-deadlocked
Framedragger: > 1 government, fair point
mircea_popescu: http://btcbase.org/log/2017-03-10#1624436 no guarantees, but likel;y, yeah/☝︎
a111: Logged on 2017-03-10 18:33 asciilifeform: (~guaranteed at 80 or so)
mircea_popescu: http://btcbase.org/log/2017-03-10#1624446 << there may be a lot of merit in this. even l1-l4 implemented via kernel table may be faster than freestanding l1 with "occasonal" (to be defined) cache miss aka collision.☝︎
a111: Logged on 2017-03-10 18:40 asciilifeform: you can use x64's page table to 'cheat' and store a sparse form !
mircea_popescu: http://btcbase.org/log/2017-03-10#1624468 << very much this.☝︎
a111: Logged on 2017-03-10 18:57 asciilifeform: i'd like to think of an elegant solution to this
mircea_popescu: http://btcbase.org/log/2017-03-10#1624481 << you can most assuredly cheat by you know, virgin hdd. but kludge of kludges and metatronium nonsense besides.☝︎
a111: Logged on 2017-03-10 19:11 asciilifeform: ( i wonder if there are any traditional fs that will actually give you 16G of contiguous blocks, if asked nicely )
mircea_popescu: http://btcbase.org/log/2017-03-10#1624496 << experimentally this is almost never above 50%☝︎
a111: Logged on 2017-03-10 19:19 asciilifeform: for completeness, i will note that you can model any 'whole disk' variant with an enormous flat file, in userland, at some speed penalty.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624508 << you also need a guarantee against being "slow" (term of art)☝︎
a111: Logged on 2017-03-10 19:33 asciilifeform: Framedragger: all you need is to check a read candidate against the current list of active writes.
asciilifeform: http://btcbase.org/log/2017-03-10#1624681 << fwiw i've used raw hdd for actual work.☝︎
a111: Logged on 2017-03-10 22:21 mircea_popescu: the noion that hdd is usable or useful is a cute pipe-dream of the web generation, unsupported in cold reality.
asciilifeform: if you just need to store linear array of blox -- nothing beats raw hdd. fuck fs.
mircea_popescu: myeah
asciilifeform: http://btcbase.org/log/2017-03-10#1624688 << if you're thinking of the iron failing -- i operate disks in raid5 (and for past couplea years -- raid5 of ssd; on this particular box in this particular room, for instance, 4 x 1tb ssd) : 0 detectable failures-with-loss of any kind.☝︎
a111: Logged on 2017-03-10 22:23 mircea_popescu: im not against the idea in the slightest. i'm just very unpersuaded by the theory hard drives work, to any spec, in any sanemanner.
mircea_popescu: http://btcbase.org/log/2017-03-10#1624512 << rebalancing the tree is actually a terrible idea for bitcoin block storage.☝︎
a111: Logged on 2017-03-10 19:33 asciilifeform: Framedragger: but not right tool for ~this~ job
mircea_popescu: because any arbitrart hash has equal chances to be seen as next hash as any other.
asciilifeform: mircea_popescu: aha! which is why i published an algo with 0 rebalance, today.
mircea_popescu: this is exactly contrarty to the engineering assumption in "balancing"
asciilifeform: (i sat down with the ext4 thing and very quickly came to the conclusion that it, and every other konsoomer fs, is 'pessimized' for our specific scenario)
mircea_popescu: asciilifeform no, no, nothing of the kind. i was contemplating no hardware whatsoever, just the tower of crud that builds up to consumer fs
Framedragger: ftr i didn't every say it was the right thing for bitcoin with uniformly distributed hashspace
mircea_popescu: the rebalancing above a fine examplke of the fundamental problems.
mircea_popescu: "your kink is not my kink"
mircea_popescu: Framedragger yeah that's exactly it, the distributions
mircea_popescu: http://btcbase.org/log/2017-03-10#1624517 << don't knock it too much, bunny-hop fuck-chair is fine tool for she who for the first time took her clothes off in public today.☝︎
a111: Logged on 2017-03-10 19:34 asciilifeform: Framedragger: i am increasingly finding that 'general purpose db' is, like the infamous 'vise-grip', The Wrong Tool For Every Job
mircea_popescu: that nobody else uses it... well.
mircea_popescu: there's still a lot of noobs.
Framedragger: (also just for poterity it's not like you can't tell a decent db which indexing mechanism it should use. but i agree with main 'general tools are ~meh' sentiment)
mircea_popescu: there is no practical way to tell postgresql to do slave reads ; nor any way to tell ~any db to do unbalancing btree stores.
asciilifeform: http://btcbase.org/log/2017-03-10#1624751 << ben_vulpes actually went off to calculate it, empirically!, earlier today, lessee what he comes back with☝︎
a111: Logged on 2017-03-10 23:06 mircea_popescu: http://btcbase.org/log/2017-03-10#1624446 << there may be a lot of merit in this. even l1-l4 implemented via kernel table may be faster than freestanding l1 with "occasonal" (to be defined) cache miss aka collision.
mircea_popescu: this theory of yours crashes on the jagged shores of a meaner reality.
mircea_popescu: asciilifeform in any case this is turning into QUITE the course on caching!
asciilifeform: ( he asked 'what do' and i answered 'take all tx, from first to top of currentheight, and count how many share their senior 32bit )
mircea_popescu: compare and contrast with the mit offering.
asciilifeform: soon we find out 'how many shared birthdays'
asciilifeform: btw it may even be worth taking top 32 and bottom 32 and comparing (fuck knows, sha(sha(x)) could produce different ! distribution, for no entirely good reason, for each )
Framedragger: also just ftr, a b-tree by definition has every part of it be of the same depth. 'unbalancing b-tree' doesn't even make sense. but you could have 'just a normal tree', and one can implement custom indices. just pointing out
mircea_popescu: http://btcbase.org/log/2017-03-10#1624528 << and this not worth doing, either, because just do p when it comes out and be done with it☝︎
a111: Logged on 2017-03-10 19:38 asciilifeform: trinque: then i'll need a cl pgp parser
asciilifeform: mircea_popescu: aha, i started at one point to do it, and realized 'wtf why'
mircea_popescu: Framedragger i just meant "don't fucking rebalance my tree, behind the scenes, idiot!"
mircea_popescu: (they do this, and they think they're smart for it, too.)
Framedragger: right right :)
Framedragger: ACID is not a bad idea yknow. i wonder if you could accomplish what you wanted with dirty slave reads via streaming replication or sth. but, eh. bbl, food
mircea_popescu: psssh.
Framedragger: (ya i know you pissed on it)
mircea_popescu: acid is not a bad idea in the sense "valentine's day" is not a bad idea.
Framedragger: more sex. more sex on acid. checks out.
mircea_popescu: if you have no relationship with your whore, it will provide some anchor for pointless, idle pretense while you "focus on your career"
mircea_popescu: http://btcbase.org/log/2017-03-10#1624534 << word, exactly.☝︎
a111: Logged on 2017-03-10 19:39 trinque: we'll make a CL lib for p
mircea_popescu: http://btcbase.org/log/2017-03-10#1624551 << these are actually pretty encouraging as such☝︎
a111: Logged on 2017-03-10 20:31 Framedragger: some (very initial) symlink stats - more stuff will have to wait - given a "here are 1mn 'transactions' which symlink to files, resolve links and read from linked files in random order for 100mn times" setup, with one-folder-deep structure, like so: "simple_f1/e5/e5edc34c57d5ea2ea99cfe16d04655aa000c3d7f268022d2b21f95928fa34674 -> /files_f1/99997.txt", most basic stats are:
mircea_popescu: fopen() times given 100000000 iterations: min = 1761 ns, max = 1506074 ns, avg = 2326.51 ns, << holy shit what.
mircea_popescu: incidentally what is your ns counter method ?
mircea_popescu: and what the fuck caused that outlier.
asciilifeform: mircea_popescu: he posted the method, in the end of the thread, http://btcbase.org/log/2017-03-10#1624601☝︎
a111: Logged on 2017-03-10 21:33 Framedragger: http://btcbase.org/log/2017-03-10#1624229 << for posterity, re. those last stats, relevant time-measurement c snippet for reference: http://wotpaste.cascadianhacker.com/pastes/NxLwl/?raw=true
mircea_popescu: ah shall get to it
asciilifeform: and these outliers are always caused by a) interrupt b) cache miss
BingoBoingo: Sorry to hear mod6, keep coming back
asciilifeform: aha, hats off to mod6
mircea_popescu: lol jurov cute
mircea_popescu: asciilifeform always worth hounding down in my experience.
mircea_popescu: time_elapsed_nanos = timer_end(time_init); << trhis is ns precision now ?
asciilifeform: mircea_popescu: his fopen() is smaller than a princely ssd's seek time. so thereby i could tell, his measurements had cache in play
mircea_popescu: i do not trust his clock ; seems to be accurate within a few ms at best.
Framedragger: struct timespec timer_start() {
Framedragger: struct timespec start_time;
Framedragger: clock_gettime(CLOCK_MONOTONIC, &start_time);
Framedragger: return start_time;
Framedragger: }
asciilifeform: mircea_popescu: i hate to break this sad noose, but nothing on a konsoomer pc is accurate to ns
asciilifeform: ever.
mircea_popescu: right.
asciilifeform: us -- IF you're lucky
mircea_popescu: right.
mircea_popescu: Framedragger you didn't measure anything mah boy!
Framedragger: allegedly this is what folks use for high precision profiling. i checked and wrote trivial version of that. but yah, clock_gettime()
asciilifeform has thought about installing a rubidium clock
mircea_popescu: Framedragger consider using the recently discussed clocking method
Framedragger: now they gonna talk about how one cant measure anything and need phuctor-made superclocks. kk. sure. but this is as accurate as it gets, sir
Framedragger will read later, food
Framedragger: what do you suggest, time()
asciilifeform: Framedragger: try rdtsc
Framedragger: ah that i've heard of. not without its problems iirc
Framedragger: !# rdtsc
asciilifeform: best you get on x86.
mircea_popescu: no no, where the fuck is it
Framedragger: !#srdtsc
Framedragger: !#s rdtsc
a111: 3 results for "rdtsc", http://btcbase.org/log-search?q=rdtsc
mircea_popescu: the people doing key extraction
asciilifeform: literally, it dun get no better on x86, than it
asciilifeform: it's just a register that increments with every tick of the main clock.
Framedragger: http://btcbase.org/log/2016-09-14#1541639☝︎
a111: Logged on 2016-09-14 12:01 Framedragger: "rdtsc is not guaranteed to be available on every CPU, or to run at a constant rate, *or be consistent between different cores.*" (emphasis mine). `get_cycles()` is recommended, but from cursory look it seems that on some architectures it uses rdtsc internally? madness.
asciilifeform: if your cpu dun have rdtsc, throw it out.
asciilifeform: srsly.
Framedragger: and alf responded to me then, actually - http://btcbase.org/log/2016-09-14#1541702☝︎
a111: Logged on 2016-09-14 13:28 asciilifeform: http://btcbase.org/log/2016-09-14#1541639 << not only is this true, but you won't be seeing scientifically accurate nanosecond timing in a konsooomer box at all. the physical clock is not up to it.
Framedragger: but yah fair enough may as well try
asciilifeform: if it dun run at a constant rate: a) adjust your bios b) if fails, Throw it out!!
mircea_popescu: http://btcbase.org/log/2017-03-06#1622520 <<<☝︎
a111: Logged on 2017-03-06 17:22 mircea_popescu: the guys did actually splendid work, read the paper, worth it.
Framedragger: that said, CLOCK_MONOTONIC ain't bad.
mircea_popescu: use an incrementor.
mircea_popescu: better clock than ~anything the kernel exposes, is the sad truth\
Framedragger: incrementor where, in another thread so that one may have more context switches for fun? those calls are blocking..
mircea_popescu: hm
mircea_popescu: they are blocking aren't they
Framedragger: i actually did check what was available before deciding on clock measurement yknow. but yah, need to try rdtsc
asciilifeform: rdtsc is the ticket, it is how profilers time, and the only timer that is really worth twopence on x86.
mod6: <+BingoBoingo> Sorry to hear mod6, keep coming back <+asciilifeform> aha, hats off to mod6 << thank you, Gentlemen.
Framedragger: yeah
mircea_popescu: asciilifeform actually are dma calls cpu blocking ?!
asciilifeform: it resets on coldboot and increments with every tick thereafter
asciilifeform: mircea_popescu: no, whole point is to copy from bucket a to b w/out using cpu
asciilifeform: mircea_popescu: (whole point of dma controller)
mircea_popescu: right.
asciilifeform: where in this mix do we get dma tho.
mircea_popescu: still, on contemplation, hard to argue against rdtsc
Framedragger: ..but if one wants to measure fs performance, he's stuck with syscalls.
Framedragger: suresure.
mircea_popescu: yeah. ok, ok. rdtsc.
mircea_popescu: he's right too, that's how profilers time also.
asciilifeform: btw if you have a 'modern' comp, you will also have bizarre variation in wall clock timings because of smm idiocy
Framedragger: asciilifeform: i thought that's why folks use CLOCK_MONOTONIC? i guess the irony is that it's not..monotonic in the end, still. :(
asciilifeform: (cpu will stop to do ??????, maybe pack key logger buffer for later monthly upload to obummer, and undetectably, but rdtsc counter still ticks, it is lowtech)
Framedragger: (vs. those other options there)
asciilifeform: this is why it is a lost cause to try to do high precision profiling on a laptop
asciilifeform: ( it is handling smm routine many times per second )
Framedragger running on barebones xeon (not docker) nao, fwiw
asciilifeform: more or less everything post-2011 comes with nonempty smm.
asciilifeform: (if you want to find out what's in yours, you can disasm your bios, can be quite enlightening)
phf: http://btcbase.org/log/2017-03-10#1624795 << fwiw i have one written. it can do an equivalent of pgpdump, but knows only a subset of packet types. if there's any interest i can drive it towards a specific goal☝︎
a111: Logged on 2017-03-10 23:27 asciilifeform: mircea_popescu: aha, i started at one point to do it, and realized 'wtf why'
asciilifeform: phf: consider publishing, it could come in quite handy at some point.
asciilifeform: (say, when we look for sha1 collisions in pgpland)
mircea_popescu: ^
Framedragger: asciilifeform: something about needing to discard 'invalid values' with rdtsc (even when set up correctly - apparently best to 'pin' particular cpu, etc etc..) oh god.
mircea_popescu: it's iffy but not unusuable
asciilifeform: Framedragger: link to who proclaimed 'invalid values' wtf
mircea_popescu: steal profiler timer code lel
asciilifeform: thing is foolproof
Framedragger: http://stackoverflow.com/questions/19941588/wrong-clock-cycle-measurements-with-rdtsc don't kill me plz
Framedragger: first answer
asciilifeform: 'When Intel first invented the TSC it measured CPU cycles. Due to various power management features "cycles per second" is not constant; so TSC was originally good for measuring the performance of code (and bad for measuring time passed). For better or worse; back then CPUs didn't really have too much power management, often CPUs ran at a fixed "cycles per second" anyway. ...'
asciilifeform: as i said just few min ago,
asciilifeform: if this happens on your box,
asciilifeform: and cannot be turned off,
asciilifeform: !! THROW IT OUT !!
mircea_popescu: heh
asciilifeform: my clock runs at constant rate.
asciilifeform: 24/7.
asciilifeform: within crystal's temp variation.
asciilifeform: (i measured. and you, also, oughta measure, this is the kind of thing to be aware of in a well-run household)
asciilifeform: and it bugs me that the mobo dun have a jack for a rubidium clock
asciilifeform: ( my fpga boards, the nicer ones, DO in fact !! )
asciilifeform: and incidentally FUCKGOATS also has !!
asciilifeform: you can plug in own clock !
asciilifeform: 'power management' is for the birds.
mircea_popescu: save the planets alf!
asciilifeform: the box i'm sitting in front of, draws 300w. this is not much.
mircea_popescu: THINK OF AFREEKA
phf: beats an xl1201 by a margin
asciilifeform: phf: aha. doesn't sound like a jet turbine either
asciilifeform: (my 3620 certainly does )
mircea_popescu just looked, he is drawing 3.5 kw with his boxen. this is before ac etc.
asciilifeform: mircea_popescu: i meant 1 particular box
mircea_popescu: yawell
Framedragger: i must write a short comedy screenplay for how discussions develop in #trilema. "i want to take a picture" ... ... ... [intense calculations of incident gamma rays to reclaim lost pixels in images] [furious rehash of late renaissance representationalism]
asciilifeform: phf: you dun run your 1201 24/7 do you.
asciilifeform: (have pity on ye olde caps!)
phf: no no, i'm missing console at the moment
asciilifeform: broke?
mircea_popescu: lol Framedragger . you prolly should.
mircea_popescu: anyone ever read http://trilema.com/2012/sorana-comedie-bufa-intr-un-act/ ?
phf: no, just haven't shipped it. i've been waiting for some friends to do west coast/east coast drive, but those always seem to NOT happen when you actually need them. i can boot it to do remote x11 though, was configured with the assumption that it will boot blind
mircea_popescu: it has the distinction of being fashioned ~entirely out of police blotted / da filings verbatim.
mircea_popescu: blotter*
asciilifeform: phf: yeah i'm not sure why a xl1201 owner would even need the console
asciilifeform: (i reverse-engineered the kbd, pretty sure was the first to publicly do so and post on www, you can build converter)