log☇︎
69400+ entries in 0.039s
ave1: I know, it was more a description of how my memory was playing tricks on me
ave1: asciilifeform, I was looking for mux just now in ffa just now and I came by the add_gated which uses a different method,.
a111: Logged on 2018-10-12 13:08 diana_coman: asciilifeform, hm, the bit version blows up buffers even more because it uses *internally* arrays of bits as per http://www.dianacoman.com/2018/02/01/eucrypt-chapter-8-bit-level-keccak-sponge/#selection-51.100-51.594
asciilifeform: anyway i dun think this is critical for any current application. but ideally would like to be able to take a checksum without leaking what-of
asciilifeform: ( even very small table can end up crossing page boundary )
asciilifeform: ave1: tables on pc are unconstanttime, on acct of cache
asciilifeform: ave1: use the method illustrated in w_mux, http://btcbase.org/patches/ffa_ch1_genesis#L870
ave1: and for the table lookup, it's the the table access part that's not constant time? (lookup(0) might take different time than loop(255))?
ave1: asciilifeform, well bits are being checked to be able to determine if xor div is needed
asciilifeform: ave1: you dun need ffa for this, yer dividing single words by single word
ave1: the div variant could be made to be constant time using the ffa primitives but currently is not.
diana_coman: ave1, thank you, I'll look through it in more detail a bit later today
diana_coman: asciilifeform, hm, the bit version blows up buffers even more because it uses *internally* arrays of bits as per http://www.dianacoman.com/2018/02/01/eucrypt-chapter-8-bit-level-keccak-sponge/#selection-51.100-51.594 ☟︎☟︎
a111: Logged on 2018-10-12 09:41 diana_coman: there is also the bit-keccak in eucrypt and that is the same really - reference code rather than production code
asciilifeform: http://btcbase.org/log/2018-10-12#1860785 << waitasec, we have a non-'bit' keccak somewhere ? i.e. one that doesn't require buffers to be blown up 8x , as in http://btcbase.org/patches/vtools_ksum#L76 ? i'd like this... ( and i suspect so would errybody ) ☝︎
asciilifeform: ( the table variant is elementarily not constant time on pc )
asciilifeform: http://btcbase.org/log/2018-10-12#1860789 << imho a ~constant time~ crc32 would be useful, and can be made from ave1's with very small effort, but i'ma leave it as exercise for him ( simply dispose of the if's ) ☝︎☟︎
asciilifeform: but if i'm overwriting a 8TB disk prior to taking it to crematorium, i'd rather urandom, i dun presently have a 200M/sec rng
asciilifeform: but yes if you're picking the next piece to fall in 'tetris', just use own prng ☟︎
a111: Logged on 2018-10-12 09:00 mircea_popescu: there's absolutely no excuse for having "urandom" as a kernel signal. applications that both a) care about entropy debit over time and b) can get away with substituting shit for entropy should simply manage their entropy/shitropy interface in a dedicated thread. let it read from /dev/random, add however many bits of 11110000 they want whenever they want to and vomit the resulting cesspool as the app that spawned them demands.
asciilifeform: http://btcbase.org/log/2018-10-12#1860778 << there are not so many legitimate uses for /dev/urandom. however the idea that it can be fully reproduced in userland without kernel knob is afaik a mistake -- the thing gives you real entropy if available, and elsewise prngolade; importantly, as a ~nonblocking~ operation. idea is that it ~always~ returns in constant time. ☝︎☟︎
a111: Logged on 2018-10-12 08:40 mircea_popescu: this creates a flow issue ?
asciilifeform: http://btcbase.org/log/2018-10-12#1860773 << sane unixes will ~lock~ it, so errybody but 1 starves. insane ones will feed same randomola to multiple threads.. ☝︎
a111: Logged on 2018-10-12 08:32 mircea_popescu: http://btcbase.org/log/2018-10-11#1860712 << suppose you stat ~the file you're about to produce~ rather than stat "the file you produced already" ?
asciilifeform: http://btcbase.org/log/2018-10-12#1860762 << that was the contemplated stat(), yes. the sad is that idjit unix does not have a general-purpose 'this block of program is atomic' primitive, and so other cpu can fuck you b/w the stat() and open() call ( tho phf did find a open() flag that supposedly atomicises ) ☝︎
asciilifeform: http://btcbase.org/log/2018-10-12#1860756 << lcd , of any type yer likely to meet with, draws =~ current regardless of what is displayed ☝︎☟︎
ave1: diana_coman, I'll add my test and fix the typo and regrind
diana_coman: you might want perhaps to add a call + check to it in the tests too?
diana_coman: typo in manifest "emplementation" -> implementation ; but one can live with that
diana_coman: nice, thanks!
diana_coman: keeping it separate certainly makes sense (so that it's not a mandatory piece in there) but if it's code it's still basically under V that it belongs
ave1: as for urandom, whenever I search for it I end up revisiting this one http://btcbase.org/log/2017-02-15#1614105. It seems both are unusable. ☝︎
diana_coman: there is also the bit-keccak in eucrypt and that is the same really - reference code rather than production code ☟︎
ave1: I'll make it a vpatch later today...
ave1: well mostly because I see this more as reference code (this is how it can be done with division) and less so as production code (the table driven works very well for that) ☟︎
diana_coman: ave1, at any rate, <pre> tags added, looks good
diana_coman: ave1, why not add it as a .vpatch on the eucrypt tree?
mircea_popescu: there's absolutely no excuse for having "urandom" as a kernel signal. applications that both a) care about entropy debit over time and b) can get away with substituting shit for entropy should simply manage their entropy/shitropy interface in a dedicated thread. let it read from /dev/random, add however many bits of 11110000 they want whenever they want to and vomit the resulting cesspool as the app that spawned them demands. ☟︎
mircea_popescu: there shall be ONE random. and if there isn't one, throw the shirt iron out with the garbage. and if there's "two", throw the "engineer" responsible out with the garbage.
mircea_popescu: (but, importantly, /dev/random === /dev/urandom is a fundamental ideological patch. fuck the fucking idiocy!)
mircea_popescu: i'm naively assuming it wouldn't, but in fairness haven't actually looked into the gnarl.
ave1: now, I'll have to check
mircea_popescu: i don't get it. you have one (kernel) reading from tty putting it into /dev/?random ; and however many apps reading from there as now.
ave1: no, that you have multiple concurrent applications all reading from the tty at the same time. Some kind of intermediate is needed to divide the bytes.
mircea_popescu: is the idea that the pool would; run out ?
ave1: the wrappers read directly from the tty and work one processor at the time. ☟︎
mircea_popescu: http://btcbase.org/log/2018-10-11#1860743 << you conceivably might want to review contentious calls at some point or w/e. but kinda like cctv tapes, ~never useful. ☝︎☟︎
a111: Logged on 2018-10-11 19:59 phf: but it also seems that before that becomes reality we either have to patch linux kernel or implement a rng daemon or somesuch
mircea_popescu: http://btcbase.org/log/2018-10-11#1860728 << afaik the agremeent on that was "rewrite dev/random and dev/urandom to be ~ diana_coman 's fg wrappers. ☝︎
a111: Logged on 2018-10-11 19:36 phf: asciilifeform: nah, you want a random seed always, because you're trying to avoid race by producing a least likely collision. the race will always exist between existance check and initial creation
mircea_popescu: http://btcbase.org/log/2018-10-11#1860712 << suppose you stat ~the file you're about to produce~ rather than stat "the file you produced already" ? ☝︎☟︎
a111: Logged on 2018-10-11 19:34 asciilifeform: phf, bvt : i thought of a possible algo for sane tmp file creation that dun need rng or global counter. 1) pick a file name in tmp dir, if none exists, take empty string, as string S 2) produce S' = H(S) , H is hash (e.g. keccak) 3) stat(S') ; if already exists, take S'' = H(S') and repeat .
mircea_popescu: http://btcbase.org/log/2018-10-11#1860710 << standard algo, guaranteed to finish in at most n steps. ☝︎
a111: Logged on 2018-10-11 17:08 asciilifeform: draws 400mW in reflector mode (~1/5 of what colour panel of that weight class eats) .
ave1: diana_coman, http://ossasepia.com/2018/10/10/eucrypt-chapter-14-crc32-implementation-with-lookup-table/comment-page-1/, it seems that wordpress filtered out the <code> tags, could you edit / fix these (I should have used pre probably)
phf: aye, thank you
trinque: phf: appears to have worked; give it a shot
phf: trinque: i've updated the key, so all you need to do is an import http://btcbase.org/data/wot/phf.asc i'm not going to do a minimization right now
mod6: TomServo: This weather is getting ole mod6 down.
BingoBoingo: Welcome back TomServo
BingoBoingo: !!up TomServo
bvt: i will have a look at what primitives i need for the implementation today and tomorrow, and will come back here if i get stuck on something.
asciilifeform: phf: seems like oughta work ( aside from boxes where /tmp is on nfs drive, but that's a perversity imho )
phf: asciilifeform: so the posix solution is to call open with O_CREAT | O_EXCL, which will attempt to create but will signal EEXIST if the file already exists. curiously default behavior O_CREAT will simply clobber ☟︎
phf: tmp file gets created when you have any sort of patching operation, gets cleaned up when you no longer need it. but you could conceivably have it for the duration of the runtime. (except instead of doing file move at the end of a single file patch, you do file copy)
phf: bvt: that doesn't contradict what i said
bvt: uhm, strace shows me here that the temporary filename changes
phf: you're probably right though, the temp_file_name interface needs to be replaced with open_temp_file
asciilifeform: ( and it is my understanding that you can, under all known unixen, simply by locking e.g /tmp/tmsr or whatever fixed path )
phf: asciilifeform: oh yeah it is, hash a bunch of random stuff, check for collision. that's the pragmatic solution. i'm talking about the take aways
asciilifeform: phf: if vpatch in particular cannot be made to work 100% reliably on existing kernels, we'll have bootstrapping boojum.
phf: but it also seems that before that becomes reality we either have to patch linux kernel or implement a rng daemon or somesuch ☟︎
phf: asciilifeform: i believe that was the real conclusion of yesterday's conversation: you want easy access to a real rng from any republican process at any time, and that makes a lot of idiotic problems go away
phf: bvt: as of right now there's only one temporary file at a time during a vpatch run, so you're essentially in the same situation temp file name or temp directory anme
asciilifeform: if you have a working rng, all you gotta do is take 256 bits from it, and no need to stat() or anyffing; the chance of collision is coupla orders of magnitude smaller than asteroid flattening the machine
bvt: perhaps better way would be to create a temporary directory (for which there is atomic create-or-die) at the vpatch start, and place all temporary files there?
asciilifeform: i suspect however that any solution that works across generic unixdom (i.e. doesn't marry linux) will appear similarly ugly
asciilifeform: phf: also can lock a fixed name in /tmp when incrementing counter or whatever wants to be atomic. tho imho it's ugly
phf: asciilifeform: counters exhibit exactly the same problem as a non-random seed
bvt: and i'm don't know yet if files created this way can be mv'ed to file system
phf: asciilifeform: well, temp_file_name makes a temporary _name_, if you're relying on an atomic create, then you need to change the interface, where you try and create a temp file, and then bail. in that case though you don't even need any randomness. just use a counter
bvt: there is a recent linux O_TMPFILE flag to open(2), but i'm not sure we want to use that
asciilifeform looked, didn't find, assumed that i simply didn't look hard enuff
bvt: right, Ada.Directories.Exists was the missing piece.
asciilifeform: phf: there's no atomic file create-or-die ?
phf: asciilifeform: nah, you want a random seed always, because you're trying to avoid race by producing a least likely collision. the race will always exist between existance check and initial creation ☟︎
phf: bvt: this doesn't need much C, it's a loop with a termination condition of not Ada.Directories.Exists(Filename), and the tricky part is the construction of said Filename. it could be a Prefix ("vpatch.") + keccak hash(salt + pid + time + ???).
asciilifeform: phf, bvt : i thought of a possible algo for sane tmp file creation that dun need rng or global counter. 1) pick a file name in tmp dir, if none exists, take empty string, as string S 2) produce S' = H(S) , H is hash (e.g. keccak) 3) stat(S') ; if already exists, take S'' = H(S') and repeat . ☟︎
bvt: i will give it a try. i'm still learning ada, will have to take a look at how strings and interfacing with C are done properly.
phf: bvt: you know you could take a stab at proper temp_file_name :> it'll have to be written anyway, to fix this bug. i suspect by the time someone else runs into this issue, a replacement will already be available.
phf: i've used a variety, including the two you mentioned. ave1's, adacore 2016 and 2018 on linux and mac
bvt: did you use ave1's gnat or the adacore-provided distro? My understanding is that it should work with both, taking 2 different code paths, and fail only with gnat17.
phf: well, it's also a reason why bug wasn't caught in development. a sequence of wtfs: linux man page says mktemp should be 3 or more X's, so project builds on a non-musl build. meanwhile POSIX mandates there to be exactly 6 X's, so a musl build fails to produce a random string, returning instead a blank one, which is when gnat decides to not only generate a temp file but also do cleanup.
bvt: that was a wtf moment for me, i'm not used to such service from the runtime
phf: i like that ada will create a temp file for you and then clean it up on empty string, though that's a bit too magical (the interface, not the behavior)
bvt: I understand this, but still it's better to drop the vpatch, at least in the logs, in case someone else stumbles upon this problem.
lobbesbot: phf: Sent 1 hour and 48 minutes ago: <asciilifeform> i recently found that lcd panel made by (defunkt) 'pixelqi co', pq3qi-01 , is still sold by chinese ( i have nfi if clones, or old stock ), ~100 $ ea; the thing toggles from 1024x600 colour/backlit into a 3072x600 greyscale reflector thing, worx with various lappies ( sadly not x60, it's a 16:9 )
phf: bvt: i think the conclusion of yesterday's thread was that using mktemp is a bad idea, and that it's better to rewrite temp_file_name to be self contained, i.e. get rid of mktemp pragma and have temp_file_name do its own random string/stat loop
bvt: Apparently those addresses are used as 'poor man's RNG'
bvt: 3. Musl calculates a simple hash over current time, address of variable on stack, and address of template to generate the random characters for mk*temp family of functions: http://git.musl-libc.org/cgit/musl/tree/src/temp/__randname.c#n6