asciilifeform: http://btcbase.org/log/2017-04-07#1639884 << is it not obvious how creating MORE things that gotta be 'nuked later' is a problem ? ☝︎

asciilifeform: it is one of the main reasons for phuctor at all.

asciilifeform: http://btcbase.org/log/2017-04-07#1639881 << i'll say that being able to paste in an unknown base64 pgp key and see a search result NOW, is not negotiable. ☝︎

asciilifeform: and this requires guaranteed consistency. right now i have this guarantee because there is one db, and not a multilevel morass.

asciilifeform: http://btcbase.org/log/2017-04-07#1639880 << imho this is a mistake. the presentation of results is not ~cleanly~ separable from the generation, because certain situations (dupes, or the omission of a concordance factor/modulus/key linkage) must be HARD-guaranteed not to happen ☝︎

asciilifeform: imho this is proper.

asciilifeform: phuctor, for instance, dropped ~half of its length each time i rewrote.

asciilifeform: it does.

asciilifeform: while retaining what must be retained

asciilifeform: that is, just how separable the light is from the heavy

asciilifeform: this remains to be seen

asciilifeform: trinque: your bot, while very spiffy, and necessary, is not an intrinsically heavy item

asciilifeform: trinque: understand, first thing, and only thing , that a brute labourer would say re phuctor, is 'buy $mil of replication', 'use clouds', etc

asciilifeform: aaaaaah looks like i gotta paste in http://btcbase.org/log/2017-02-19#1615434 again ☝︎

asciilifeform: tbh i dun think anybody is getting rich on fg...

asciilifeform: which is why fixes that retain it, seem like total waste of time to me ( i put a good deal of time into experimenting with db configs the last time we had this thread. and it made 0 measurable difference, for instance. )

asciilifeform: trinque: i'd much like to get off it

asciilifeform: *unreasonable

asciilifeform: http://btcbase.org/log/2017-04-07#1639628 << 'put wifi in everything! only an unteasonable terrorist would refuse!!' -- everyone not-tmsr ☝︎

asciilifeform: orbitz

asciilifeform: mircea_popescu: python certainly isn't

asciilifeform: instead of working-but-complained-about.

asciilifeform: if i had insisted, phuctor would still be a waited-for thing ☟︎

asciilifeform: but turns out, the necessary work is gargantuan.

asciilifeform: and yes, originally phuctor front end was to be a cl proggy

asciilifeform: remember, it also has to handle arbitrary binary garbage without misbehaving.

asciilifeform: in cl

asciilifeform: it dun exist

asciilifeform: trinque: you wanna write gpg parser ? that handles all cases ?

asciilifeform: i'd luvvv to know in what to redo this

asciilifeform: but now it means two wholly separate subsystems

asciilifeform: now i can make a separate queuer just for offline subs

asciilifeform: aha, nice, but not compatible with queueing.

asciilifeform: yes.

asciilifeform: the 'do we know it'

asciilifeform: mircea_popescu: not the phuctoring!

asciilifeform: it is.

asciilifeform: trinque: another problem: i thought of having new submissions go to a queue, instead of main db. but! that would nuke the vey valuable 'paste in an unknown pgp key and know ~immediately~' feature.

asciilifeform: mircea_popescu: i think he was proposing it as an ad hoc parallelism

asciilifeform: Framedragger: problem is that the cache gotta expire some time.

asciilifeform: Framedragger: i have it already

asciilifeform: aah yeah

asciilifeform: and i certainly don't want html in my db wtf

asciilifeform: trinque: i don't like state

asciilifeform: trinque: that sounds potentially useful. i'll have to investigate

asciilifeform: a very recent one . but i'll have to get back to you Framedragger re : which

asciilifeform: so why then is this not standard? why locks exist at all ?! ☟︎

asciilifeform: what actually happens if you read a row being written

asciilifeform: do they create chance of reading liquishit ?

asciilifeform: do they create risk of data loss ?

asciilifeform: Framedragger: briefly describe what these do ?

asciilifeform: lel

asciilifeform: aah

asciilifeform: http://btcbase.org/log/2017-04-07#1639630 << also vanished? ☝︎

asciilifeform: the wikilicks dump?

asciilifeform: http://btcbase.org/log/2017-04-07#1639625 << vanished... ☝︎

asciilifeform: www is fundamentally retarded paradigm

asciilifeform: and yes trinque

asciilifeform: that can actually do things without being prodded.

asciilifeform: but again needs multithreaded proggy.

asciilifeform: this - yes

asciilifeform: aah

asciilifeform: mircea_popescu: trilema doesn't potentially update 100 / day

asciilifeform: mno. i refuse to marry sql db !!!

asciilifeform: instead of rss?!

asciilifeform: what's that do

asciilifeform: it would not do to have them all come out at once

asciilifeform: another gotcha is that the process that does the actual crunching is written in c and communicates solely via the db. it spends ~3min a day reading, but then ~all day checking 'do we already have this factor? ' writing it, in short bursts. but each of these bursts must show up in rss.

asciilifeform: ergo depythonization

asciilifeform: and yes trinque is right, it needs multiprocess queues, etc

asciilifeform: unfortunately not doable without full rewrite.

asciilifeform puts on list.

asciilifeform: ^ good idea

asciilifeform: maybe i misunderstand what means 'update' ?

asciilifeform: and yes this can be done in bursts, as mircea_popescu described a while ago. but still gotta query whole db for mods

asciilifeform: and for that matter, if key is new at all

asciilifeform: new key entered? must find if we know its mods, any of them, already

asciilifeform: observe how phuctor is structured

asciilifeform: no, if only

asciilifeform: and so is 'set all row to c'

asciilifeform: formatting disk is fast!11

asciilifeform: yes

asciilifeform: but ues

asciilifeform: or was that after

asciilifeform: mircea_popescu: fwiw i was not able to load trilema while this was taking place.

asciilifeform: mircea_popescu: db vacuum is not a sequence of writes for the purpose of subj

asciilifeform: trinque: one of these days you gotta expand on how you did it

asciilifeform: who is writing 1000 writes / sec to trilema ?!

asciilifeform: pg also claims to. the knob did 0.

asciilifeform: all sqltrons have same fundamental problem.

asciilifeform has tried it in the past.

asciilifeform: mno.

asciilifeform: postgres must die.

asciilifeform: to run it regularly and still be able to do anything else.

asciilifeform: the problem is that there is simply not enough db 'bandwidth'

asciilifeform: trinque: correct

asciilifeform: the use of python for frontend was a mistake. i did it to avoid having to write gpg disasmer from 0, and www liquishit from 0

asciilifeform: http://btcbase.org/log/2017-04-07#1639647 << in point of fact this is not simple -- right now the thing is wholly reactive -- proggy only runs per request. and is written in a lang having no actual threading... ☝︎

asciilifeform: http://btcbase.org/log/2017-04-07#1639649 << i would like to static the whole thing, but currently have nfi how to do that while retaining the 'what are all keys with mod M ?' , 'what are all mods with factor F?' , 'search for string S in userids' etc ☝︎

asciilifeform: http://btcbase.org/log/2017-04-07#1639653 << correct ☝︎