2100+ entries in 0.015s
Framedragger: i wonder how you'd categorise the 'affect heuristic' as kahneman et al. call it.; i.e., when you make decision with at least partial influence of current emotion. it's very *fast* (thereby falls under kahneman's type 1). ridiculous scenario where it's useful: i'm chased by tiger, see two paths in woods, one has sign 'DANGER' in red, i don't have time to even parse word without making decision,
Framedragger: kk. yeah i won't argue further without doing some decent mp1 thought here :)
Framedragger: (but i'm probably conflating things too much, maybe you'd call such reactions something else)
Framedragger: they are mp2, the one with lower ordinal, and my point was that it's still hella useful. can you employ the methodical and analytical mp1 when you touch hand on hot stove? no time to route through brain.
Framedragger: but you don't see merits of heuristics and biases in humans?
Framedragger: also just to note, (obvs) type 1 (mp-type 2) has its merits and is important. but yeah i got your point
Framedragger: well historically in terms of planet timespan, type 1 got our asses to here..
Framedragger builds hasty internal map of 'mp type 1 is kahneman type 2, mp type 2 is kahneman type 1'..
Framedragger: mircea_popescu: re "nothing here" the idea would be that upon user submitting gpg key, page reloads *instantly* (vs. near-instant page load on current phuctor - reloads only after key inserted into db), shows permalink and "waiting for insertion" msg; html could have property to auto refresh every second until master responds with any links to other keys. but i understand asciilifeform's reasons against this
Framedragger: mircea_popescu: i thought one of the parts was a c proggy. quite certain it is, asciilifeform mentioned it multiple times. python is the web backend afaik
☟︎ Framedragger: last note before i fuck off: the "user submits key; gets permalink (immediately); meanwhile key gets sent to master (immediately), and master puts it into "to be inserted" queue" + "all inserted + updated rows get sent back to slave via streaming replication and pg trigger" doesn't look like petrocheese to me. is all.
Framedragger: (i'll just note that the *permalink* can be derived on the slave box, and nowhere did i say something contradicting this)
☟︎ Framedragger: asciilifeform: yeah i said some stupid things, apologies for depressing you there
Framedragger: if people decide to compromise on things after an agreement was made not to, i mean.. fuck those people
Framedragger: rewriting 60000 lines for this.. how does that follow..
Framedragger: asciilifeform: fwiw i'd be willing to set up a read slave on 16gb dedi box (siphnos, unmetered connection) for fri, that part is not hard imho. sure, if/when decision would be made to *actually expose it to everyone*, things could be moved (and maintained by someone else), but this is one way to prototype things.
Framedragger: (just so i don't feel like just throwing random verbiage at you, what i mean is something like `CREATE TRIGGER setup_send_update AFTER INSERT OR UPDATE ON phuctor FOR EACH ROW EXECUTE PROCEDURE send_update);` - it wouldn't be hard to try.)
Framedragger: that is fine, and a postgres trigger would mean that any affected row gets sent off. i do understand that new-key-addition is a db-intense process, of course
Framedragger: which means, "whenever new row, send it off [without having to execute a new select query etc]"
Framedragger: it is updated in ~realtime from the master via streaming replication
Framedragger: but you no longer reading from the front (in the sense of sql queries), which in db-land is good for performance.
Framedragger quite certain he would get erection from that small c program in phuctor, and that is great
☟︎ Framedragger: it was retarded of me to just throw keywords at you saying "that would do it".
Framedragger: people are telling you that there is a setup which is better but not multi-mil. it does require planning, etc.
Framedragger: you are presenting a false dichotomy consisting of {current phuctor; $multi-mil db setup}. you propose to escape it by writing new db as the only way to escape it.
Framedragger: (again, i did realise that the "oh very simple" angle isn't strong here)
Framedragger: can you not picture this setup with a read-only slave, with a *separate* "give new key to master" interface (separate so that the replication doesn't become write-write but stays write-read)
Framedragger: and you get your 5 seconds. if the db is loaded, it gets inserted "fast", and user is able to see results upon refresh (because when it gets inserted, update gets immediately sent to slave via trigger rule.)
Framedragger: alright. i will grant you that the end system wouldn't be "oh so elegantly simple", because you would have to have a submission queue (maybe something that trinque had in mind). user submits key; gets permalink (immediately); meanwhile key gets sent to master (immediately), and master puts it into "to be inserted" queue. under normal loads, the insertion happens ~immediately,
☟︎ Framedragger: however, you won't agree to drop it, i take it.
Framedragger: yes that's the tricky part. my natural answer to that would be to "drop it, have user be able to come back right away - to a permalink - but results only displayed when the upstream db actually processes and inserts the new entry. (this entry would then get fed into the slave via streaming replication)"
Framedragger: you can give permalink to user on the www read-only box.
Framedragger: hold on, hold on. first of all, regarding the permalink: you can have this anyway, because the hash is computed in python anyway (i ~recall the procedure that you once gave me). so,
Framedragger: because gotta serve all the our democracy right here and now?
Framedragger: are you absolutely unwilling to have a delay there?
Framedragger: asciilifeform: aha, right, that. i literally forgot about that lol. but wait, first we need to clear up the question about whether you want an immediate result to be shown to user upon gpg key submission.
Framedragger: (again tho, it's very easy to just point finger at features, so i'll gladly shut up)
Framedragger: well, i'm not that certain, but i am assuming you have more experience there with me. i will only remark that you merely need a *read slave*, not an actual mirror db which can handle writes and sync state. the syncing would go one way only. (hence the multiple references to pg streaming replication.)
Framedragger: now, complexity management-wise.. maybe; having same person manage both boxes may not be best idea (and the alternative has its own advantages). but the current "phuctor is down, i don't know why, it's a black box" isn't the greatest example of current setup, either. (this may be a red herring, i'm not sure)
Framedragger: bandwidth required for the www box would be ~same as what's currently needed from phuctor.
Framedragger: bandwidth? bandwidth required for phuctor box would be "what's needed to send new rows to this other box".
Framedragger: of course being able to point to a working prototype would do so much more than arguments. unfortunately that would most likely require the needed modifications on the phuctor box, so a bit of chicken-and-egg.
Framedragger: asciilifeform: hold on. the idea was to separate reads from writes. having a separate box for www which gets updates from phuctor box, and having pg indices on it for quick search is *not* resource-intensive. i can cite examples but basically i'm quite certain that a <= 16gb memory box would suffice. phuctor box is 256 gigs yes, but it does *so much more*.
Framedragger: (granted, these are just nice abstract words.)
Framedragger: asciilifeform: the parallel to "this is how winblowz blew up" breaks, imho, if you consider the splitting-off of www not as an addition, but as actual splitting-off, i.e., the box with phuctor on it may no longer have a www interface (just an option, i know you may be against it). if you picture it that way, it's more about modularisation vs. fixing and inflating a single monolithic thing.
Framedragger: the 'www side can live on another server, even' degree of separation may be most easily achievable (given lack of resources to rewrite everything as of now) via pg notify / streaming replication. but maybe asciilifeform would insist that this is 'marrying the db'
☟︎ Framedragger: (when you query a mat view, data is returned directly from it, without touching the source live table)
Framedragger: mircea_popescu: in materialized view, postgres can persist results of query in table-like form. so it *does* cache them; hence you can't say it's convenience only.
Framedragger: btw, i once screwed around with memoization in flask. iirc something can be done here as well, asciilifeform (it may be as trivial as adding a decorator before the function which handles GET request)
Framedragger: ...no, lol, the *results* are 'cached', you still regen html (but this is computationally trivial neh)
Framedragger: and ^ is effectively a cache of sorts (useful here)
Framedragger: obtw which postgres version on phuctor asciilifeform? those two features not available in old versions. i don't remember which ver is in debian stable
Framedragger: again, no, and as per ACID, no dirty reads (*not a bad thing here*)
Framedragger: materialised views - ditto regarding *reading only*
Framedragger: not afaik. these are to do with *reading* only. but trinque may chime in
Framedragger: index-only scans allow to search in a table without 'touching', or 'locking' the actual rows (which may be simultaneously written-to, at the same time)
Framedragger: asciilifeform: have you tried postgres index-only scans? have you tried materialised views? if not, the "postgres must die" sounds sorta surface-level, imho
Framedragger: current situation is that it does that for every web request..
Framedragger: especially as regards *aggregated* stats (the index stats page) - so what if it lags for up to $interval
Framedragger: yes indeed, however asciilifeform's initial objection (besides other less clear unwillingness) was the requirement for phuctor to show 'live data'. which imho is not necessary.
☟︎ Framedragger: bhah she's actually retracting her words so i guess not!
Framedragger: (apparently some couple of russians had a fight and one shot the other over a dispute regarding kant. i can believe that)
Framedragger considers linking to godel's ontological proof of god in Coq
Framedragger: hey backup&restore process tested successfully
Framedragger: mircea_popescu: so you went through every row checking if it has turd at start, and removing that turd? or found binary log to roll back the transaction (which should have aborted and rolled back, something something)
Framedragger: i mean even if mircea_popescu aborted a db write, it shouldn't have forked into gigs of crap?..
Framedragger: (one internet says "No, it is not always reliable when you have binary blobs. In that case you MUST use the "--hex-blob" flag to get correct results." << untrusted source)
Framedragger: mircea_popescu: by text-only i meant the *output*, it can certainly handle binary blobs
Framedragger: but actually, i haven't run mysqldump in forever and not 100% certain about docs, so i don't know
Framedragger: (but output of mysqldump is *supposed* to be a set of sql statements.)
Framedragger: (i don't know about that "db" file, tho, i mean, that one's binary)