67900+ entries in 0.487s
Framedragger: can you not picture this setup with
a read-only slave, with
a *separate* "give new key to master" interface (separate so that the replication doesn't become write-write but stays write-read)
Framedragger: alright. i will grant you that the end system wouldn't be "oh so elegantly simple", because you would have to have
a submission queue (maybe something that trinque had in mind). user submits key; gets permalink (immediately); meanwhile key gets sent to master (immediately), and master puts it into "to be inserted" queue. under normal loads, the insertion happens ~immediately,
☟︎ Framedragger: yes that's the tricky part. my natural answer to that would be to "drop it, have user be able to come back right away - to
a permalink - but results only displayed when the upstream db actually processes and inserts the new entry. (this entry would then get fed into the slave via streaming replication)"
Framedragger: are you absolutely unwilling to have
a delay there?
Framedragger: well, i'm not that certain, but i am assuming you have more experience there with me. i will only remark that you merely need
a *read slave*, not an actual mirror db which can handle writes and sync state. the syncing would go one way only. (hence the multiple references to pg streaming replication.)
Framedragger: now, complexity management-wise.. maybe; having same person manage both boxes may not be best idea (and the alternative has its own advantages). but the current "phuctor is down, i don't know why, it's
a black box" isn't the greatest example of current setup, either. (this may be
a red herring, i'm not sure)
Framedragger: of course being able to point to
a working prototype would do so much more than arguments. unfortunately that would most likely require the needed modifications on the phuctor box, so
a bit of chicken-and-egg.
Framedragger: asciilifeform: hold on. the idea was to separate reads from writes. having
a separate box for www which gets updates from phuctor box, and having pg indices on it for quick search is *not* resource-intensive. i can cite examples but basically i'm quite certain that
a <= 16gb memory box would suffice. phuctor box is 256 gigs yes, but it does *so much more*.
Framedragger: asciilifeform: the parallel to "this is how winblowz blew up" breaks, imho, if you consider the splitting-off of www not as an addition, but as actual splitting-off, i.e., the box with phuctor on it may no longer have
a www interface (just an option, i know you may be against it). if you picture it that way, it's more about modularisation vs. fixing and inflating
a single monolithic thing.
a111: Logged on 2017-04-07 14:44 trinque: mircea_popescu: more clearly stated, I do not see
a www as part of the algorithm of phuctor. it is one source of input where there could be many, and one output idem. with
a clearly defined line between www and phuctor (even allowing for that www may require cached copies of phuctor data to operate properly), this gives you something you can nuke later and replace.
mircea_popescu: no i don't mean that. i just mean, yes in principl;e doing something with the plebs is not
a bad idea ; but the exact what and wherefore coulo use more conteplation.
trinque: mircea_popescu: more clearly stated, I do not see
a www as part of the algorithm of phuctor. it is one source of input where there could be many, and one output idem. with
a clearly defined line between www and phuctor (even allowing for that www may require cached copies of phuctor data to operate properly), this gives you something you can nuke later and replace.
☟︎ shinohai:
http://btcbase.org/log/2017-04-07#1639637 <<< This is what I said to Steemit evangelist R. Hilarski
a few days ago when he accused me of just hating on altcoins. "When you lose all your BTC in scams, at least your wife is kinda hot so she can come by trilema and bare her tits for bits."
☝︎ a111: Logged on 2017-02-19 03:54 asciilifeform: (iirc we had
a thread where i described how corporate ameritards, if given
a problem like phuctor, would happily soak up
a few $mil and megawatt of iron)
trinque: really the right thing to do here is sell
a buncha fuckgoats, get asciilifeform rich, hire stable boy
trinque: which is only
a thing because www is 'daddy buy me
a pony now!'
Framedragger: (when you query
a mat view, data is returned directly from it, without touching the source live table)
trinque: thing's designed for parallelism yes, but of the same exact data in same spot is
a big ask
trinque: it's
a speedup for someone loading the stats page
mircea_popescu: trinque> materialized views are nothing more than
a named query which stores its results in
a table << this is
a convenience not
a speed-up
trinque: it grew beyond this scope the same reason bash did, or anything else that was originally
a user interface
Framedragger: btw, i once screwed around with memoization in flask. iirc something can be done here as well, asciilifeform (it may be as trivial as adding
a decorator before the function which handles GET request)
trinque: it's
a goddamned spreadsheet; you're supposed to compute elsewhere and then fart your results in there so somebody managerial can ask it questions
trinque: so here's what relational db was supposed to be for, like bash originally had
a 'supposed to be for'
Framedragger: and ^ is effectively
a cache of sorts (useful here)
shinohai: lol Jason Dreyzehner "Head of Design" at buttpay followed me on twitter. Perhaps staring at T&
A will be useful in making UI decisions.
trinque: materialized views are nothing more than
a named query which stores its results in
a table
Framedragger: again, no, and as per ACID, no dirty reads (*not
a bad thing here*)
Framedragger: index-only scans allow to search in
a table without 'touching', or 'locking' the actual rows (which may be simultaneously written-to, at the same time)
mircea_popescu: go right ahead, produce
a list of 5k rss items, THEN dole them out slowly.
trinque: could get me
a pg connection and I could listen for pg_notify on
a trigger of that table
mircea_popescu:
a db write is either insert or update ; finding out things is usually
a select. it ~may~ be worthwhile to separate your reads from the writes, because it is technically possible (thopugh i'd hope unlikely) the db is dumb enough to put write locks in for something like "update x on y" even though it should be "select y if z then update x".
trinque: oh and 3) rendering www is
a stage of work
trinque: two thoughts, beyond which
a look at the actual thing would be needed. 1) queues and workers, always 2) db acts as your queue, schema reflects the stages of work
trinque: man, I've worked on pg databases that did nothing but write *and* read concurrently while sitting on
a few hundred million rows
trinque: there's already
a function that renders that page...
a111: Logged on 2017-04-07 13:12 trinque: simple fix is to crap out
a stats page static file at interval.
trinque: simple fix is to crap out
a stats page static file at interval.
☟︎ mircea_popescu: btw, whosoever wants
a good dose of mosfilm in english, try "west point story". cagney's blond even.
BingoBoingo: "Member when we had
a white president who liked our country"
mircea_popescu: more accurately, it'd look like what
a thing that looked like something used to look.
BingoBoingo: "Support our troops" took Dubya approval ratings from toilet to 80%+ Imagine what
a war that ended would do!
mircea_popescu: so spend 2mn to make
a pothole costs 300 bucks to fix ?
mircea_popescu: i might be
a little slow on the uptake, but what is tomahawk supposed to do to airstrip ?
mircea_popescu: lmao "one count of possession of
a silencer" ? really ?
mircea_popescu: ah lol. i thought for
a moment they actually were going for the fortifications there.
mircea_popescu: trinque well it'll be
a little difficult to start another iraq seeing how the ru planes actually work.
BingoBoingo: "“Unlike the previous administration, President Trump confronted
a pivotal moment in Syria and took action,” said Sens. John McCain (R-AZ) and Lindsey Graham (R-SC)." << Apparently Trump satisfied the Lich's war boner.
BingoBoingo: Bah, those are smaller ships and
a couple converted ballistic missile submarines
BingoBoingo: And recently russian has expressed "limits" to support of Assad. Perhaps
A-10 and Su-25 regime change Syria in harmony to MWGA
mircea_popescu: for the record, i was trying to watch
a comedy. half hour in i'm like... feh, i wonder what trilema's up to
mircea_popescu: except it was
a distance the helicopter actually flew at.
mircea_popescu: know from backchannels and the fact that the govt isn't making
a point "oh, they had bazooka"
mircea_popescu: check it out asciilifeform isis shot down
a helicopter
Framedragger: (apparently some couple of russians had
a fight and one shot the other over
a dispute regarding kant. i can believe that)
trinque: I by way of
a sad life always write the where clause first
mircea_popescu: and then it crashed because suddenly turning 10s of ks of article into
a 3mb turd inflated the db.
Framedragger: i mean even if mircea_popescu aborted
a db write, it shouldn't have forked into gigs of crap?..
CompanionCube: in the sense that it/the dump tool produced
a overly-large 40gb file?