77200+ entries in 0.701s
trinque: consider the case where
a form is generated based on db state, and the validity of that form depends on db state
phf: mircea_popescu: rds is amazon's hosted relational database solution(tm) which is
a postgresql on
a unixbox
phf: well, it's also reason why the kind of stuff you could run on
a beefy dreamhost now requires $5k/mo amazon rds instance
a111: Logged on 2016-12-30 17:56 mircea_popescu: jurov no ; but i am fine with wwwtron ocasionally reading
a field that has meanwhile been updated, and giving old, of an unspecified age but less than x time.
phf: dirty read would definitely solve me
a lot of headache now, though not enough motivation to switch to mysql. not so much when i worked on oracle for
a g-sib where you want acid, so instead "avoid bad writes"
mircea_popescu: dude the fact that every other girl in your class is
a slut isn't going to feed you or your baby.
davout: ok. imma give it
a shot
phf: davout: you don't get consistent, uninterrupted, sequential chain of blocks. the actual distribution pattern is
a mess, that "orphanage" was bandaiding
mircea_popescu: filtering
a chain out of the soup outside like BingoBoingo is not without merit.
davout: well, either
a block verifies, or it doesn't
davout: asciilifeform: it does have
a command that shits hex at me given
a block hash
a111: Logged on 2016-12-30 17:40 davout: maybe i'm too lazy to script this and can live with waiting
a month to sync!
davout: phf: you're saying postgresql doesn't have
a "read uncommitted" transaction isolation level like innodb?
phf: well, "you either expect" is because ~sql~ as
a db language is specified to have acid. there are databases that support dirty reads/writes they are just not "sql"
mircea_popescu: and i'm supposed to care about the fact that they don't know how to write
a db that doesn't spit out passwd ?
mircea_popescu: and THIS is what i mean re "problems in the field". whopee, idiots who can't code still want to be "at the forefront of computing" so they made
a modern db that doesn't work.
davout: jurov: i think it's more like nobody gives
a shit if static wwwtron is out of sync with DB
mircea_popescu: jurov no ; but i am fine with wwwtron ocasionally reading
a field that has meanwhile been updated, and giving old, of an unspecified age but less than x time.
☟︎ mircea_popescu: and here's exactly the problem of superficiality : "you either expect consistency or there's no point in discussing". there's LEVELS. maybe i expect all my writes to be consistent and don't care by
A CLASS of reads being consistent. this is
a consistency model that's consistent.
mircea_popescu: meanwhile inconsistency within the actual db are
a different matter.
mircea_popescu: phf here's the problem : moder(field) consists of take field, redefine it in
a practically useless but superficially persuasive way, then bad_words() to whoever dares ask if your "field" solves any important questions in the field. because of course it doesn't, MIT is the premier institution in science(*) and technology(*) in the werld.
☟︎ phf: you have your basic database requirements: atomicity, consistency, isolation and durability. these are axiomatic, you either expect them to hold or there's no point in further elaboration. at least SQL from the conception guaranteed the four requirements. "dirty read" violates consistency. your table might be half way through an update, you do
a "dirty read", which is necessarily faster than update, and you have half the results with
davout: and re locking, how's
a RDBMS to provide ACID guantees without locking?
mircea_popescu: because, again,
a semaphore exists because the user does not know what the user is doing.
davout: maybe i'm too lazy to script this and can live with waiting
a month to sync!
☟︎ davout: so yeah, prb does have
a way to dump
a block to hex from
a block hash, and
a way to get
a block hash from
a height, looks like this could work
mircea_popescu: phf think for
a second : the whole FUCKING POINT of
a semaphore, of any kind, is that user can't know what the other item involved is doing. if they could know, they wouldn't "avoid the locks", they'd avoid the bad write outright.
a111: Logged on 2016-12-30 16:14 asciilifeform: the fastest sync method, supposing one has access to
a synced node, but also supposing that it won't do to simply copy the blocks (and it won't, you want to verify) is an eater-shitter system
phf: mircea_popescu: sop outside of mysql world. dirty read is considered
a liability, so whole point of db systems design is to ensure that you don't hit locks when you shouldn't.
jurov: Flly lockless dirty read is likely
a security hazard (due to race conditions, you may end up reading memory you ough not to)
phf: typically you handle it by not making your query lock the entire table, using
a where clause of some sort. like if you're inserting things in batches, you can use
a batch counter, and you query against max last known batch counter or less (or
a variation of)
mircea_popescu: asciilifeform as to "how to make www respond", you use the method we were discussing last time, whereby www is
a cached image and if out of date tough for viewer ; as to nursery "do we have this ? how about this?" you really want the db to do that for you, it's ~the only thing it;s good for.
mircea_popescu: so : if loading the whole batches of keys through the user-wwwform process is what 99% of the machine time goes to, then yes, put the batches into
a single, sorted query, make the workmem should be 256mb or 2gb or w/e it is you actually need to cover your query (yes this can be calculated, but can also be guessed from
a few tries) and then run bernstein after every such query, on the db not on "nursery" (which yes, it's
a ter
phf: asciilifeform: no, it's
a command that you run, like REINDEX index_of_things; it simply queries what's already in DB and warms up the cash
phf: is it
a hash index? it has the least overhead (it isn't logged amongs other things, so you have to rebuild it on crash, but conversly it's kept in memory and only supports = operation) indexes will make your queries more cheap, but writes more expensive, so you want to make sure it's the cheapest possible
phf:
a sort of impolite question, but is there's an index on hash column?
mircea_popescu: this is
a piddly excuse, "no duplicate logic", case of luser with wwwform and case of 100k keys in batch form are different enough to warrant duplicate logic. that's why computers even exist, to account for such level of difference in code.
mircea_popescu: yes, well, that's then the problem. they should go in as
a single query the size of the batch, with the items sorted within it
phf: i'm not even arguing with you, i'm saying that the ~full extent~ of what "move it to psql" is going to do is ~eliminate cross-boundary issue~ that is all. so it'll shave some significant overhead, but it's not
a silver bullet.
phf: precompiled queries are
a fraction of cross-boundary issue
phf: what i'm saying is that
a significant fraction of "1000s of queries AND ..." is the cross-boundary. you compile queries on c side, you send them to psql, it then parses, prepares results, serializes, sends it to c side, c side has to now parse all over again
phf: well,
a more practical approach would be to adapt phuctor c part to
a postgresql loadable module interface. in which case you he will eliminate the cross-boundary overhead (serialize/deserialize over the "wire").
mircea_popescu: you're not addressing the idea. currently you use
a pile of c code you labeled for purely personal reasons "
a db" to store some data for you, and another pile, you labeled phuctor, to bernstein and do other things on the db-stored data. because the interface is the bottleneck, it then becomes clear you must merge this. one way is to merge by lifting the db code and putting it into phuctor, making it you know, its own db like
mircea_popescu: and you do it as prepared queries, which get precompiled to
a degree
trinque: asciilifeform:
a temp table is in RAM
phf: asciilifeform: oh so you do insert to
a set, every time there's
a result, and you query for the whole set before you start
a cycle of process?
mircea_popescu: you implement bernstein IN the db. it is actually
a programming language.
mircea_popescu: trinque 's idea, bernstein as prepared queries, may be
a gain.
mircea_popescu: and work_mem should prolly be larger than 4mb, but hard to guess how large without
a profile, and this is
a major resource consumption ticket, so you actually want to do some maffs.
mircea_popescu: incidentally asciilifeform since we're now doing open source db optimization shared_buffers is probably
a larger concern. what is it ? defaults to 125mb but i'd readily see it 1-4gb in your case.
a111: Logged on 2016-12-30 14:20 asciilifeform: i even spoke with career dbists, answer was 'your application is monstrous abuse and you need
a cluster'
a111: Logged on 2016-12-30 12:28 davout: was there
a discussion of the use case where one wishes to create, and sign transactions from an arbitrary set of unspent inputs?
mircea_popescu: if he weren't
a total fucking retard on top of being
a consumate conman, he'd actually have 30k to buy an isp now.
mircea_popescu: and for the record : the dood colluded with sonny vleisides / the rest of the 'ndrangheta running "bfl" scam (which, obviously, the usg hasn't ever prosecuted, in spite of loud violation of, eg, parole termas, because hey, partners in crime) to falsely claim that he received
a miner delivery so as to scam bitbet into misresolving
a bet, on which they had ~500 btc.
mod6: when I get
a free moment, i'll throw the latest eulora on there. can be my mining box. :]
mod6: oh, hey, actually. so I've got
a box.
diana_coman: got to some snow, sledged downhill, even bruised
a knee , got back to peaceful coding now , lol
mircea_popescu: asciilifeform planeshift is
a mmorpg that the many-eyes beast took 10 years to make. it uses cs which is
a sort of game engine, which is built on cal3d which is
a gfx lib.
diana_coman: planeshift asciilifeform ,
a swamp not worth gettting stuck into