phf: there's no "escape" as such. instead it generates an adhoc escape (say replace dot with ZZ or whatever) and then patches that one line using s///.

phf: well, that's not inband magic! still "ick"

phf: it's \n.\n

phf: newline is not a non-printable ☟︎

phf: ffs, i resent being placed in a position of defending something that i'm not responsible nor care for. diff -e would've been closer to "teco macros", but it's the "sane teco macros" we're talking about here, etc. etc. etc.

phf: picture a spherical horse in vacuum (tm) (c)

phf: asciilifeform: i will have to concede, (graphic-char-p #\newline) NIL

phf: well, (graphic-char-p #\space) T

phf: hehe, as much? no. but that's only because if there's one lesson i learned from naggum, all this rage is not healthy.

phf: also i learned long time ago that americans* aren't taught how to argue properly, so when they do they have a really hard time keeping the thread, keeping more than one point, developing an argument, bringing it back to original point, etc. consensus intelligentsia are all very civil, so when you do get them railed up, they just flail, sort of discourages from even trying

phf: ben_vulpes: well, we're kind of constrained by the hardcode -p1 behavior, but i've no idea if that's an implementation detail or a spec

phf: of v that is

phf: so the simplest solution would be to at least parametrize an equivalent of -p1 on lisp side

phf: -p1 means a/foo/bar gets pressed as foo/bar

phf: with p2 same guy gets pressed as bar

phf: i think it treats one of the names as canonical ☟︎

phf: actually patch seems to do ... something magical

phf: that's what btcbase does basically. it finds position where common suffix starts and then works from there..

phf: but i'm starting to think it's an overkill anyway, because it doesn't accommodate for all possible insane patch inputs.

phf: patch/diff lets you have a patch with --- foo +++ bar in which case it seems to ~check if foo exists, then try and press against foo, otherwise press against bar~

phf: so if you were to produce a patch with a/old-veh.lisp and b/veh.lisp. existing vtrons will happily press it, though it's a total clusterfuck ☟︎

phf: actually that's a bad example because that'll work, but a/old.lisp and b/veh.lisp

phf: ben_vulpes: where'd that 0 come from?

phf: also you're missing genesis

phf: http://wotpaste.cascadianhacker.com/pastes/udP9e/?raw=true

phf: also the old school way of making a genesis http://p.bvulpes.com/pastes/JQxyU/?raw=true

phf: tee writes piped input to stdout and to a file

phf: vdiff here http://p.bvulpes.com/pastes/r01nj/?raw=true it's the same old vdiff except if you pipe into it, it assumes you're piping in a patch, otherwise it acts as normal vdiff

phf: http://btcbase.org/log/2016-06-20#1485519 ☝︎

phf: will probably cover majority of cases(tm)(c)

phf: huh, apparently plan 9's diff/patch doesn't even implement unified diff format

phf: i noticed that btcbase supports filenames with spaces in them: if you start a filename with " it will read until a closing ". i have no idea where i got this from, because gnu diff/patch don't support spaces in names.

phf: ben_vulpes: you mean like ~parse~ the output of patch?

phf: aah

phf: so you basically snapshot your entire dataset back into the database at certain times, and snapshot is an equivalent of set merge?

phf: asciilifeform: oh so you do insert to a set, every time there's a result, and you query for the whole set before you start a cycle of process?

phf: asciilifeform: i'm just trying to establish the dataflow here, for my own curiosity

phf: well, a more practical approach would be to adapt phuctor c part to a postgresql loadable module interface. in which case you he will eliminate the cross-boundary overhead (serialize/deserialize over the "wire").

phf: asciilifeform: did you understand what i said?

phf: what i'm saying is that a significant fraction of "1000s of queries AND ..." is the cross-boundary. you compile queries on c side, you send them to psql, it then parses, prepares results, serializes, sends it to c side, c side has to now parse all over again

phf: precompiled queries are a fraction of cross-boundary issue

phf: i'm not even arguing with you, i'm saying that the ~full extent~ of what "move it to psql" is going to do is ~eliminate cross-boundary issue~ that is all. so it'll shave some significant overhead, but it's not a silver bullet.

phf: asciilifeform: right, i was going to get to that :}

phf: a sort of impolite question, but is there's an index on hash column?

phf: is it a hash index? it has the least overhead (it isn't logged amongs other things, so you have to rebuild it on crash, but conversly it's kept in memory and only supports = operation) indexes will make your queries more cheap, but writes more expensive, so you want to make sure it's the cheapest possible

phf: asciilifeform: no, it's a command that you run, like REINDEX index_of_things; it simply queries what's already in DB and warms up the cash

phf: pretty sure not on postgresql, they are strict about their acid

phf: typically you handle it by not making your query lock the entire table, using a where clause of some sort. like if you're inserting things in batches, you can use a batch counter, and you query against max last known batch counter or less (or a variation of)

phf: jurov: well, it's not clear where "disregard all locks" comes from in the original request. if the actual operations are as asciilifeform describes, i.e. sporadic inserts, and sporadic selects, then there will be no locks. my point is that there's no "disregard all locks" in postgresql, you solve it by knowing what lock you're hitting, and then designing your query to sidestep the lock

phf: mircea_popescu: sop outside of mysql world. dirty read is considered a liability, so whole point of db systems design is to ensure that you don't hit locks when you shouldn't.

phf: mircea_popescu: yes, ~having to deal with locks~ happens past the limit of db designer's competence

phf: mysql solution is to creatively relax acid and hope things will "just work", which is the flip side of "mysql crashes all the time"

phf: mircea_popescu: i'm not quite groking what the bad write is. are you saying that instead of intermingling writes and reads, you should batch them, and not write while you're reading?

phf: mircea_popescu: that's not quite what i'm saying.

phf: that's not even close to what i'm saying though.

phf: you have your basic database requirements: atomicity, consistency, isolation and durability. these are axiomatic, you either expect them to hold or there's no point in further elaboration. at least SQL from the conception guaranteed the four requirements. "dirty read" violates consistency. your table might be half way through an update, you do a "dirty read", which is necessarily faster than update, and you have half the results with

phf: old values, other half with new values. you ~can~ guarantee four requirements ~without~ using locks

phf: but that's the slowest option, so you have strategies for increase of speed that involve strategic placement of locks

phf: well, "you either expect" is because ~sql~ as a db language is specified to have acid. there are databases that support dirty reads/writes they are just not "sql"

phf: davout: no, nor does oracle :o

phf: i'm wrong, sql92 allows dirty read in read uncommitted

phf: davout: you don't get consistent, uninterrupted, sequential chain of blocks. the actual distribution pattern is a mess, that "orphanage" was bandaiding

phf: not just postgresql mind you. oracle definitely, mssql as far as i know

phf: dirty read would definitely solve me a lot of headache now, though not enough motivation to switch to mysql. not so much when i worked on oracle for a g-sib where you want acid, so instead "avoid bad writes"

phf: well, it's also reason why the kind of stuff you could run on a beefy dreamhost now requires $5k/mo amazon rds instance

phf: mircea_popescu: rds is amazon's hosted relational database solution(tm) which is a postgresql on a unixbox

phf: true, but alf also "won't touch the web"

phf: for the republic, not for hitler. he lacks that certain "good german" spirit, ja

phf: everyone sensing there ought to be a fight, but everyone's agreeing

phf: ftr, least i somehow become sql acid proponent, i'd like to point out that i'm the only person running tmsr infrastructure ~not on a sql database~

phf: no wai

phf: asciilifeform: a log is just push-vector with checkpoints. entire thing barely takes up 200mb of in memory data

phf: sbcl right now is at 450mb on average, which drops to 270mb after full gc

phf: har