sina: good morning sirs/ladies
sina: mircea_popescu: suggestions on avoiding ad-hoc datatypes? one idea I was thinking was for example rather than ../messages/<msg_hash> file with contents "sender,delivered_by,message", have a directory ../messages/<msg_hash>/sender|delivered_by|message files?
a111: Logged on 2017-07-08 20:17 trinque: anyhow, curious what the lisp graybeards have to say about CLOS over the wire
mircea_popescu: i dun have a general suggestion, perhaps just sit down with the whole thing and make an actual data diagram thing ?
a111: Logged on 2017-04-10 16:53 trinque: so we agree that this thing called "database" is really distinct tools which some idiot welded together
a111: Logged on 2017-07-08 22:31 mircea_popescu: one doesn't follow from the other.
mircea_popescu: asciilifeform ah yeah, it wasn't the width, it was that they didn't actually have standardized memory in any sense, or any concept of actula machine word
mircea_popescu: and yes, very much pre-tmsr times the lords were stuck with this "well we gotta cypherpunk, ie, pretend nothing can ever happens -- because if anything were to happen it's straight to vessenes' scam foundation"
mircea_popescu: come a long way, have we, from those early bumbling days of early bumbling intellects.
mircea_popescu: (a punk, it bears reminding, is a young man who takes it up the ass on the quiet.)
mircea_popescu: at least it's conceptually a thing, as opposed to "here's some data, and here's some half-tagging, and here's some half-code with it"
mircea_popescu: can take the ram from the crapple and put it in the pc or camera or w/e
☟︎ trinque: worse still, index.html on your favorite website, and hurr durr XML hypertext.
trinque: there, gopher wins against
http in that it enforces a strict tree
trinque: can't just "this is how I answer" but also, "how do I ask", and "how do I know what I may ask"
trinque: concretely, I would like to connect to your lisp instance from mine and be able to interrogate it for classes of objects it contains, for particular instances (i.e. there is a global notion of identity, global addressing, see threads on GNS), get instances which match some predicate...
☟︎ trinque: "connect" too, is not quite right, as it would be a superior model for me to huck a request to you, which ~maybe~ you get, and which ~maybe~ you drop on the floor at wire speed, and maybe you huck me something back, if you want.
trinque: why huck just to me either, can turn on the luby hose and let multiple folks drink together, when they have the same question
a111: Logged on 2017-07-08 00:17 sina: yes and it's understood, I'm not trying to make "gossipd for alfs dystopian future" just have some fun for me
trinque: "concurrency" in the sense that you handle scads of independent requests for the same item, also complete nonsense.
jhvh1: BingoBoingo: Bitstamp BTCUSD last: 2544.82, vol: 4602.79560558 | BTC-E BTCUSD last: 2512.918, vol: 2713.53486 | Bitfinex BTCUSD last: 2524.5, vol: 8817.7847669 | BTCChina BTCUSD last: 2603.395587, vol: 5605.06310000 | Kraken BTCUSD last: 2537.468, vol: 4239.07829148 | Volume-weighted last average: 2546.02907198
BingoBoingo: " Huel @gethuel Jun 19 A new concept in food - 100% vegan with protein, fatty acids, fiber and 27 essential vitamins and minerals." << Poor naming. "Huel" is mostly known to USians as the name of the fat pickpocket on some "prestige" tv show.
mircea_popescu: in other lulz, i took girl + car out on a survey of local brothels tonite.
☟︎ mircea_popescu: trinque amusingly, candi does sort-of this, as a prototype of it.
mircea_popescu: lol lynched to death. what other kinds are there, lynched to orgasm ?
a111: Logged on 2017-07-09 03:18 mircea_popescu: can take the ram from the crapple and put it in the pc or camera or w/e
BingoBoingo: <mircea_popescu> lol lynched to death. what other kinds are there, lynched to orgasm ? << Well there's the incomplete version which is just a beating
shinohai: Lynched to orgasm sounds like a great porn title though.
☟︎ phf:
http://btcbase.org/log/2017-07-08#1680544 << nope, in fact my "optimized" one i.e. the version where cmucl doesn't complain about unknown types with (speed 3) (safety 0), is about twice as slow as Go version. i'm curious why that would be the case, but haven't had a chance to investigate.
☝︎ a111: Logged on 2017-07-08 02:18 sina: did you make a lispy one go faster?
phf: in vaguely related annoyances screw/half-screw i*POS is the likeliest candidate for overflows, and it does where your fixnum is 32 bit. go version sidesteps that issue by being 64-bit
a111: Logged on 2017-07-09 04:11 trinque: concretely, I would like to connect to your lisp instance from mine and be able to interrogate it for classes of objects it contains, for particular instances (i.e. there is a global notion of identity, global addressing, see threads on GNS), get instances which match some predicate...
mircea_popescu: what "bitcoin nodes" currently do is a very shitty attempt at just this. and they are as autonomous as you are conical, also because they fail to correctly implement this.
phf: which made me think that hashes are typically implemented in terms of machine words, so you don't have this kind of issues. fhf might be inherently "unoptimizable" since units that it operates on must be bignums
☟︎ mircea_popescu: anyway, i'm very much convinced that the various "oh, it's the same thing really" bad choices hash designers and implementers make, very much typified by the above "machine word so as to '''elegantly''' avoid these problems" very much plays into nsa needs. it's again that same old convenience that makes the usg usg.
mircea_popescu: somehow peeps got the idea it's "unfair", like little kids playing a game, to use computers in certain ways that expose them for being mere machines, rather than your girlfriend. such as making them do things like fhf.
phf: presumably as asciilifeform keeps implying you could rewrite mp's fhf in some closed form that does account for these issues, but..
phf: well, one thing that ascii said "don't operate at a byte level", might also be the reason why lisp version is slower. bit vectors are stores as octet arrays, so tweaking single bit pulls a whole octet, does bit twiddling on it, and then puts it back. presumably can be sped up if one were to figure out how to work over an octet at a time
a111: Logged on 2017-07-09 15:48 phf: which made me think that hashes are typically implemented in terms of machine words, so you don't have this kind of issues. fhf might be inherently "unoptimizable" since units that it operates on must be bignums
phf: asciilifeform: literally five lines lower in the logs
phf: well, there's no solution, just same thing you said :)
phf: i haven't had time to do any of this
a111: Logged on 2017-07-09 08:36 mircea_popescu: in other lulz, i took girl + car out on a survey of local brothels tonite.
phf: lotta "romanian" girls now, i.e. darker, gypsy looking ones. is that normal for romanian girls, or that sort of type is purely gypsies?
☟︎ a111: Logged on 2017-07-09 13:41 shinohai: Lynched to orgasm sounds like a great porn title though.
mircea_popescu: phf romanian girls look almost exactly like ukrainian girls. so is it normal for them to look turkish ? mnope.
phf: right then, that's what i thought
mircea_popescu: sure, the majority of ukrs are brunettes. romanians too. but the milky kind.
mircea_popescu: also, there's not so much darker gypsy association. one of mah first youthful loves was this blonde horseriding gypsy girl from the danube delta. hot as hell, and about as dark as hillary clinton.
mircea_popescu: i suspect gypsydom is a lot like brazildom. they dumped all the slaves in there, the results are rather varied, but strong contingent of fair brasileiras, even if lots and lots of mulattos.
mircea_popescu:
http://btcbase.org/log/2017-07-09#1680947 << actually this is interesting, and i mean from a metaperspective. consider, randomly defined cult, they come up with some crankwagen, whatever it may be, say the mpfhf or the correct way rto diet or to cure cancer or w/e. at which point ranking member promises to come out with proof it's shit, and the result is ?
☝︎ mircea_popescu: a) you know you're going to have a schism and b) you know nobody's going to manage to evaluate the claims, present or future, for any kind of coherent truth value.
mircea_popescu: asciilifeform with those calves she's tipping you over.
mircea_popescu: dun talk smack about the pole dancing and stilt wearing crowd until you greco-roman wrestle a coupla. they're NOT THAT EASY
phf: cmucl has REMOTE and WIRE packages which let you do rpc between two instances (naturally it's bit rotted in the trunk, and only working version at this point is on my branch). it's a combination of simple "spin up server/connect to server", serializer/deserializer and an rpc convention.
mircea_popescu: phf how does one node discover the other node's symbolic representation of the universe ?
mircea_popescu: in a very harsh example : i somehow knew how to say "like ukr girls" above. i found this out somehow. how does the lisp instance find out what to say so it's meaningful ?
phf: same as you would do it locally. it's a transparent rpc, not a specified one. so either the operator does manual discovery, or you define a discovery convention. (and then there's middle ground, where you use standard tooling, like code to list packages and symbols to do discovery for you)
mircea_popescu: the point here being that what we need is the convention ; and it'd better be good.
phf: well, right now we don't even have standardized wires (irc and
http come closest in tmsr infra)
mircea_popescu: asciilifeform you are running a computer. (not crap, an actual tmsr is happy with machine). when i start mine up, i want to get some bits from you. maybe it's "what sina said about mp on gossipd". maybe it's "the nth block in the current blockchain". maybe it's whatever it is. how does it do this ?
mircea_popescu: and don't tell me "by-hand optimized datastructs for each app and ad hoc protocols for each connection" if tyou will
mircea_popescu: why not just make a datastruct polymorpher then, and have the program just come up with random shit each run
mircea_popescu: mp's rule of computing design : if you know it's gonna be bad, take the trouble to make it actually horrible.
phf: in the WIRE/REMOTE approach you connect to instance, and do a function call that the author told you about ahead of time. in one case it might (gossipd:all-message-since timestamp) in another (ben-vulpes-block-chainer:get-block n)
mircea_popescu: you want me to sit here and say "there's fifty different protocols in this box" ?
phf: mircea_popescu: i think you could design a pretty straightforward discovery protocol once you decided on the wire. we can see this process with irc bots here
mircea_popescu: phf no, i ~know~. he doesn't expect, i don't expect either, but the difference is im being actively optimistic about it.
mircea_popescu: the problem is again one of synthesis, someone gotta sit down and write that scary text. not the one to which there's nothing to add, but the one from which there's nothing left to remove. "here's how a discovery protocol goes".
phf: (all-exposed-functionality) -> ((gossipd:all-message-since "give me all the gossipd message since a certain timestamp") ((timestamp fixnum "timestamp (in seconds since epoch) since which messages") ...))
☟︎ mircea_popescu: and yes, thius could be done. like, toiday. and yes trinque 's irc bot could be expanded to be a proper connectivity point. and so on.
phf: i thought it would be kind of cool if bots could sling signed lisp strings in privmessages, 0ldsch001 scenetech!
mircea_popescu: if you recall the data exchange protocol was moving away from csv towards lispenstrings anyway.
a111: Logged on 2017-07-08 07:23 sina: mircea_popescu: I am guessing tmsr is not fond of things like JSON or YAML
mircea_popescu: by the time newcomers start asking, the writ is on teh bathroom wall.
phf: note that common lisp's sexps are not an ideal data exchange format
☟︎ phf: (common lisp sexps are backed by a full blown reader, with multiple non-trivial dispatch macros, so for example '(1 #.(+ 1 1) 3) is a valid sexps that's read as '(1 2 3), i.e. (+ 1 1) is evaluated at read time. there's a dispatch for structs, like #S(FOO :A 1 :B 2) results in a structure foo with two slots a and b set to 1 and 2 respectively, but there's no corresponding constructor for classes. there's a reader for arrays, but that one doesn't let you
phf: express the full range of possible array settings, etc.)
phf: there's ISLISP standard, which was supposed to create a subset of multiple different lisps (like common lisp and scheme, but also eulisp, since designed by europeans). i don't think anyone (particularly sexp library authors for other languages) ever tried to conform to it. typical solution is to have a JSON-like subset of sexps, so that you can express (FOO "abc" 2), i.e. symbol, string, number and list and nothing else
mircea_popescu: moreover im affraid sexpr doesn't actuallty separate data from code.
mircea_popescu: phf your excited attempts at playiung with candi's ass, back when she was a virgin, is a plain and very obvious indication of what ~every lisp hacker knows but none dare admit, or think about. it's a fundamentally dirty language in that it doesn't, actually, admit there is even in principle differences between data and code. "oh, it's an array, and here's the associated macro, so this .dat file is actually turing complete". f
mircea_popescu: or the same money it could be excel/emacs (for what is emacs but excel with built in virtual basic ?)
phf: mircea_popescu: that whole data/code equivalence is one of the core (advertised) principles
mircea_popescu: yes, but i don't want my number five to have the capacity to really be four.
mircea_popescu: it's bad enough for my letters abx to have the capacity to really be smileyface or whatever. and i dun like that, either.
phf: lisp has its own kind of trisector: a person whose mind is so blown by the whole code/data thing that he's convinced one can build "real AI" using a handful of self modifying macros
mircea_popescu: in the end, here's mp's heresy : lisp never caught on for none of the usually discussed reasons. lisp never caught on because that core advertised principle is both fucking dumb and fucking useless.
phf: mircea_popescu: well, in that case there's no value to lisp ~at all~, since the bulk of lisp advantage comes from this fact. but pretending that you somehow can get not-this with a turing machine results in what we have now
mircea_popescu: i STILL don't want my number five to have the capacity to really be four.
mircea_popescu: and this is fundamental and won'[t go away nor can it with money and reputations involved.
phf: i mean, any system can be tweaked into "number five to have the capacity to really be four". pretty much a standard exercise "oh tweak this memory location and now all your TRUEs are FALSE111"
mircea_popescu: phf but that is, for the needs of this mental model, a broken machine. not broken data.
mircea_popescu: the very possiblity of even having such a thing as "broken data", by eg naming your kid "-- DROP table students;" is why i agree with the sql sucks sentiment.
mircea_popescu: everything sucks where there can be such a thing as "this is the table of possible inputs and these are the three that set it on fire".
phf: can't you say the same about lisp? you establish your data format (i.e. a READTABLE that you're going to use for reading it), at which point modifications to that readtable makes for broken machine. the standard specifies how, e.g. number reader works, so if your readtable uses standard number reader 5 is 5
mircea_popescu: phf i don't know if you can or if you can't, not sufficiently experienced with the thing.
mircea_popescu: but i can say you not merely should, but absolutely must. the cornerstone of wire protocol is a firm, protocol-derived guarantee of there being no such thing as bad data.
phf: well, i brought up the whole mutability of readtable, because that fact usually trips up novices when dealing with lisp reader. "oh i can just use sexps for everything" and then you do (format ...) on your largish data structure and you get something that's rather non-trivial to parse (an example would be the WOT.sexp)
mircea_popescu: phf and the reason for this is that it tries to take it as code.
mircea_popescu: the best and handiest way to illustrate this folly is, "imagine if your vlc tried to mpfhf every film you played"
phf: so if you want to use common lisp's readtable/printer for data interchange, you'd better ~specify~ what your sexp actually is
phf: mircea_popescu: well, sexp ~is~ code, primarily it's for writing your programs, not ~storing~ data. there's no such thing as sexp internally
mircea_popescu: i need someone hacking on my bitcoin like i need someone feather-tickling my spleen.
mircea_popescu: but nevermind the processor for a second. the discussion here is data interchange.
phf: most lisp rpc is done using serialization/deserialization protocols. personally, that is also my preferred method, something like asn.1 or the format cmucl's wire uses
mircea_popescu: phf this is like saying "this website prevents xss by using base64".
mircea_popescu: if you;'re going to have adnotated data and the adnotations are macros and who the fuck knows what goes on, you've got a miserable protocol.
phf: asn.1 is nothing of the kind, nor is cmucl's wire. it's your typically byte packed binary formats
mircea_popescu: phf serialization can at best be a packaging. the issue is, can i say "here's a function that takes a parameter and outputs "your mom's a python" and the data payload is function(is that you daddy)" in order to convey the string "your mom's a python" ?
phf: not with asn.1, because it's a serialization format, not actual RPC. you can certainly do the call with wire/remote, and no "macros" involved in the process
mircea_popescu: for the curious : yes, i also have this problem in speech, especially when discussing complicated things with complicated women, and the solution THERE is the sort and short rule. which in relevant part says here that "(1 #.(+ 1 1) 3)" could NEVER be interpreted as "(+ 1 1)" because "(+ 1 1)" was a shorter statement of that and it wasn't used.
mircea_popescu: and moreover, and perhaps most importantly, "(+ 1 1)" is NEVER 2. because 2 is 2 and fuck you.
phf: did you intentionally put #. or it just happened to be there? i think that you're looking at that sexp and you recognize its form, but #. is just some fluff?
mircea_popescu: asciilifeform different sort of problem, that's for data entry this is for proper outputs.
phf: well, i put it there ~intentionally~
phf: well, the intent was to make this thing evaluate before the data structure is fully read.
mircea_popescu: ah i thought you were just loading a parser spuriously.
mircea_popescu: "o look, it also has #. in it, gotta load #.parser on top of + parser now".
phf: like i said elsewhere, sexp is not a data exchange format, it's a way of writing common lisp code. various extensions are there to facilitate the process of programming. it's a folly to look at sexp and go "oh, so simple! lets just store our data in it! lets rpc with it too!" because that's not really what it's designed for
phf: at least in lisp world (in symbolics source code anyway) rpc is done using your good old byte packing protocols. <count of integer><sizeof integer><integer><size of integer>....
phf: depends on the protocol
mircea_popescu: what happens if i do data like "1 byte|ff|2bytes|fff" ?
mircea_popescu: what happens if i do data like "1 byte|fff|2bytes|fff" ?
phf: it was an example of what "byte packing protocol" is. designing proper byte packing protocol is left as an exercise to the reader
phf: generally "things that might have different size" is not a trivial thing to pack. (that's why there's complexity in ASN.1, because of variable run length encodings)
mircea_popescu: moreover, and here's where it gets biting : if we go with
http://btcbase.org/log/2017-07-09#1681006 ; which i expect we should, you can end up with implicit types. "you asked me for the 999999th bitcoin block, fucker". what about that ? you will need bounds adnotation yes ?
☝︎ a111: Logged on 2017-07-09 16:19 phf: (all-exposed-functionality) -> ((gossipd:all-message-since "give me all the gossipd message since a certain timestamp") ((timestamp fixnum "timestamp (in seconds since epoch) since which messages") ...))
mircea_popescu: so more like all-exposed-functionality) -> ((gossipd:all-message-since "give me all the gossipd message since a certain timestamp") ((timestamp fixnum positive "timestamp (in seconds since epoch) since which messages") ...))
phf: well, wire/remote is cavalier with byte packing, it knows how to pack various sized types, but variable sized stuff is probably not the most bullet proof (i vaguely suspect it's similar to bitoind, i.e. <typetag><count><item><item>...
phf: i also doubt that's what we'll ultimately go to. wire/remote is tied to cmucl, and nobody's going to untie it
mircea_popescu: but the problem becomes really complex when you consider "bitcoin block index" is as of right now "positive integer under 474974", and won't stay that way for long.
mircea_popescu: and if asked for "bitcoin block", i do expect to be asked a number according to that exact type.
mircea_popescu: or to put this belaboured thing in different words : if your data exchange protocol includes the concept of "error message", you do not in fact have a data exchange protocol.
mircea_popescu: i don't care if it's a "gui" or a telex or whatever it may be.
phf: arbitrarily sized numbers are a variable sized type. gotta figure out ~some~ way of packing them. you're essentially left with type tagged size variation (if a byte is 0, then next struct is 8byte, if a byte is 1, then next struct is 16byte), but at the end you still will end up falling back to "read this number to know how many bytes to read"
☟︎ phf: unless by decree you say that "numbers are packed in 64bit blobs"
mircea_popescu: phf i would be friendly to the notion of a machine word yes.
phf: well, then you're basically in the whole "nobody's ever going to need more than 64bytes of block size!!1"
mircea_popescu: i am ok with chunks of machine word size, ie, quantify data by n bits rather than by 1 bit.
mircea_popescu:
http notably gets this half-right : there's no such thing as an error message the server ever expects to see from a client, such as i dunno, "this page has porn on it, unacceptable, send me another version" or whatever.
a111: Logged on 2017-07-09 16:14 asciilifeform: i dun get the fixation with 'universal db language for all walks of life'
trinque: this goes right along with his "omg social skillz" thing
trinque: blind spot right where one node talks to another
mircea_popescu: yet obviously swiss-army knife wrong bitsize for every problem!
trinque: nobody's asking you to compute in your home in $standardLang; fuck in perl and shit in brainfuck, but speak something civilized in public.
ben_vulpes: whoreticultural trip report: ranier cherries are amazing this year
BingoBoingo: <mircea_popescu> dun talk smack about the pole dancing and stilt wearing crowd until you greco-roman wrestle a coupla. they're NOT THAT EASY << Original Sport!!!