log☇︎
81900+ entries in 0.649s
shinohai: When your shitcoin needs a hardfork to fix a hardfork: https://github.com/ethereum/go-ethereum/releases/tag/v1.5.2
mircea_popescu: https://www.youtube.com/user/HarrisHarrington << this just takes the cake of poptardation. so here's a guy too young to have an undergraduate degree doing self-help posturing over youtube.
mircea_popescu: i actually planted a bunch in romania, for the flowers.
shinohai: Why having a boring, colorless psychosis
mircea_popescu: and yes, putting an -A "TMSR agent ; contact X for discussion" on all curls can't hurt anything.
mircea_popescu: this isn't a replacement for archive.is, but more of a defense-in-depth measure.
mircea_popescu: then it could bundle all of these together in a base64'd blob each week and deedbot them.
mircea_popescu: http://btcbase.org/log/2016-11-19#1571048 << it's not a matter of "cpu at 100%" nor is it a matter of free disk space. if on that os ssh hangs off eg dbus, and if dbus gets locked out by kernel because "dirty page" or "waiting on journal update" or whatever similar idiocy, your process is stalled. and these are just random examples, so much can go wrong in a modern box it's not even worth my time drawing the broad strokes. ☝︎
deedbot: http://www.contravex.com/2016/11/19/why-you-cant-afford-globalisation-anymore/ << » Contravex: A blog by Pete Dushenski - Why you can’t afford globalisation anymore.
mircea_popescu: the entire stack is built pretty much to create this - first, we have a phuctor, and 2mn keys looks like the whole world, and any finds look improbable as shit. then some finds are found, and more keys are fed, so now 50mn looks like the whole world and a few finds a day are expected. BUT THEN a way is found to crack thousands of keys in a week, and well, the echafaudage which held up the original is struggling.
Framedragger: anyway, i imagine a bunch of d(a)emons fighting for i/o... and yeah no one is saying that there's a straightforward solution...
mircea_popescu: i dunno that alf is a db engineer by trade, so it's entirely possible specific measures could help, especially if they're of the magic number ilk of "set X to Y in config file, we didn't docum,ent this anywhere but its tru!"
Framedragger: this reminds me. you know, sometimes postgres prefers to do sequential read instead of using a reasonable index, because, as it estimates, using index would involve too much seeking etc. *but* with SSD, random access is much much faster. (and btw postgres does not automatically know about seek times in SSDs..)
mircea_popescu: a similar situation to how compression works great on literature and poorly on (proper) random strings.
mircea_popescu: in general db optimization / low consumption success stories rest on a very opposite situation - the lines are only ever interesting in one perspective and light only lights one facet at a time sorta things.
mircea_popescu: lol. yeah, might work here, it's not clear. the problem, you have to understand, is how integrated the data is. sure a db with however many lines did fine in whatever box. the problem is that everything phuctor has, phuctor uses, and so it's very close to the unmitigated nightmare which is "random access"t
a111: Logged on 2016-11-19 18:55 asciilifeform: we are at the farthest possibly limit of what can be done on one box, at anything like reasonable budget (whether paying the cost of a small european flat for a server for public service is 'reasonable' is separate question)
mircea_popescu: http://btcbase.org/log/2016-11-19#1570951 << for the record, separate dbs for selects and inserts is the way to go. from experience it can rescue a large project / save 9x% off the hardware costs.the way you do it is that you have a master db copy which is the only one that takes the inserts, and slave dbs which are the only ones that take the selects. replication can be at dedicated sql cluster level or above, slave dbs can ☝︎
mircea_popescu: and from my pov, V is truly great in the sense that it allows a very simple test for when april next rolls around. phf's viewer trivially allows to see what signatures are actually active in the deployed branches of republican code. join that set with the set of box owners and you have a very good first approximation of the l1.
a111: Logged on 2016-11-17 14:43 Framedragger: (and on a side note, "hang out on #trilema after splitting with gf" has been one of the more constructive choices i've made in my life)
BingoBoingo: There will always be enough stuff to be done in the future. This is not a reason to not make things and move into the done category.
mircea_popescu: Framedragger technically yes you can - make a V root for it, like properly sane people, then he'll just import that / patch it himself when he's ready.
BingoBoingo: WoT browser is like log viewer, who the fuck cares how many there are so long as there is a non-zero number!
BingoBoingo: Framedragger: Wot browser must also maintain existing url structure to retrieve a nick's profile
mircea_popescu: http://btcbase.org/log/2016-11-19#1570790 << amusingly enough, this is actually true. "i would like to lecture these monkeys in modern psychology" "ok ?" "can you design a shitproof semipermeable membrane that still allows my precious words to reach them ?" "uh. not really. whatever the fuck it is, sooner or later the shit will clog it" ☝︎
Framedragger: sure. "javascript in the backend", etc etc etc.; i mean, it's still a horrible language. but yeah.
a111: Logged on 2016-11-19 17:03 Framedragger: regarding visualization, a more condensed question: if a javascript-using thing were delivered, would this be hated upon (and berated by asciilifeform) and accepted if otherwise good and properly maintained, or hated upon and dismissed (and berated by asciilifeform)? :)
mircea_popescu: might discover it fails via resource exhaustion in the very large dataset that is tmsr wot, but who knows. worth a shot.
a111: Logged on 2016-11-19 16:57 Framedragger: re. visualization, i like stuff like this (mouse over on labels around the circle), but it's a hella lot of JS, and i share the hate towards the latter: http://bl.ocks.org/mbostock/7607999 - what's nice about btcalpha visualization is that it uses by-now standard html5 canvas directives (<path>) with no need for JS.
mircea_popescu: i'm not for a second saying HE could have done much better / differently.
mircea_popescu: point in case here. it is beyond even the shadow of the most absurd doubt that alf's code locked the box. but he's willing to spend time fighting the obvious, rather than anything else, because hey, this is a bitter pill to swallow.
mircea_popescu: asciilifeform you gotta learn to judiciously allocate your time ; Framedragger the problem isn't so much locking up the box - it's that the box will fuck up, this being "foss" bullshit, and then the owner will be like "i dun wanna pay for a box i can't use, notwithstanding this is what i claimed i want".
a111: Logged on 2016-11-19 16:36 mod6: Anyway, now that it seems that private a more rich ticketing system is wanted, these things can be considered for sure. Salud!
Framedragger: (this is a case where i am more cynical than asciilifeform: a DC you don't know, what illusions does one have; okay, one could install a sensor which signals if box was 'tampered with'; they could still take box offline, clone memory, set up a replica. etc etc etc.)
asciilifeform: i had occasion to install such a machine a few yrs ago, somewhere, it came with 'kvm chipset', but this feature was not advertised. the advertised feature is just that, kvm, where you get a tty prompt.)
mircea_popescu: afaik it can't actually dump ram or do useful debugging. but it can reboot the system, which is you know, a crossed wire.
asciilifeform: this was a fiddybuck/mo dc.
asciilifeform: it cost 0. (but there was a queue, sometimes a whole hour long!)
asciilifeform: well the one i subscribed to in the past, had a physical cart, with wheels
asciilifeform: i had nfi that we were even buying a 'fritz chipped' box.
asciilifeform: a 'welded shut box' + dc supplies GB pipe + mains current dc would be mega-product.
a111: Logged on 2016-11-19 16:16 asciilifeform: i wrote, in the qntra piece, 'examine debianized boxes for nsaware'. now 'owner' will have a chance to clean up before any mass 'examination' takes place.
Framedragger: (i remember having use from a super stupid `free > memory.log` cron job every minute or so)
asciilifeform: btw werker uses db for approx. 40 seconds of a run, to dump keys, which subsequently all load into memory.
asciilifeform: i had a RUNNING SESSION 24/7 on own display, and it ground to a half
asciilifeform: a RUNNING SESSION no less
mircea_popescu: you probably ran into a disk or io locks issue.
mircea_popescu: though i must confess syn flood had a better ring to it altogether.
mircea_popescu: asciilifeform in point of fact i have asked the public force to locate car drunk ho abandoned "dunno where". they did. i didn't throw a shitfit about "o noes surveillance state". this because a) i asked them to do it and b) obviously they just put the number in the stolen cars interface and then a patrol saw it.
mircea_popescu: it's a fucking admin interface bridged into the fucking bus, what the everloving shit would it care about your derpy os's notions of "users". as if those fucking EVEN WORK irl. ☟︎
mircea_popescu: ip-kvm will show you a sort of top.
mircea_popescu: anyway, make a call, do i have it rebooted or let it be for a while and see if it digs itself back out ?
mircea_popescu: it's a fucking computer.
Framedragger: asciilifeform: of course. and also don't treat this as high priority, i prolly wont look at it before wednesday anyway. but would be interesting to take a look - and i'd take a look
asciilifeform: Framedragger: you are welcome to a copy of db as soon as i get my hands on it again
Framedragger: i'm thinking whether it'd be worth it to just have a static replica of db-as-it-currently-is, for now. as in for "i want to touch data, there's an outdated html file on loper-os i guess?" cases.
Framedragger: i'm sorry but you didn't convince me in regards to the 'amount' of data. > 100mil row postgres with > 100 gb of data in a 8GB ram server ran fine. and while phuctor may be a more demanding beast, shouting '5 mil keys, MILLION!' doesn't convince
asciilifeform: it is a db of ~5 mil KEYS, each of which is from 1 to 20kB, and links, etc.
asciilifeform: it is not a wordpress blog with ~static 5 mil words.
Framedragger: i have no doubt that a sillicon valley version would be a datacentre of mongodb nodes which constantly fail and corrupt data, and cost millions. :)
asciilifeform: because, for instance, the key-eater is a separate process. and has nfi that someone else wants the db.
asciilifeform: if you want anything from the db, you wait. for , possibly, HALF A MINUTE
asciilifeform: we are at the farthest possibly limit of what can be done on one box, at anything like reasonable budget (whether paying the cost of a small european flat for a server for public service is 'reasonable' is separate question) ☟︎
asciilifeform: Framedragger: you do not seem to understand, so i will give illustration: http://www.swapmeetdave.com/Humor/Workshop/Lumber-Car-A.jpg
Framedragger: there's a slave/clone db, it gets updates efficiently from master.
Framedragger: asciilifeform: http://btcbase.org/log/2016-11-19#1570745 may be worth a try, actually. ☝︎
Framedragger: asciilifeform: have you profiled an http request to a permalinkable phuctor page? where's the bottleneck? curious if you could insert a thing into flask which crawls through everything and stores locally.
mircea_popescu: "oh it's on phuctor and why do i need to do anything", which is how we got "those debian experts are flyeyeing the code so why should i have a clue".
mircea_popescu: "oh maybe i have a page from 2005 somewhere"
mircea_popescu: Framedragger your scriba could crawl all links in chan and archive them. phf said a while back he almost has this but i've yet to see it
mircea_popescu: if you have a fp or w.e and wanna see go to phuctor.
Framedragger: asciilifeform: why can't a separate box be set up to just crawl through all of phuctor pages, and then determine which of them are 'static' / won't ever change, for starters. and re-query the dynamic ones (at least the /phuctored) every $x amount of time
mircea_popescu: you can't search worth a shit right now because overflow.
asciilifeform: mircea_popescu: that would make problem even worse, could no longer search logs for a gpg fp
Framedragger: that would indeed be nontrivial and would take quite a bit of your time
mircea_popescu: a damn that's also there.
Framedragger: asciilifeform: (instead of doing a more obscure "query-able phuctor-and-stuff db" thing i could help with some kind of phuctor-public-display-vps infrastructure setup. just sayin'. thing's not clear in my head.)
mircea_popescu: asciilifeform we have a bot run by peterl which supposedly snapshots EVERY LINK IN CHAN
asciilifeform: i dunno that anyone other than google and yandex ever made a ~full phuctor~ snapshot.
mircea_popescu: fucking worthless, really, there's a total of SIX links to phuctor that went in chan this month ?
asciilifeform: mircea_popescu: there is an independent snapshot on my blog, also. but also a month old.
Framedragger: a couple of vps providers i used have nice APIs
asciilifeform: i really ought to have made entire thing a c proggy that shits out static html once in a while.
Framedragger: have a way (rss or better version of rss, or whatever) to sync it every $n hours.
asciilifeform: but yes, 'phuctored.html' is a few MB currently.
Framedragger: i'm not saying that going for cheapest ad-hoc option is accetabru. just, a display box showing static content needs much less.
asciilifeform: that's a few dozen GB.
Framedragger: asciilifeform: wouldn't be sure re. costs. vps can be $5/$10 a month, and stuff i used for ssh key crawling (scaleway) can bill hourly
mircea_popescu: in a sense it is "social responsibility", ie, "the data was provided, what did republic do with it".
mircea_popescu: they really couldn't, and didn't, give a shit about phuctor working throughout the years it worked.
Framedragger: to the point of having a ready-made system image (no, does not imply need to use docker), deployable at vps center in a matter of minutes.
mircea_popescu: yes, but there are two concerns that are separable : a) flood stops processing and b) flood stops display.
mircea_popescu: yes, but much easier to a) have a lot b) move around
Framedragger: because 1) other sites' experience may be impacted, and 2) phuctor db would place some load on things. why = because i'd create a few indices, those would hog some memory, and assuming users want to do quite a bit of sorting etc, would take some cpu time as well. just sayin'. nothing scientific.
mircea_popescu: asciilifeform the wisdom of having the data in a cheap vps is becomingf ever more apparent.
a111: Logged on 2016-11-19 17:18 ben_vulpes: Framedragger: i am also curious to know what kind of requirements you have for a vps that your current loggotron doesn't serve.
Framedragger: http://btcbase.org/log/2016-11-19#1570793 << current loggotron also runs on vps, and in itself it requires very few resources. no db use, even. at this point there's a bunch of stuff and other people's sites running on that vps, i don't feel comfortable adding additional load. ☝︎
asciilifeform: the box per se is ultra-responsive, even when 'werker' is firing (i leave a core open)
trinque: hacking on it now actually, maybe have something up in a week
mod6: This would be a really great project to work on, we're long overdue to get this working with deedbot, etc.