3900+ entries in 0.109s
mircea_popescu: be marked as dirty after each insert if need be etc. possibly
phuctor has grown industrial enough this is actulaly needed.
Framedragger: i'm sorry but you didn't convince me in regards to the 'amount' of data. > 100mil row postgres with > 100 gb of data in a 8GB ram server ran fine. and while
phuctor may be a more demanding beast, shouting '5 mil keys, MILLION!' doesn't convince
a111: Logged on 2016-11-19 16:58 ben_vulpes: Framedragger: if you go down the '
phuctor visualizer/dash' route, you might consider leaning on pg streaming replication
Framedragger: asciilifeform: have you profiled an
http request to a permalinkable
phuctor page? where's the bottleneck? curious if you could insert a thing into flask which crawls through everything and stores locally.
mircea_popescu: "oh it's on
phuctor and why do i need to do anything", which is how we got "those debian experts are flyeyeing the code so why should i have a clue".
Framedragger: (again, many
phuctor pages will simply timeout, iirc. but maybe can adjust; and still worth doing.)
Framedragger: asciilifeform: why can't a separate box be set up to just crawl through all of
phuctor pages, and then determine which of them are 'static' / won't ever change, for starters. and re-query the dynamic ones (at least the /phuctored) every $x amount of time
Framedragger: asciilifeform: (instead of doing a more obscure "query-able
phuctor-and-stuff db" thing i could help with some kind of
phuctor-public-display-vps infrastructure setup. just sayin'. thing's not clear in my head.)
mircea_popescu: fucking worthless, really, there's a total of SIX links to
phuctor that went in chan this month ?
a111: Logged on 2016-11-19 16:52 Framedragger: asciilifeform: at least with public-static and
phuctor boxes being separate, you'd have access to the latter if it were private. (but i guess you could object with "private/undisclosed box on the internet, what is this oxymoron!")
mircea_popescu: they really couldn't, and didn't, give a shit about
phuctor working throughout the years it worked.
mircea_popescu:
phuctor wasn't ever so snappy on display even when it was accessible.
Framedragger: because 1) other sites' experience may be impacted, and 2)
phuctor db would place some load on things. why = because i'd create a few indices, those would hog some memory, and assuming users want to do quite a bit of sorting etc, would take some cpu time as well. just sayin'. nothing scientific.
Framedragger: ben_vulpes: i'll think about this next week. it may be that it's not really needed, yeah - i agree. and integration of things such as ssh server banners with
phuctor keys is something that asciilifeform said he'd be up to do on
phuctor's db itself..
ben_vulpes: 'read only sql users to a replica of the
phuctor db' is, while an interesting project, not wanted by the people doing most of the analysis afaict
trinque: alright I'll take that and you can focus on
phuctor viz
a111: Logged on 2016-11-19 16:52 Framedragger: asciilifeform: at least with public-static and
phuctor boxes being separate, you'd have access to the latter if it were private. (but i guess you could object with "private/undisclosed box on the internet, what is this oxymoron!")
ben_vulpes: Framedragger: if you go down the '
phuctor visualizer/dash' route, you might consider leaning on pg streaming replication
☟︎ Framedragger: asciilifeform: at least with public-static and
phuctor boxes being separate, you'd have access to the latter if it were private. (but i guess you could object with "private/undisclosed box on the internet, what is this oxymoron!")
☟︎☟︎ a111: Logged on 2016-11-19 14:30 mircea_popescu: ok so you aim to take some data from
phuctor, integrate it with some data you already have, and perhaps with other data you can obtain, and present it conveniently ?
Framedragger: i guess another thing is, it would be nice for #trilema folx to be able to test their own
phuctor-related hypotheses without having to download all the data. but, it's not as if it's exabytes of data or anything...
mircea_popescu: ok so you aim to take some data from
phuctor, integrate it with some data you already have, and perhaps with other data you can obtain, and present it conveniently ?
☟︎ Framedragger: mircea_popescu: yes, if by pipe you mean, bulk-import into VPS, not torture
phuctor's own db 24/7.
Framedragger: i'm unclear what to do with 'live data feed', but i don't think it's a problem. it could just sync with
phuctor at timed intervals.
mircea_popescu: so you want to pipe data from
phuctor to a vps, and display it there ?
Framedragger: mircea_popescu: by db i mean a postgresql populated with
phuctor data.
Framedragger: hosts the
phuctor data - p, q, e - and metadata - including country codes from geoip where applicable.
mircea_popescu: not terrible, has all the needed parts except "which shall provide a platform for those interested in analysing RSA keys and the surrounding metadata collected and provided by
Phuctor" could benefit from (such as : by x, and y, and z) as it's currently not even vaguely clear what i could do on that platform.
Framedragger: mircea_popescu: i plan to set up a "
phuctor results investigation dashboard". (iirc trinque or someone planned to do sth of the kind, but since this overlaps with the "check distinct banner amounts etc", i may as well do it properly). nothing fancy so as to keep it actually deliverable, but i have a plan.
Framedragger: asciilifeform linked to that phoronix thread, and there were in point of fact good replies on that thread, who acknowledged significance of
phuctor, but of course asciilifeform chose to focus on the stupid kiddos there.)
a111: Logged on 2016-11-18 17:49 asciilifeform: ( in related 'news',
phuctor 'fan' :
https://archive.is/dQFvt >> 'Oh, the
Phuctor guy. Ignore him, he has a history of spreading FUD about PGP being horribly broken and accusing anyone who corrects him of being part of some elaborate cover-up (and in at least one case, threatening to contact the university of the person who did so. ' )
BingoBoingo: <asciilifeform> !!key BingoBoingo << /me reflexively checked my key's health on
Phuctor at this invocation