Framedragger: asciilifeform: but cross-checking makes sense, say if you do ssh-keyscan for a small randomized sample, nothing bad can come out of it. let me know if you want scriptz

Framedragger: asciilifeform: but have you considered that there are fewer implementations for ssh (and the better part of the ssh servers in the wild run openssh), and more implementations for all kinds of broken pgp? so it may simply be less likely to spot a badly generated ssh key. ☟︎

Framedragger: ..soo, this generates ssh keys based on the broken debian prng: https://github.com/g0tmi1k/debian-ssh - github keyset has been matched against these - wonder about them host keys....

Framedragger: would still be curious to write/find a tool which detects low hanging fruit for EC ssh host keys / keys with shitty parameters

Framedragger: mircea_popescu: if you mean "nobody smart", then that's one thing, but if you mean in terms of actual numbers, then i'd probably have to disagree. shitloads of ecdsa ssh host public keys anyway (stats and breakdown to be published at some point in teh future)

Framedragger: jurov: have you seen this - analysis of github users ssh keys: https://blog.benjojo.co.uk/post/auditing-github-users-keys

Framedragger: asciilifeform: tar archive of all archives (3 GB): http://95.85.10.71:8000/all/openpgp/ssh_openpgp_all_2016-06-20.tar ; checksums http://95.85.10.71:8000/all/openpgp/SHA256SUMS.txt ☟︎

Framedragger: asciilifeform: openpgp'd ssh keys (thanks jurov for PGPy hack): http://95.85.10.71:8000/all/openpgp/ - 13 bz2 archives, each of which contains directory with 800k-900k files (one file per key, assumption was that this'd be easiest for bulk import)

Framedragger: asciilifeform: yeah so i'll prolly end up dumping 10M ssh keys on phuctor's ass, via its web interface. i wonder if phuctor can handle parallel requests, to a point :P i won't be an asshole about it. ☟︎

Framedragger: look i just came back from an awesome mountain hike, can't even move my legs, but have just enough energy to re-use someone else's precious hacky code to test the ssh-rsa db entries

Framedragger: asciilifeform: do you think it's a sensible idea to try and convert ssh public keys into rfc4880, and then submit them to phuctor (possibly in bulk)? or is that something i should leave to you?

Framedragger: asciilifeform: gotcha. i have thing which converts ssh pubkey format to e,N,IP. i'll probably have a thing which generates rfc4880 (inserting ip address as comment field, say) from e,N,IP. thanks!

Framedragger: asciilifeform: ah wait lol: i'd be parsing ssh rsa keys, not pgp keys - different format - though also base64 etc. i'll check!

Framedragger: i guess right now i'm more curious to see general statistics / trends, e.g. distribution of ssh server versions per given geo region / AS etc., not that it may be too useful, but just genuinely curious

Framedragger: the latter would be quite useful - i may spin up some simple analysis thingie which shows info for those ssh keys, and it'd be nice to be able to link to corresponding phuctor entries

Framedragger: (i used one vps at a time (switched due to abuse complaints and so that i'd be sure there's no filtering / results are not biased) for the "which IPs out of all IPv4 are alive behind :22" scan, and i then used 13 cheap VPSes for the actual ssh key extraction out of the 20.8M alive candidates. if curious re cost: will get the bill for the 13 VPSes later, but it was basically 11 hours of scanning, 2.99eur / mo. per VPS, hourly billing, so

Framedragger: this is what i've been able to extract (timeout for ssh server to provide ssh keys: 7.0 seconds), but now that i better understand the nuances of this particular undertaking, re-scanning / re-extraction would be much easier. in particular, doable in 36 hours for under 2 euros of cost (and doable in shorter amount of time for more euros and more concentrated work). which means that costs are absolutely negligible, unless you want to keep

Framedragger: asciilifeform: mircea_popescu: i'd like to interrupt this joyful occasion of copulation of you two by noting that ssh-rsa keys of all of ipv4 are now ready for phuctoring and further analysis; they are in the format of e,N,ipaddress. there's 10.6M of them, out of 20.8M something-listening-on-port-22 hosts. rejoice!

Framedragger: * 20.8M of 22-port-open ssh servers after deduplication ☟︎

Framedragger: ..sooo, ~21.1M ssh hosts with port 22 open in ipv4 space. i expected more, but a time will come for a re-scan, and i finally got around to 'streamlining' the process, and vc's idontcareaboutscans policy helped here, so re-scan will be a piece of cake.

Framedragger: someone also tried spidering ssh keys from .onion hosts, and then matching them with ssh keys of ipv4 space; i want to re-try that, too

Framedragger: (gun resume and finish ipv4 open ssh port scans tonight with vc's node (vc: it's randomized ip range scan and only 30 kpackets/s, before you ask), and then deploy ~10 vps nodes for ssh key extraction, feeding port-22-open-list from the former into the latter.)

Framedragger: hehe yeah, ssh keys, welcome to idiotland i guess

Framedragger: http://btcbase.org/log/2016-06-02#1475260 << (i'm busy with afk stuff and most probably won't be able to resume the ipv4 ssh keyscan thing until some time next week. but i'll ping you once this is done) ☝︎

Framedragger: vc: re. https://box.cock.li/ - do you tolerate responsible and low-bandwidth netscans (TCP SYN and/or ssh-keyscan (which doesn't attempt any ssh auth)) perchance?

Framedragger: now i'll try scaleway / online.net and i don't use them for anything else so i'm fine with being banned. they said OK to my "responsible academic scanning" query. doesn't mean they'll follow through. but i'll just keep scanning / doing ssh extraction there, and rsyncing often, so that if/when i get eventually banned this wouldn't be a problem.

Framedragger: (again, general ipv4 mapping is no prob, it's the more particular ssh key extraction that is slow and very hax0ry apparently)

Framedragger: canyoubelieveit, i used *ssh*!!11 it's..super effective

Framedragger: oh mein gott, the language of these "abuse complaints"... your Server/Customer with the IP: [...] has attacked one of our servers/partners. The attackers used the method/service: *ssh*

Framedragger: they'd look like "31.204.153.252:22 SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u4"

Framedragger: i'm afraid these keys won't contain any additional comments or email addresses. while the latter are stored on the ssh host, they are not transmitted during key exchange, viz.: http://wotpaste.cascadianhacker.com/pastes/e9318ddf-dfb5-494b-8c75-ecda5ebe30b1/

Framedragger: asciilifeform et al.: a couple of representative ssh host key samples (e,N,ip_address) - two /8's - pipeline was masscan (sort of nmap+zmap with custom tcp stack) -> ssh-keyscan (manually parallelized) -> custom conversion into CSV ☟︎

Framedragger: http://btcbase.org/log/2016-05-22#1470431 << hmh actually not sure now. it doesn't seem that the email addy in the ssh host's pubkey is sent to client. yet you're right in that the email addy is included (in e.g. /etc/ssh/ssh_host_rsa_key.pub). will check (ssh-keyscan explicitly doesn't give/relay it though) ☝︎

Framedragger: http://btcbase.org/log/2016-05-22#1470431 << not server keys (but ssh client keys do, yes), it would appear. my stolen and modified ssh key unpacker doesn't see any more info - only PEM'd e and N (as well as ip and type of key, e.g. "ssh-rsa") ☝︎

Framedragger: i guess i'll give the results in that format for those 95k ssh hosts (actual number of keys will be lower, i can see that some of those hosts are providing clients a nil set of encryption mechanisms, etc etc

Framedragger: mircea_popescu: asciilifeform: in regards to ssh key spidering, it's best then to produce output in the form of e,N,comment - is that right? (where comment in this case would be the ip addr)

Framedragger: i'm now running the reliable but slower ssh-keyscan on the 95k ssh-running IPs just to get a decent sample. will later revisit wtf zmap is doing

Framedragger: ..hrm. zmap finished scanning sixteen /8's (so a sixteenth of ipv4 space, minus reserved blocks), but in those blocks there were two known servers running openssh which was picked up by ssh-keyscan. they weren't picked up by zmap. if the thing is unreliable then it's worthless (it still found > 95k of ssh servers though, but...) ☟︎

Framedragger: mircea_popescu: to get list of machines listening on port 22 using zmap on amazon aws micro instance with 100 mbps pipe, it takes approx 9-10 minutes per /16. note, only what's listening on port 22, but ssh-keyscan should then plough through pretty quickly. will later run more tests, i'll leave a single instance to scan some /16s to then feed into ssh-keyscan

Framedragger: i'm considering running zmap on high-throughput server to quickly get all internet-connected machines in ipv4 space, and then feeding that into ssh-keyscan. may be more efficient. also need to get some disposable ip addresses or something, cause according to internet my coupla server IPs will soon be added to some shitlist

Framedragger: mircea_popescu: with timeout per host set to 5sec (default) and unaltered parallel scanning setting (ssh-keyscan does parallel stuff pretty well but it may not be best for scanning huge numbers of hosts; case in point: default version for ubuntu 14.04 terminates if a single remote host closes conn prematurely - needed to patch this..), it took ~65 minutes (i've started logging timestamps afterwards but this is prob quite accurate). this

Framedragger: mircea_popescu: (got 2768 ssh-rsa keys on a test run on a populated /16... this will be fun.)

Framedragger: so, like, presumably, one would want keys of ssh servers that are behind *all* routable public ipv4 addresses?

Framedragger: ssh-keyscan nicely gives all keys presented by server, usually includes ecdsa

Framedragger: anyway. ssh server key spider. *considers setup*

Framedragger: i suppose i should search logs for previous discussions? unless anyone wants to give a more particular summary? spider as many keys from as many hosts running ssh as possible?

Framedragger: "also, anyone want to run a ssh server key spider ?" << heh, k, i'm up for this, might as well be useful while i brood ☟︎