log☇︎
14300+ entries in 0.009s
asciilifeform: diana_coman was entirely right , there is no moar time for waitings, BingoBoingo .
BingoBoingo: asciilifeform: Recently it filled up big. Someone put in a huged controlled access fence taking up 1 new aisle of Tower 4 and their racks are full. But yes I suspect they may be bankrupt/insolvent.
asciilifeform: BingoBoingo: are you ready to execute 1->2->3 ?
asciilifeform: that dc was less % full than our rack.
asciilifeform: it won't surprise me if turns out that latech is actually bankrupt.
BingoBoingo: And yes, It appears it is time to start shopping the case for litigation
BingoBoingo: I suspect Latecho's situation degraded substantially and recently within the past month. It appears the folks who could run things quit or were chased out. Despite a recently installed giant multirack cage I assume is some streaming cache in the datacenter, Latecho does not appear healthy.
ericbot: Logged on 2019-10-06 20:27:07 mp_en_viaje: if indeed they just hitched you to an existing pipe and pretended, it's time to serve them papers registered with the whatever local intendencia handles business fraud.
asciilifeform: BingoBoingo: for the handtruck there will be time later, when folx actually start to demand evacuation. presently you ought to be visiting 1) chairman's office, w/ contract in hand, and ready to explain exactly what will happen if service not restored (and compensation paid ). 2nd stop -- http://logs.ericbenevides.com/log/trilema/2019-10-06#1941177 , 3rd -- law office .
BingoBoingo: asciilifeform: Maximillian is the sales manager.
asciilifeform: fucking orcs. they went to sabotage the heat exchanger so could then blame delay on 'cooling problem', didnt they.
BingoBoingo: There was a poor miserable looking girl in the glass box as a the receptionist.
BingoBoingo: asciilifeform: Just as concerning as the Airconditioner fuckup is who was addressing it. When you were here, you met Joaquim who is(was?) the guy in the glass box. He did glass box stuff. I have no earthly idea why Maxi the sales manager and Oscar the Telecom manager were THE PEOPLE to be fucking with the air conditioners.
ericbot: Logged on 2019-10-07 15:22:09 asciilifeform: BingoBoingo: now, find that door. and if phone rings while yer fighting the guards, you can pick up.
asciilifeform: the other q , BingoBoingo , is : who the hell is maximiliano ? is he the chairman of latech ? ( cuz if not, what happened to this ?? )
BingoBoingo: dump from Oscar. (now to answer asciilifeform's first question in as much detail and context as I can)
BingoBoingo: After leaving with Oscar's assurance informaiton will be sent and with the final request that I do receive everything, I swung by Tienda Inglesa and bought a hand truck because I'll probably need one. I arrived back at the desk to see one email from Maxi asking if I could send him my phone number again because my Movistar Uruguay phone number doesn't appear to be a Uruguay phone number. I have yet to receive a datadump or link to a data
BingoBoingo: At one point they did ask IF I HAD ANY INFORMATION! I responded no. I received no ransom notes or anything that suggests an attack. (there's more)
BingoBoingo: customer service and that their ASN's BGP declares Pasawar Intl peers with Antel. When pressed on what would happen if this was Antel I received a lot of "I dunno" and head hanging. (there's more)
BingoBoingo: They did mention they and dedicado were looking into "Arbor", this suggest that they googled DDoS mitigation. Maxi offered 1 MONTH as the timeframe to implement a reasonably stable solution other than blackholing. I responded that if that is the case, I am going to need ALL the information and I am going to need it now. There were deflections to Century Link and the wait there, and I responded with CenturyLink's notoriously poor
asciilifeform: BingoBoingo: plox to continue ?
BingoBoingo: asciilifeform: I am submitting this testimony as I recollect it while it is fresh.
BingoBoingo: only other non-Antel option they could have pursued in country is Telefonica/Movistar. Maxi boy appeared legitimately shocked when I pointed out Telefonica was shopping their Uruguay operation Pretrobras style and looking for an exit. Maximiliano said he was unfamiliar with that news. He his face contorted like an injured child's when I told him the news was in the local papers recently (there's more)
BingoBoingo: They were pressed several times on the BGP question and Antel's listing as their major peer. They denied Antel and instead repeatedly claimed they hired our pipe out to "Dedicado" (Local firm mostly sells radio point to point IP connections in areas without Antel) and that Dedicado's upsteam CENTURYLINK!!! was not being very helpful in responding to Dedicado or their requests. They declared "Century Link is the Biggest Global ISP, the
asciilifeform: 'maneuver with air conditioners?' what is this, chernobyl reactor 4 ?!
asciilifeform: BingoBoingo: let's have the more
BingoBoingo: that there will be no nines of uptime this month. (there's more)
BingoBoingo: I paid for my own coffe drink called a 'lagrima' in cash and waited as they fiddled with the Pasawar Intl billing information to order their drinks. Oscar did seem inclined to forward information, but Maximiliano, like his predecessor Rodrigo interjected a couple times that he wants the information passed when "the situation is resolved". Each of these interjections was answered with "I am running a business" and repetition of the point
BingoBoingo: there will be zero nines of uptime for October.' That point kept being repeated... (coffee shop continuation next message)
BingoBoingo: they washed their hands we proceeded the quieter coffee shop one floor up for the breaking of the balls. On the way up I mentioned how a client's other datacenter in "third world moldavia" managed to sent .pcaps and detailed information. I was told the files they have are not .pcaps but "huge", I told them I want them anyways, that more than anything I need all the information because 'this first week of October has already settled that
BingoBoingo: asciilifeform: Back. I went to the freezone office, persisted at Latecho's intercom lock long enough to discover Maximiliano and Oscar together in the Tower 4 datacenter which houses our rack. Unlike the couple times last night where I visited a cool datacenter, it was hot and loud with the noise of angry fans. They declared that this isn't normal and that they had tried a routine "maneuver" with the air conditioners which failed. After
lobbes back to saeculum for a spell
lobbes: Though, now that I think of it, I may have just inadvertently scratched the inside of my eyelid the other day while rubbing it; irritation seemed to begin after that event. I'll let it alone and give it until tomorrow in any case before seeing doctor (perhaps save some time/money this way)
ericbot: Logged on 2019-10-07 16:15:02 diana_coman: lobbes: any chance some spider/insect simply bit you? it can get very spectacular but nothing to do with the eye itself.
lobbes: http://logs.ericbenevides.com/log/trilema/2019-10-07#1941363 << could be insect (def. buggy here much like asciilifeform-istan). Though there are no visible 'bumps' that would indicate a bite. I've been prone to "styes" before as well, but in this case it is a more uniform swelling of the entire lid vs a stye which tends to be focused on one area.
asciilifeform: a press that stops on 'MANIFEST' is dead giveaway tho.
asciilifeform: ( atm the barf you get if you , like asciilifeform , hamhandedly forgot to include a patch in 'patches' , is cryptic )
asciilifeform: ( but when will get to do this, do not know atm )
asciilifeform: hrm i prolly oughta patch vtron so that actually gives detailed barf when encounters an apparently 'martian' antecedent, and for ~all~ such encounters in full list.
asciilifeform: so far 180 seems like entirely reasonable value to asciilifeform .
asciilifeform: lobbes: it's a very simple mechanism. could probably be made moar sensitive ( turn knob down from 180 ) but it'd create a risk of spurious disconnects.
lobbes: asciilifeform: aha, I did notice it detected the disconnects like a champ
asciilifeform: i had a chance to deploy right before the pipe's last epileptic fit .
asciilifeform: lobbes: the autodisconnector in fact was thoroughly tested, while dulap was 'half alive' on sat.
asciilifeform: tips for fyooyoor vtronicists : if press barfs that cannot find antecedent, use the 'o' (origin) knob on the unfound hash. will say if is even in tree anywhere
lobbes: in that case I will deploy detect_disconnect tonight methinks. The mp-wp bot will probably end up forking from this point in the logotron tree as well
asciilifeform was quite ready to believe that he fucked even this, after last coupla days
asciilifeform: (however does not include removal of the doublequote thing, will be in next.)
asciilifeform: ok i was wrong, in fact was missing 'sept_errata' in the test press.
asciilifeform: or hm i ought to check again for 3rd time
asciilifeform: will make the removal of the broken doublequote thing a separate patch tho.
asciilifeform: it's braindamaged tho and i'ma have to do it again, made coarse error of pilotage when vdiffing
lobbes: asciilifeform: roger that. I've yet to deploy latest patch anyways, so this'll work out
diana_coman: asciilifeform: I have pressed only to active_disconnect, not further
asciilifeform: meanwhile, diana_coman , lobbes , i'ma have to regrind the latest patch, it is defective (and not simply on acct of the double quotes thing) , apparently in 'a' of diff had wrong copy of MANIFEST .
asciilifeform: diana_coman: happens to asciilifeform regularly, in this swampy penal colony.
diana_coman: lobbes: any chance some spider/insect simply bit you? it can get very spectacular but nothing to do with the eye itself.
lobbes: in other news, yesterday afternoon my right eyelid inexplicably became very swollen. Woke up today and it is even worse. Slightly concerning; may need to seek doctor. Made log reading a challenge this morning at any rate
lobbes: not sure if you want to wait for whole thing to be done to sit it down, or will want to deploy piecemeal as well
lobbes: The second chunk, which includes the searching and pingbacks portion, I am not sure yet.
ericbot: Logged on 2019-10-07 06:37:31 mp_en_viaje: lobbes, when is it you'll need a server to sit down orchestra on ?
lobbes: http://logs.ericbenevides.com/log/trilema/2019-10-07#1941226 << The 'first chunk' which spits lines into the mpwp db should be complete by the end of this month (as in patched and ready to go). That's a conservative estimate as well; hopefully sooner.
asciilifeform waits for word from the front.
asciilifeform going towards tea, brb.
BingoBoingo: asciilifeform: Going towards door
asciilifeform: BingoBoingo: now, find that door. and if phone rings while yer fighting the guards, you can pick up.
asciilifeform: megabux -- in, bullshit -- out. this is actionable even in zimbabwe.
asciilifeform: BingoBoingo: seems to me that we're being elementarily defrauded, and that the very next thing to do will be to get whatever passes for attourney in that shithole.
asciilifeform: what kinda africa is this?!
asciilifeform: BingoBoingo: so, you show up there with a copy of the fucking signed contract, and 0 doors will open?!
BingoBoingo: asciilifeform: The freezone office is door, behind a couple doors I don't have keys too
BingoBoingo: But we do have Oscars: "We are working on that with the ISP. As soon as we get the information we will send you." showing they in all likelyhood don't know what pcap is
asciilifeform: or wat, we're paying $maxint, and this doesn't come with a fucking door to bang on?!
asciilifeform: find where. and go there. holyfuq.
asciilifeform: BingoBoingo: dun tell me you dun even know where?!
BingoBoingo: asciilifeform: They are either in the freezone nearby or aguada park on the other side of time
diana_coman: so they should get out of that "meeting" and talk already; you've been waiting *since Friday* for them to "talk to me", don't take anymore waiting because you really don't have much time left for waiting
asciilifeform: and how about the actually in-command bipeds ? they are in brazil or where ? why can't get to'em in person ?
BingoBoingo: diana_coman: Someone is in a meeting/wants to talk to me
asciilifeform: BingoBoingo: how come it has to be via phone ? i was in that thing, there was a glass box with bipeds in it
diana_coman: BingoBoingo: did they not even answer or why call back
BingoBoingo: asciilifeform: Phone call this morning, waiting on a call back
diana_coman: because so far from the info coming out they have been doing just one thing, namely nullrouting your IPs and precisely nothing else.
diana_coman: BingoBoingo: can you actually talk directly to anyone there and find out exactly, specifically, concretely just *what* they have been doing, step by step, since Friday attack-time until now?
asciilifeform: hey BingoBoingo they have an office in there, don't they ? why are you stuck waiting for email replies ? go there
asciilifeform: ( why was diana_coman's 'we' able to digest the whole thing in <10min, and our 'we' takes 4days and still aint digested ? )
asciilifeform: cuz that 'we' dun seem to be worth a dime
asciilifeform: 'We are working on that with the ISP.' << who's 'the isp' ? antel ? can we cut out the layer of 'we' who 'are working with isp' ?!
BingoBoingo: asciilifeform: In the morning's hopper http://paste.deedbot.org/?id=Xr-u
asciilifeform waves to BingoBoingo
asciilifeform: atm it seems that not 1 single box is reachable.
diana_coman: asciilifeform: tbh that's the mind-boggling thing ie what exactly are/were latech doing; and I'd say BingoBoingo_ goes there with a flamethrower and doesn't leave until he finds out.
asciilifeform: diana_coman: if BingoBoingo is in the cage nao, bazaring w/ latech, as was proclaimed ( and not, saw, cowering in a hole ) i expect today we'll get something resembling an answer.
diana_coman: asciilifeform: re data in any case, it's the initial incident, I didn't specifically ask them for *all requests to my server*; I asked them exactly for "attack", apparently some 8 minutes on Friday before they updated their rules and got rid of it pretty much.
asciilifeform: unsurprising tho, ddostron likely switched to the remaining working ips, and the latech derps added them also to blackhole.
diana_coman: it's been unreachable since yesterday evening (or at least that's when I checked and found out nope)
asciilifeform: apparently whole orchestra dead in water? BingoBoingo are you alive in there ? wtf ?
ericbot: Logged on 2019-10-07 06:32:48 diana_coman: upon asking my moldavian ISP for the data re attack on my server, they sent over a .pcap file + a .pdf file with an analysis; it says max 1.5Gbits/s on Oct 4th; the whole thing seems to be coming in from 8 IPs; I still have to look at the .pdap in detail
ossabot: Logged on 2019-10-07 11:40:34 asciilifeform: http://logs.ericbenevides.com/log/trilema/2019-10-07#1941224 << can i persuade diana_coman to share that pcap ? dulap ended up unplugged just before i could set up tcpdump on it .
diana_coman: http://logs.ossasepia.com/log/trilema/2019-10-07#1941290 - you know, I *want to* publish it but I have no working blog.