22500+ entries in 0.048s
mod6: <+mircea_popescu> asciilifeform aws can definitely not be part of our canonical stuff huh. << agreed. i'll continue to get my sea-legs with what I've got so far, but what we end up supporting in the end will be a gentoo on a bare-rails environment. unless any objections?
mod6: have to pick it up tomorrow.
mod6: * rc-update: local already installed in runlevel `default'; skipping << hmm.
mod6: awe crap, might have missed `rc-update add local default`. aight one more try here.
mod6: anyone know if service starts up on its own in the uclibc version?
mod6: I'm having a bit of ssh pubkey auth issues again.
mod6: Alright I've got a Gentoo AMD64 uclibc hardened booted successfully. I basically just had a few minor tweaks to make to the notes; i.e., uclibc version doens't install grub by default, there is no /etc/locale.gen by default. That kinda thing.
mod6: Update: gathering performance metrics on v0.5.3.1-RELEASE full blockchain sync
mod6: ok all, I'm gonna rebuild this thing with uclibc, I'll have an update for ya in a few hours :]
mod6: anyway, hanbot: I'll be updating the document as I go through this whole process. I'll let you know when I get something that's bare-rails specfic. Thanks for the help! :]
mod6: for 3rd party packages right?
mod6: as far as "looking into" anything specific... i didn't. to me it just seems similar to freebsd's ports or something.
mod6: yeah, well, I did add the "portage-latest" into the handrolled gentoo.
mod6: gabriel_laddel: yeah, you'll have to forgive me, i have no idea what you're referring to.
mod6: <+gabriel_laddel> mod6: have you looked into the portage sources? << what do you mean by "looked into"?
mod6: And lastly, I really need to get some sort of physical environment here to support that last part. So I'll work on figuring out something for that as well. Maybe even an old laptop.
mod6: Apparently this is not much different than what I've already done, but I really don't know.
mod6: So basically that's where we are. I've gotten to the point where I can create an AWS AMI -> deploy hand-rolled Gentoo instance. I'll certainly continue to build out the blessed toolchain over the next week and beyond. Further, then I'll take what I've learned from all of this and apply it to building a hand-rolled Gentoo on a bare-rails environment.
mod6: Upon news of my endevour, Mr. P. asked me to document what I'd done carefully so we all can be on the same page -- which is also good because I don't know what I'm doing with Gentoo anyway. Having people look over what I'm doing provides some bit of steering in-case I get off course.
mod6: So the idea was that I would learn a few things about Gentoo, get a hand-rolled stage3 + a fully blessed tool chain and be able to attch my "btc-dev" storage volume and just pick up where we left-off -- at lesat as far as what artifacts I'd already collected.
mod6: One of my goals was to get this going as soon as possible in AWS so we could continue with builds/testing/development and ditch Debian6. And additonally, I have a 350Gb storage volume there that I wanted to not just discard as it has at least half-a-dozen fully sync'd blockchains on there.
mod6: To begin with, I was completely unfamiliar with gentoo in any sense -- aside from once trying to set it up from the Handbook, which I found that I didn't have time to work my way through and get it set up properly. So my goal was to get familiar with Gentoo and to start to build a system that can be agreed upon that suits our needs for building bitcoind.
☟︎ mod6: Allow me to elaborate for a minute here and I'll explain why I did what I did.
mod6: <+hanbot> ... hey mod6, could you make a version of this that isn't aws-specific? or wouldja be willing to me make one? << yea, absolutely. I'll get there at some point soon hopefully.
mod6: anyway, thanks for taking a look!
mod6: <+mircea_popescu> +an 18th ~200mb << 247M blk0018.dat
mod6: anyway, mircea_popescu I'll talk to ben_vulpes about helping me write something up for these charts.
mod6: 17 2.0Gb blk*.dat files = 34Gb + 9.5Gb blkindex.dat == 43.5Gb
mod6: so the size of that entire directory is not exactly an accurate picture
mod6: the debug.log itself is 977Mb
mod6: yeah, this is the size of the full bc sync (v0.5.3.1+OrphanageThermonuke)
mod6: 44G /mnt/btc-dev/.bitcoin
mod6: # du -sh /mnt/btc-dev/.bitcoin
mod6: if you're looking at timing metrics, I can just use 'time' or just rely on the time started in PS and subtract that from the end time.
mod6: i don't have the kinda hardware for this sort of ting.
mod6: so, i probably just run them consecutively unless it /really/ matters.
mod6: the one alone costs me $89/mo. and now i've got 2 others for gentoo, and another one is gonna get pricy for this month.
mod6: ascii_field: parallel?!
mod6: anyway, sorry if the charts were less then stellar, but at least /something/ to look at.
mod6: then maybe we have something to compare too... and maybe even a v0.5.3 original baseline as well.
mod6: anyway, I was considering running another full-sync without the OrphanageThermonuke patch -- just a v0.5.3.1-RELEASE baseline.
mod6: no idea. i can post the raw data if someone wants to pour through it.
mod6: most of the time, looks like 0? except for the beginning.
mod6: maybe that just means there was only a few page faults.
mod6: yeah, some of these charts don't make sense to me.
mod6: ended yesterday, the 11th.
mod6: started on 20150505
mod6: yeah, full blockchain sync
mod6: the box was doing nothing else while this thing was running.
mod6: which chart are you looking at mircea_popescu?
mod6: i'll maybe try another tool next time.
mod6: so this is just my first shot at it, really.
mod6: i just kicked it on and let it run. i've never used it before.
mod6: Hmm, the swap and & disk are blank for some reason. im sure I just didn't do something correct with those.
mod6: im gonna work on running through my doc & setting that up instead tonight.
mod6: trinque: <+mod6> bah. so now that i know a bit about uclibc, perhaps I need to go through this again with this "[19] hardened/linux/uclibc/amd64" instead of "[14] hardened/linux/amd64/no-multilib << yeah
mod6: it's kind-of baby-steps on this thing.
mod6: *nod*, I'll work on trying that out and getting an update in there. :]
mod6 just saw that option
mod6: bah. so now that i know a bit about uclibc, perhaps I need to go through this again with this "[19] hardened/linux/uclibc/amd64" instead of "[14] hardened/linux/amd64/no-multilib"
mod6: gotta get phase-2 sorted, then will create non-aws guide.
mod6: asciilifeform: yeah. i agree. maybe when I have a better idea of what's actually going on, i'll have a bare rails guide or something similar.
mod6: I'll eventually publish a final form of this to the btc-dev mailing list once I have these guys execute it manually & verify that it works for them.
mod6: ah, ok. well, if you get it in shape, let me know. i do think a lot of this could be automated.
mod6: this is a ``living-guide'' for the time being as this is pretty much ``phase-1'' of this process; ``phase-2'' will entail getting a tool-chain in place that satisfies requirements.
mod6: alright, i've got a first-draft of the gentoo stage3 AMI guide put up for review. Feel free to test this out if anyone wants. I'm having some local guys try this out for me as well. Will report any updates back here.
mod6: alright, so I'm gonna wrap up this gentoo stage3 guide, and then I'll dig in and see if I can find what's going on in there.
mod6: asciilifeform: yeah, makes sense.
mod6: but anyway, yeah if it's still too large for the pogo, i guess that doesn't help us exactly.
mod6: so i guess I'm wrong then, if that 3255776 bytes is actually available
mod6: -/+ buffers/cache: 673144 3255776
mod6: well, a lot of it was cached, yes. as opposed to "free" or "buffered"
mod6: however, it did not die. which is a first for me.
mod6: yeah, i can say, that it looks like (from my testing) that it did use pretty much almost all of the RAM available to my environment.
mod6: I did look through them.
mod6: check points are included.
mod6: great work asciilifeform!
mod6: <+mircea_popescu> it's so stupid it can not be expressed in other words. "x86 sparc" << lol