asciilifeform: (that is, by plastering over retardation with magic)

asciilifeform: that is how you get shitgnomatrons in the first place

asciilifeform: magical allocators are entirely the wrong place to begin.

asciilifeform: but,

asciilifeform: none of it is hard.

asciilifeform waits for mircea_popescu to eat the log

asciilifeform: mircea_popescu is awake !

asciilifeform: it does -not- appear to grow significantly as the thing runs

asciilifeform: (see earlier link)

asciilifeform: decimation: that might be the bdb cache

asciilifeform: is there even any reason ever to keep more than one or two (second being, one that is in the process of load/verify) blocks in ram at one time

asciilifeform: there is absolutely no reason for bitcoind to ever use more than at most a few MB more than when it first boots.

asciilifeform: let's put it another way:

asciilifeform: trivially verified with pmap

asciilifeform: aha

asciilifeform: i want everybody to try this at home.

asciilifeform: i'd fix it, but these links are not intended as permanent

asciilifeform: yes, it's a typo

asciilifeform: http://www.loper-os.org/pub/1172660_mem_max.txt

asciilifeform: this is -exactly- my complaint with valgrind.

asciilifeform: where the hell did -these- go ?

asciilifeform: ;;calc 144897106 - 74026911

asciilifeform: then again, it is running on a very generously oversized box

asciilifeform: decimation: not noticeably

asciilifeform: 172660, meant.

asciilifeform: rather, extra 1

asciilifeform: l0l extra 0 in there

asciilifeform: ^ where it is now

asciilifeform: http://www.loper-os.org/pub/1172660_mem_max.txt

asciilifeform: and

asciilifeform: http://www.loper-os.org/pub/1172660_mem_live.txt

asciilifeform: didn't i kill those already ?

asciilifeform: fuck the fixed hostnames

asciilifeform: every piece of shit that gets stored & kept around for no reason

asciilifeform: every idiot dependency that just adds retardation (e.g., dns)

asciilifeform: every kind of cache

asciilifeform: every piece of 'orphan' nonsense, where thing works 100% as well with it gone

asciilifeform: just by taking every piece of retarded cruft and shooting it in the head.

asciilifeform: decimation: aaaaand without any major rewrite.

asciilifeform: in fact, i want it in 32M. ☟︎

asciilifeform: there is no acceptable reason for the thing not to go in this very generous box.

asciilifeform: whatever it takes.

asciilifeform: decimation: my goal is to get it to sit down in 120M or so.

asciilifeform: this is not valgrind! it does not need the process to terminate.

asciilifeform: as in, actually alive

asciilifeform: it is in use !

asciilifeform: well no shit it wasn't freed

asciilifeform: decimation: all i noticed is the same 'pocket change' leaks as valgrind shows

asciilifeform: and i will add, nothing was killed, that thing is running even now, without interruption

asciilifeform: decimation: where do you see leaks ?

asciilifeform: this -never- happened at any point when i actually picked up a pen on a daily basis (school)

asciilifeform: i even found myself, as of 4-5 yrs ago, often writing the 2nd letter of a word first

asciilifeform: interestingly, the longer i go without writing by hand, the more this part of brain rots

asciilifeform now can only write in the worst possible print

asciilifeform would, but had handwriting neural circuitry utterly nuked by alphabet switch in childhood

asciilifeform: iirc there might be some fancy html-output mode also. but - have not tried this yet.

asciilifeform: (if you have libs with debug symbols, will display line #s therein as well)

asciilifeform: also the thing appears to have better coverage of dynamic libs

asciilifeform: instead of waiting for the thing to die, as in valgrind.

asciilifeform: and you can get these -during runtime- now

asciilifeform: aha.

asciilifeform: ^ both orphanage-destroyers applied for this example run

asciilifeform: ben_vulpes, mod6, mircea_popescu, et al: ^^^^ sample output

asciilifeform: http://www.loper-os.org/pub/blockheight-115500_igprof_mem_live.txt

asciilifeform: nd the number of calls involved.'

asciilifeform: 'MEM_LIVE records the “live” memory – memory that hasn’t been freed. If the profile statistics file you are processing came from the end of the application’s run, this will be the memory leaked by the job. If the profile statistics file was triggered during the running of the job, it is a snapshot of the heap, i.e. a heap profile. The statistic is accurate (not statistical) and records the number of bytes allocated a

asciilifeform: http://igprof.org/analysis.html

asciilifeform: other options (e.g., MEM_LIVE) described in manual.

asciilifeform: this gives MEM_TOTAL (how much churned)

asciilifeform: (just like many other profilers, this one craps out somewhat terse encoded transcript by default)

asciilifeform: gives the plain text output

asciilifeform: and so forth

asciilifeform: igprof-analyse -d -v -g -r MEM_TOTAL heap1 > heap1.txt

asciilifeform: btw, after dumping a heap profile snapshot,

asciilifeform: http://igprof.org

asciilifeform: mod6: i have never seen, smelled, touched - either.

asciilifeform: 2nd one is some dark magic, only serious sappers need apply

asciilifeform: (the network is)

asciilifeform: ben_vulpes: your hardware is not the limiting reactant here

asciilifeform: mod6: aha

asciilifeform: (goes without saying that you need igprof on your box for this to work)

asciilifeform: no hurry

asciilifeform: just call getinfo along with profileheapdump

asciilifeform: mod6: you can use this to create a blockheight vs heap plot

asciilifeform: (EXPERIMENTAL) IgProf Hooks Apparatus !

asciilifeform: http://therealbitcoin.org/ml/btc-dev/2015-May/000095.html

asciilifeform: ben_vulpes, mod6, mircea_popescu, et al:

asciilifeform: achtung, panzers!

asciilifeform: aaaaaand now for something completely different...

asciilifeform: ben_vulpes, mod6, mircea_popescu, et al: ^^

asciilifeform: re: memory hunger: anyone investigated effect of https://docs.oracle.com/cd/E17076_04/html/api_reference/CXX/envset_memory_max.html ?

asciilifeform: 184650, ~166M

asciilifeform: and in the case of 'pogo', the door is not in your house, but in... your car.

asciilifeform: imagine having a special door in your house through which perfect strangers are invited to dump in up to five thousand tonnes of gravel.

asciilifeform: 'hoarders' belong in lunatic asylums.

asciilifeform: and yes, even with the count-bound, the in-practice bound on the tx orphanage bytewise is ridiculous - 5GB ?

asciilifeform: ben_vulpes, mod6, mircea_popescu, et al: incidentally, even though 'transaction orphanage amputator' may not be a magical pill against oom-on-pogo (my preliminary investigation suggests that it is -not-) it still rips out the idiotic ddos vector that every unbounded cache of anything whatsoever is.

asciilifeform: danielpbarron: iirc you had something set up for this ^ ?

asciilifeform: danielpbarron ^

asciilifeform: ben_vulpes, mod6, mircea_popescu, et al: incidentally, now that we have a bounded memory footprint, it may be time to try running pogo with swap space again