468100+ entries in 0.304s
decimation: asciilifeform: it appears a big chunk of
the memory in use is
tied up in
the CBlockIndex class
decimation: asciilifeform: your complaint is
that valgrind doesn't catch all
the memory use?
mod6: OH, i didn't actually need
to
take
the extranious '1' out. gotcha
decimation: asciilifeform: does it slow
the processing down (igprof)?
mod6: that other page about "text-output-format.html"
threw me off.
mod6: <+ascii_modem>
there are no leaks in
the usual sense <+ascii_modem>
the flat graph from mod6
tells us
this << right
mod6: <+ascii_modem> nonono not leaks << ok i see, so with -mp in
the MEM_LIVE output we're just seeing a live heap. not leaks as
the other doc page said.
decimation: I'm not sure, I need
to get back on
the patch
train
decimation: yeah
there's
tons of cruft in
there, no doubt
decimation: heh. you realize
this means you are gonna probably have
to discard all
the C++ shit
decimation: so
that it fits more comfortably in
the pogo?
decimation: asciilifeform: so is your goal
to
trim out all
the 'inactive' memory use?
mod6 goes back
to reading
decimation: ah so it wasn't killed,
that makes sense
mod6: haha. was just
typing
that out ^^
mod6: so
the docs say, "MEM_LIVE records
the ``live'' memory -- memory
that hasn't been freed. If
the profile statistics file you are processing came from
the end
decimation: "MEM_LIVE records
the ?live? memory ? memory
that hasn?t been freed. If
the profile statistics file you are processing came from
the end of
the application?s run,
this will be
the memory leaked by
the job. If
the profile statistics file was
triggered during
the running of
the job, it is a snapshot of
the heap, i.e. a heap profile.
The statistic is accurate (not statistical) and records
the number of bytes allocated and
the number of c
decimation: the igprof documentation says
that
the report shows memory leaks
mod6: <+decimation> I see what ascii means about
the flat graph (memory.png on
the orphanage
thermonuke) << ahh
mod6: and now, maybe learn
the ropes on igprof.
mod6: in addition
to
the valgrind
tests
that ascii did earlier
this month, i was going
to dig into some of
that myself in june.
decimation: I see what ascii means about
the flat graph (memory.png on
the orphanage
thermonuke)
mod6: yeah, im not sure. i need
to read more
there.
decimation: it defintely
thinks
those areas are leaks, but
that might be because of how it was killed, not because it was actually leaked (in
the sense of
the pointer was lost)
mod6: but as far as
the nmon
tests, I wanted
to system level profile
the entire sync process -- get some visuals, see what we're looking at as far as overall system usage, etc.
mod6: yeah, im not even sure what i'm looking at anymore with
that igprof stuff yet.
decimation: presumably
the 'leaks' were found by igprof because he killed it after running a certain
time, so memory wasn't freed?
mod6: is
that directed at me with my perf
tests? or at asciilifeform?
decimation: so
the goal here is
to reduce memory use, not
to find leaks?
decimation: mod6: so you did a graph of vmstat calls? Is
that what ascii is
talking about?
decimation: asciilifeform: all roads lead
to home, yes, but home is a good place
to start examining each potential pathway
ascii_modem: so naturally will appear 'heavy' because 'all roads lead
to rome'
ascii_modem: and - plz remember how
to read profilers -
the 'turd' is merely
the procedure
that is at
the root of call graph
tree
mod6: ah, ok, couldn't remember
the amt.
ascii_modem: and 256m is insane, we only have 128
total
ascii_modem: as for
the memlimit
thing, it has
to do with bdb only!
mod6: aight, me goes back
to it.
decimation: There is all kinds of allocation
that happens in
that fuction
mod6: asciilifeform: oh nm, 115500, didn't see
that in
the URL
decimation: if you follow
that
trail you find it's AddToBlockIndex
mod6: asciilifeform: how long (how many blocks) was bitcoind sync'ing while you
took
this sample?
decimation: yes but noted
that
the leaks were all in its children
mod6: lolol
that's horrendus. ProcessBlock leaked 26`847`114 bytes over 346`507 calls?!
☟︎ mod6: wait,
these are not just bytes allocated during calls,
these are LEAKED BYTES?!
mod6: have you looked at
ThreadMessageHandler2 ?
decimation: by my eye, it looks
to me
that
ThreadMessageHandler(void*) is a
turd
decimation: "Steve?s design was rejected, not because it was unsound, but because NSA did not want
to see ANY encryption work going on in
the public domain ARPA project, some say because
they did not want
to see
the world be ?too secure? by default. (Rivest and friends had just invented RSA, and
the government was
trying
to declare it
Top Secret,
then later prohibited under ITAR munitions control export laws)."
mod6: asciilifeform: in
the igprof_mem_live output, is
the "Total" column in aggregate bytes - a sum of all bytes from
the call's of said function?
decimation:
http://www.reed.com/blog-dpr/?page_id=6 < "One project where my friend and officemate Steven
T. Kent (now chief scientist and vice president at BBN, and a chief advisor
to NSA) and I lost was our strong argument
to put mandatory end-to-end encryption into
TCP (and adaptations of
the ideas
to UDP-based protocols, such as RTP, hich I worked out but abandoned). "
mod6 reminds himself
to setup
that pogo
mod6: What's
the memory cap
that you wanna use on a pogo? 256Mb?
decimation: but when you consier
that apple bought renasas for $0.48 bn, you gotta wonder if
they are overpaying
assbot: Logged on 28-05-2015 14:52:47; pete_dushenski: "Broadcom is known as a fabless company. It outsources all semiconductor manufacturing
to Asian merchant foundries, such as GlobalFoundries, Semiconductor Manufacturing International Corporation, Silterra,
TSMC, and United Microelectronics Corporation." << ahaha. so what
the fuck did $37 bn even buy ?
mod6: asciilifeform: full sync of bitcoin-v0_5_3_1-RELEASE + { OrphanageBurner } + {
TX Orphanage Amputation } underway
decimation: asciilifeform: you can get some of
this kind of data with 'perf'
ben_vulpes: asciilifeform: is
there a russian equivalent
to cursive?
ben_vulpes: <asciilifeform> interestingly,
the longer i go without writing by hand,
the more
this part of brain rots << expected behavior
ben_vulpes: and hyu with
that auto-downloading pdf
mod6: well, maybe after
this sync is done with
these 2 patches anyway. gotta still write
this SoBA yet.
mod6: i'll work on building
this
tool here
this weekend.