[Opendnssec-user] ods-signerd 1.4.0a1 running out of memory and crashing on null context

[Matthijs Mekking]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Paul,


On 05/21/2012 11:11 PM, Paul Wouters wrote:
> 
> Looks like ods-signerd ran out of memory after a few days
> 
> /var/log/messages-20120520:May 21 00:27:15 signer02 kernel: [58267]
> 630 58267 1990597   795193   2       0             0 ods-signerd 
> /var/log/messages-20120520:May 21 00:27:15 signer02 kernel: Out of 
> memory: Kill process 58267 (ods-signerd) score 599 or sacrifice
> child /var/log/messages-20120520:May 21 00:27:15 signer02 kernel:
> Killed process 58267, UID 630, (ods-signerd) total-vm:7962388kB, 
> anon-rss:3180692kB, file-rss:80kB
> 
> This is for a 1M test zone, no optout, on a machine with 8GB of
> RAM. This is using 1.4.0a1 with the delete rrset/nsec3 patch
> applied.
> 
> It has signed this zone repeatedly before, so my guess is that
> there is a leak causing this to happen over the course of a few
> days. I have added smoe memory statistics logging to confirm this.

How much is the signer using regularly for the 1M test zone? Could it
be that there is a slight increase in zone size and thus memory? Could
you provide the memory statistics?

> On restarting the signer I got :
> 
> ods-signerd: signer/rrset.c:667: rrset_sign: assertion ctx failed

Do you also got this log message?:

ods-signerd: [worker[1]] error creating libhsm context

I am not sure how to deal with a failing libhsm. We could abort the
drudger, stop the signer, retry until eternity, ...


Best regards,
  Matthijs

> 
> (note that the machine is using ldns 1.6.13rc1)
> 
> 
> (gdb) bt full #0  0x0000003988232885 in raise () from
> /lib64/libc.so.6 No symbol table info available. #1
> 0x0000003988234065 in abort () from /lib64/libc.so.6 No symbol
> table info available. #2  0x0000000000418016 in ods_fatal_exit
> (format=<value optimized out>) at shared/log.c:368 args =
> {{gp_offset = 40, fp_offset = 48, overflow_arg_area =
> 0x7fffeebfcde0, reg_save_area = 0x7fffeebfcd20}} #3
> 0x000000000041fd5e in rrset_sign (ctx=0x0, rrset=0x7fffe800d3c0, 
> signtime=1337631628) at signer/rrset.c:667 zone = 0x0 newsigs = 0 
> reusedsigs = 0 rrsig = 0x0 rr_list = 0x0 signature = 0x0 locator =
> 0x0 inception = 0 expiration = 0 i = 0 domain = 0x0 dstatus =
> LDNS_RR_TYPE_FIRST #4  0x00000000004119e1 in worker_drudge
> (worker=0x880c10) at daemon/worker.c:659 engine = 0x857480 zone =
> 0x85ca80 status = <value optimized out> ctx = 0x0 task = <value
> optimized out> rrset = 0x7fffe800d3c0 superior = 0x88a7f0 #5
> worker_start (worker=0x880c10) at daemon/worker.c:704 __func__ =
> "worker_start" #6  0x000000000040d6c1 in worker_thread_start
> (arg=<value optimized out>) at daemon/engine.c:366 worker =
> 0x880c10 #7  0x00000039886077f1 in start_thread () from
> /lib64/libpthread.so.0
> 
> _______________________________________________ Opendnssec-user
> mailing list Opendnssec-user at lists.opendnssec.org 
> https://lists.opendnssec.org/mailman/listinfo/opendnssec-user

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPu0InAAoJEA8yVCPsQCW5ihUIAJ6EnstDv6YTOPFTh3msB867
0fHs3R9eJEeHFkSQ35+grN5wHtylR3Lz3P/PAYC0izVt+hMcFy+8XQlGjPN51Trl
c0IWOdctjl0BEhF9brg2CXo6KO2cAgd9S1JfMDIwgUf0uj1409EiYJcd5cMt4cLD
Xp+jjgZkR52T3i2QmiUGnD3KUAB7733KDnEcZ/f9/hMcbIqIxJhiW0K/iMm4Tizr
0J72Aa53WR/HC3ZDtmEPCtNbHnpqHd9dMzuEMSKk+RIqCdu4QnofbxAskhjCaZtU
nXXidO2q6aAroY6rhM0tvXIROIF+I4BztO9QmRuflzRR1umFB3N18OVOKd0VjXs=
=98YT
-----END PGP SIGNATURE-----