[Opendnssec-user] Questions regarding OpenBSD port

Thu Aug 21 14:18:49 UTC 2014

Hello,

On Tue, May 27, 2014 at 09:58:37PM +0200, Patrik Lundin wrote:
> 
> My focus is on porting the current state of the code, adding
> features is outside the scope for now :P.
> 

I have been updating the ports of opendnssec and softhsm to 1.4.6 and
1.3.7 respectively. While doing this I extended my testing from amd64
and sparc64 to i386.

Sadly this revealed a segfault in ods-signerd that I had not been able
to spot on the other arches (it was still there in 1.4.5 though).

First of all, while probably not related to the segfault, I notcied a
new warning while bulding 1.4.6:
===
shared/file.c: In function 'ods_build_path':
shared/file.c:135: warning: assignment discards qualifiers from pointer target type
===

./signer/src/shared/file.c:
===
125 ods_build_path(const char* file, const char* suffix, int dir, int no_slash)
126 {
127     size_t len_file = 0;
128     size_t len_suffix = 0;
129     size_t len_total = 0;
130     char* openf = NULL;
131     char* f = "root";
132
133     if (file) {
134         if (ods_strcmp(file, ".")) {
135             f = file;
136         }
===

I suspect the problem is that "file" is const while "f" is not. Not sure
what the best approach is to fix this, but maby someone has an idea.
(The less warnings thrown the happier I am of course :).

I have also noticed that ods-signerd is now started with the argument
"-c /etc/opendnssec/conf.xml" which is new. I guess this is OK.

Now to the main reason for my message, the segfault of ods-signerd on
i386.

After installation i perform the following steps:

Configure softhsm:
===
# grep -v '^#' /etc/softhsm.conf

0:/var/opendnssec/softhsm/slot0.db

# softhsm --init-token --slot 0 --label OpenDNSSEC
The SO PIN must have a length between 4 and 255 characters.
Enter SO PIN:
The user PIN must have a length between 4 and 255 characters.
Enter user PIN:
The token has been initialized.
===

Fix permissions on the db file:
===
# chown _opendnssec /var/opendnssec/softhsm/slot0.db
===

Bootstrap OpenDNSSEC:
===
# ods-ksmutil setup
*WARNING* This will erase all data in the database; are you sure? [y/N] y
fixing permissions on file /var/opendnssec/db/kasp.db
zonelist filename set to /etc/opendnssec/zonelist.xml.
kasp filename set to /etc/opendnssec/kasp.xml.
Repository SoftHSM found
No Maximum Capacity set.
RequireBackup NOT set; please make sure that you know the potential
problems of using keys which are not recoverable
INFO: The XML in /etc/opendnssec/conf.xml is valid
INFO: The XML in /etc/opendnssec/zonelist.xml is valid
INFO: The XML in /etc/opendnssec/kasp.xml is valid
WARNING: In policy default, Y used in duration field for Keys/KSK Lifetime
(P1Y) in /etc/opendnssec/kasp.xml - this will be interpreted as 365 days
WARNING: In policy lab, Y used in duration field for Keys/KSK Lifetime
(P1Y) in /etc/opendnssec/kasp.xml - this will be interpreted as 365 days
Policy default found
Info: converting P1Y to seconds; M interpreted as 31 days, Y interpreted as
365 days
Policy lab found
Info: converting P1Y to seconds; M interpreted as 31 days, Y interpreted as
365 days

# /etc/rc.d/opendnssec start
opendnssec(ok)
# ps auxww | grep ods
_opendnssec 23017  0.0  2.5  1716  6448 ??  Ss    11:22AM    0:00.04 /usr/local/sbin/ods-enforcerd
_opendnssec 10192  0.0  2.4 10928  6264 ??  Ss    11:22AM    0:00.20 /usr/local/sbin/ods-signerd -c /etc/opendnssec/conf.xml
===

Add an unsigned zone file:
===
# cp /root/example.com /var/opendnssec/unsigned/
# cat /var/opendnssec/unsigned/example.com
$ORIGIN example.com.
$TTL 300
@ SOA ns1.example.com. hostmaster.example.com. (
               1369872000 ; Serial
               3H         ; Refresh after three hours
               1H         ; Retry after one hour
               1W         ; Expire after one week
               1D )       ; Minimum one day TTL

@               3600    IN      NS      ns1.example.com.
@               3600    IN      NS      ns2.example.com.

ns1             3600    IN      A       10.0.0.1
ns2             3600    IN      A       10.0.0.2
===

Add the zone (and verify that the processes still exist):
===
# ods-ksmutil zone add --zone example.com --policy default
zonelist filename set to /etc/opendnssec/zonelist.xml.
Imported zone: example.com

# ps auxww | grep ods
_opendnssec 23017  0.0  2.5  1716  6448 ??  Is    11:22AM    0:00.04 /usr/local/sbin/ods-enforcerd
_opendnssec 10192  0.0  2.4 10928  6264 ??  Is    11:22AM    0:00.20 /usr/local/sbin/ods-signerd -c /etc/opendnssec/conf.xml
===

After running "notify" ods-signerd disappears (and no file appears in the
signed/ directory):
===
# ods-control enforcer notify
Notifying enforcer of new database...
# ps auxww | grep ods
_opendnssec 23017  0.0  2.6  1768  6820 ??  Ss    11:22AM    0:00.68 /usr/local/sbin/ods-enforcerd
===

Trying to run ods-signerd manually at this point results in a segfault:
===
# /usr/local/sbin/ods-signerd -c /etc/opendnssec/conf.xml -d
OpenDNSSEC signer engine version 1.4.6
Segmentation fault
===

I managed to create a coredump (after setting the _opendnssec user shell
to /bin/sh via vipw):
===
# su _opendnssec
$ ls /var/opendnssec/tmp/
$ /usr/local/sbin/ods-signerd -c /etc/opendnssec/conf.xml -d
OpenDNSSEC signer engine version 1.4.6
Segmentation fault (core dumped)
$ ^D
# ls -l /var/opendnssec/tmp/
total 22400
-rw-------  1 _opendnssec  _opendnssec  11445288 Aug 20 11:26 ods-signerd.core
===

Here is the result from running gdb and printing a backtrace:
===
# gdb /usr/local/sbin/ods-signerd /var/opendnssec/tmp/ods-signerd.core
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-unknown-openbsd5.6"...
Core was generated by `ods-signerd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libpthread.so.18.0...done.
Loaded symbols for /usr/lib/libpthread.so.18.0
Loaded symbols for /usr/local/sbin/ods-signerd
Reading symbols from /usr/lib/libcrypto.so.30.0...done.
Loaded symbols for /usr/lib/libcrypto.so.30.0
Reading symbols from /usr/local/lib/libldns.so.6.1...done.
Loaded symbols for /usr/local/lib/libldns.so.6.1
Reading symbols from /usr/local/lib/libxml2.so.15.1...done.
Loaded symbols for /usr/local/lib/libxml2.so.15.1
Reading symbols from /usr/lib/libz.so.5.0...done.
Loaded symbols for /usr/lib/libz.so.5.0
Reading symbols from /usr/local/lib/libiconv.so.6.0...done.
Loaded symbols for /usr/local/lib/libiconv.so.6.0
Reading symbols from /usr/lib/libm.so.9.0...done.
Loaded symbols for /usr/lib/libm.so.9.0
Symbols already loaded for /usr/lib/libpthread.so.18.0
Reading symbols from /usr/lib/libc.so.77.0...done.
Loaded symbols for /usr/lib/libc.so.77.0
Reading symbols from /usr/libexec/ld.so...done.
Loaded symbols for /usr/libexec/ld.so
Reading symbols from /usr/local/lib/softhsm/libsofthsm.so...done.
Loaded symbols for /usr/local/lib/softhsm/libsofthsm.so
Reading symbols from /usr/local/lib/libbotan-1.10.so.0.1...done.
Loaded symbols for /usr/local/lib/libbotan-1.10.so.0.1
Reading symbols from /usr/local/lib/libbz2.so.10.4...done.
Loaded symbols for /usr/local/lib/libbz2.so.10.4
Reading symbols from /usr/local/lib/libgmp.so.9.0...done.
Loaded symbols for /usr/local/lib/libgmp.so.9.0
Reading symbols from /usr/lib/libstdc++.so.57.0...done.
Loaded symbols for /usr/lib/libstdc++.so.57.0
Reading symbols from /usr/lib/libsqlite3.so.27.0...done.
Loaded symbols for /usr/lib/libsqlite3.so.27.0
#0  0x19242700 in denial_nsecify (denial=0x7a46f8a0, nxt=0x7a46f760, num_added=0x8042b134) at signer/denial.c:301
301     {
(gdb) bt
#0  0x19242700 in denial_nsecify (denial=0x7a46f8a0, nxt=0x7a46f760, num_added=0x8042b134) at signer/denial.c:301
#1  0x19245c47 in namedb_nsecify (db=0x829b4440, num_added=0x8042b180) at signer/namedb.c:898
#2  0x19228ddd in adapi_trans_full (zone=0x7f98aa00, more_coming=0) at adapter/adapi.c:139
#3  0x1922c5cf in adfile_read (zone=0x7f98aa00) at adapter/adfile.c:315
#4  0x192292a8 in adapter_read (zone=0x7f98aa00) at adapter/adapter.c:177
#5  0x1924c4f6 in tools_input (zone=0x7f98aa00) at signer/tools.c:135
#6  0x19237550 in worker_work (worker=0x7aa0e5c0) at daemon/worker.c:354
#7  0x19237c57 in worker_start (worker=0x7aa0e5c0) at daemon/worker.c:766
#8  0x192328d2 in worker_thread_start (arg=0x7aa0e5c0) at daemon/engine.c:371
#9  0x0d24480e in _rthread_start (v=0x829b5400) at /usr/src/lib/librthread/rthread.c:145
#10 0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95
(gdb)
===

I'll also show the backtrace of all threads just in case someone wants
to see it:
===
(gdb) thread apply all bt

Thread 11 (process 2339):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2
#1  0x0d246a8a in pthread_cond_wait (condp=0x81d0ff4c, mutexp=0x81d0ff48) at /usr/src/lib/librthread/rthread_sync.c:501
#2  0x1923f482 in ods_thread_wait (cond=0x81d0ff4c, lock=0x81d0ff48, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:135
#3  0x192381fc in worker_start (worker=0x80d27bc0) at daemon/worker.c:688
#4  0x192328d2 in worker_thread_start (arg=0x80d27bc0) at daemon/engine.c:371
#5  0x0d24480e in _rthread_start (v=0x7c8ba600) at /usr/src/lib/librthread/rthread.c:145
#6  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95
Current language:  auto; currently asm

Thread 10 (process 6989):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2
#1  0x0d246a8a in pthread_cond_wait (condp=0x81d0ff4c, mutexp=0x81d0ff48) at /usr/src/lib/librthread/rthread_sync.c:501
#2  0x1923f482 in ods_thread_wait (cond=0x81d0ff4c, lock=0x81d0ff48, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:135
#3  0x192381fc in worker_start (worker=0x8224ba00) at daemon/worker.c:688
#4  0x192328d2 in worker_thread_start (arg=0x8224ba00) at daemon/engine.c:371
#5  0x0d24480e in _rthread_start (v=0x829b5900) at /usr/src/lib/librthread/rthread.c:145
#6  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

Thread 9 (process 11568):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2
#1  0x0d246a8a in pthread_cond_wait (condp=0x81d0ff4c, mutexp=0x81d0ff48) at /usr/src/lib/librthread/rthread_sync.c:501
#2  0x1923f482 in ods_thread_wait (cond=0x81d0ff4c, lock=0x81d0ff48, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:135
#3  0x192381fc in worker_start (worker=0x7aa0ef40) at daemon/worker.c:688
#4  0x192328d2 in worker_thread_start (arg=0x7aa0ef40) at daemon/engine.c:371
#5  0x0d24480e in _rthread_start (v=0x7ffba400) at /usr/src/lib/librthread/rthread.c:145
#6  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

Thread 8 (process 15665):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2
#1  0x0d246a8a in pthread_cond_wait (condp=0x81d0ff4c, mutexp=0x81d0ff48) at /usr/src/lib/librthread/rthread_sync.c:501
#2  0x1923f482 in ods_thread_wait (cond=0x81d0ff4c, lock=0x81d0ff48, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:135
#3  0x192381fc in worker_start (worker=0x80d27880) at daemon/worker.c:688
#4  0x192328d2 in worker_thread_start (arg=0x80d27880) at daemon/engine.c:371
#5  0x0d24480e in _rthread_start (v=0x7c961100) at /usr/src/lib/librthread/rthread.c:145
#6  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

Thread 7 (process 20299):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2
#1  0x0d246eaf in pthread_cond_timedwait (condp=0x8224b070, mutexp=0x8224b074, abstime=0x8404cbd8) at /usr/src/lib/librthread/rthread_sync.c:354
#2  0x1923f3ff in ods_thread_wait (cond=0x8224b070, lock=0x8224b074, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:133
#3  0x19235a04 in worker_sleep (worker=0x8224b040, timeout=2) at daemon/worker.c:787
#4  0x19236bd3 in worker_work (worker=0x8224b040) at daemon/worker.c:642
#5  0x19237c57 in worker_start (worker=0x8224b040) at daemon/worker.c:766
#6  0x192328d2 in worker_thread_start (arg=0x8224b040) at daemon/engine.c:371
#7  0x0d24480e in _rthread_start (v=0x7c8bad00) at /usr/src/lib/librthread/rthread.c:145
#8  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

Thread 6 (process 9477):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2
#1  0x0d246eaf in pthread_cond_timedwait (condp=0x7ac2a0f0, mutexp=0x7ac2a0f4, abstime=0x7fbdaad8) at /usr/src/lib/librthread/rthread_sync.c:354
#2  0x1923f3ff in ods_thread_wait (cond=0x7ac2a0f0, lock=0x7ac2a0f4, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:133
#3  0x19235a04 in worker_sleep (worker=0x7ac2a0c0, timeout=2) at daemon/worker.c:787
#4  0x19236bd3 in worker_work (worker=0x7ac2a0c0) at daemon/worker.c:642
#5  0x19237c57 in worker_start (worker=0x7ac2a0c0) at daemon/worker.c:766
#6  0x192328d2 in worker_thread_start (arg=0x7ac2a0c0) at daemon/engine.c:371
#7  0x0d24480e in _rthread_start (v=0x7c8bac00) at /usr/src/lib/librthread/rthread.c:145
#8  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

Thread 5 (process 24475):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2 #1  0x0d246eaf in pthread_cond_timedwait (condp=0x8224b830, mutexp=0x8224b834, abstime=0x7cfc63f8) at /usr/src/lib/librthread/rthread_sync.c:354
#2  0x1923f3ff in ods_thread_wait (cond=0x8224b830, lock=0x8224b834, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:133
#3  0x19235a04 in worker_sleep (worker=0x8224b800, timeout=2) at daemon/worker.c:787
#4  0x19236bd3 in worker_work (worker=0x8224b800) at daemon/worker.c:642
#5  0x19237c57 in worker_start (worker=0x8224b800) at daemon/worker.c:766
#6  0x192328d2 in worker_thread_start (arg=0x8224b800) at daemon/engine.c:371
#7  0x0d24480e in _rthread_start (v=0x7c8ba400) at /usr/src/lib/librthread/rthread.c:145
#8  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

Thread 4 (process 7159):
#0  0x01a6f8cd in select () at <stdin>:2
#1  0x0d247ce0 in select (nfds=0, readfds=0x88d88968, writefds=0x88d888e8, exceptfds=0x88d88868, timeout=0x0) at /usr/src/lib/librthread/rthread_cancel.c:494
#2  0x1926e3fe in pselect (n=0, readfds=0x88d88968, writefds=0x88d888e8, exceptfds=0x88d88868, timeout=0x0, sigmask=0x0) at pselect.c:39
#3  0x19257197 in netio_dispatch (netio=0x824fb080, timeout=0x0, sigmask=0x0) at wire/netio.c:279
#4  0x19231a7c in xfrhandler_start (xfrhandler=0x795a9e80) at daemon/xfrhandler.c:132
#5  0x1923219d in xfrhandler_thread_start (arg=0x795a9e80) at daemon/engine.c:257
#6  0x0d24480e in _rthread_start (v=0x7c8baf00) at /usr/src/lib/librthread/rthread.c:145
#7  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

---Type <return> to continue, or q <return> to quit---
Thread 3 (process 4845):
#0  0x01a6f8cd in select () at <stdin>:2
#1  0x0d247ce0 in select (nfds=4, readfds=0x80be90c8, writefds=0x0, exceptfds=0x0, timeout=0x0) at /usr/src/lib/librthread/rthread_cancel.c:494
#2  0x1922d531 in cmdhandler_start (cmdhandler=0x7c8bab00) at daemon/cmdhandler.c:975
#3  0x19232142 in cmdhandler_thread_start (arg=0x7c8bab00) at daemon/engine.c:133
#4  0x0d24480e in _rthread_start (v=0x7c961b00) at /usr/src/lib/librthread/rthread.c:145
#5  0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95

Thread 2 (process 18282):
#0  0x01a7f3b9 in _thread_sys___thrsleep () at <stdin>:2
#1  0x0d246eaf in pthread_cond_timedwait (condp=0x795a905c, mutexp=0x795a9060, abstime=0xcfbe8968) at /usr/src/lib/librthread/rthread_sync.c:354
#2  0x1923f3ff in ods_thread_wait (cond=0x795a905c, lock=0x795a9060, wait=The value of variable 'wait' is distributed across several locations, and GDB cannot access its value.

) at shared/locks.c:133
#3  0x192330fc in engine_run (engine=0x795a9000, single_run=0) at daemon/engine.c:666
#4  0x19234dfd in engine_start (cfgfile=0xcfbe8ba7 "/etc/opendnssec/conf.xml", cmdline_verbosity=0, daemonize=0, info=0, single_run=0) at daemon/engine.c:1063
#5  0x19227d9a in main (argc=4, argv=0xcfbe8b04) at ods-signerd.c:161

Thread 1 (process 31436):
#0  0x19242700 in denial_nsecify (denial=0x7a46f8a0, nxt=0x7a46f760, num_added=0x8042b134) at signer/denial.c:301
#1  0x19245c47 in namedb_nsecify (db=0x829b4440, num_added=0x8042b180) at signer/namedb.c:898
#2  0x19228ddd in adapi_trans_full (zone=0x7f98aa00, more_coming=0) at adapter/adapi.c:139
#3  0x1922c5cf in adfile_read (zone=0x7f98aa00) at adapter/adfile.c:315
#4  0x192292a8 in adapter_read (zone=0x7f98aa00) at adapter/adapter.c:177
#5  0x1924c4f6 in tools_input (zone=0x7f98aa00) at signer/tools.c:135
#6  0x19237550 in worker_work (worker=0x7aa0e5c0) at daemon/worker.c:354
#7  0x19237c57 in worker_start (worker=0x7aa0e5c0) at daemon/worker.c:766
#8  0x192328d2 in worker_thread_start (arg=0x7aa0e5c0) at daemon/engine.c:371
#9  0x0d24480e in _rthread_start (v=0x829b5400) at /usr/src/lib/librthread/rthread.c:145
#10 0x01a70bb6 in __tfork_thread () at /usr/src/lib/libc/arch/i386/sys/tfork_thread.S:95
Current language:  auto; currently c
(gdb)
===

Does anyone have an idea what might be causing this? I am somewhat
suspecting that it could be caused by the different sizes of data types
on 32bit vs. 64 bit. But this is just an unqualified guess at this
point.

Let me know if you wan't me to test something!

Regards,
Patrik Lundin