[Opendnssec-user] signer does not find a key

Sebastian Castro sebastian at nzrs.net.nz
Wed Dec 17 02:15:53 UTC 2014



On 17/12/14 12:56 am, Emil Natan wrote:
> Hi Matthijs and thank you for your reply.
> 

Hi Emil:

Your problem seems really odd, but for some reason not strange. We've
done some testing with the AEP Keyper, and it seems there is a mapping
between key id and HSM used that lives in a BerkeleyDB file somewhere in
the file system.

I don't recall the location of the file at the moment, and don't have
notes, but came across with something similar before.

You can find where the file is while stracing the command

ods-hsmutil generate Keyper rsa 1024

Also you can try with ods-hsmutil to generate a DNSKEY from an existing
key, perhaps the problem is your program doesn't have access to read the
mapping file.

If you run

ods-hsmutil dnskey 39a954b0fccb0f5ed73614d5fc1a8144 test.

as the root used should work, but if you run

sudo -u opendnssec ods-hsmutil dnskey 39a954b0fccb0f5ed73614d5fc1a8144 test.

it should fail.

Let us know how it works, I'll ask internally to find out if someone
remembers the name of the bloody file!

> Here is how it goes for me.
> 
> I start with:
> Zone:                           Keytype:      State:    Date of next
> transition:
> XXX                              KSK           active    2016-01-16
> 09:49:45 
> XXX                              ZSK           active    2015-04-18 22:40:55
> 
> root at debugsigner002:~# ods-hsmutil purge Keyper
> Purging all keys from repository: Keyper
> 12 keys found.
> 
> Are you sure you want to remove ALL keys from repository Keyper ?
> (YES/NO) yes
> 
> Starting purge...
> Key remove successful: fdd17d120d3e548a104dda856d84c770
> ...
> Key remove successful: db97ded0cc231c3908f8f20f5ce21229
> Key remove successful: f81e4b2cb33eec780320b6ceeb6f6bb8
> Purge done.
> 
> root at debugsigner002:~# /opt/Keyper/PKCS11Provider/inittoken
> ...
> PKCS11 Slot     : 0
> PKCS11 Label    : aepkeyper                       
> Keyper Model    : Keyper Ent 1126 
> Keyper Serial   :         
> Keyper version  : 2.0
> App             : 020
> ABL             : 029
> AL              : 02
> --------------------------------------------
> Token initialised OK
> ********************************************
> 
> To remove the zone I actually comment it out from zonelist.xml, then:
> 
> root at debugsigner002:~# ods-ksmutil update zonelist
> zonelist filename set to /ods-data/etc/opendnssec/zonelist.xml.
> kasp filename set to /ods-data/etc/opendnssec/kasp.xml.
> Removing zone XXX from database
> Notifying enforcer of new database...
> 
> I stopped both ODS daemons.
> 
> root at debugsigner002:~# ps auxww | grep ods
> root     14452  0.0  0.0  11744   896 pts/2    S+   13:31   0:00 grep
> --color=auto ods
> 
> Initialize ODS, all the warnings are skipped, but no errors.
> 
> root at debugsigner002:~# ods-ksmutil setup
> 
> *WARNING* This will erase all data in the database; are you sure? [y/N] y
> zonelist filename set to /ods-data/etc/opendnssec/zonelist.xml.
> kasp filename set to /ods-data/etc/opendnssec/kasp.xml.
> Repository Keyper found
> No Maximum Capacity set.
> RequireBackup set.
> INFO: The XML in /ods-data/etc/opendnssec/conf.xml is valid
> INFO: The XML in /ods-data/etc/opendnssec/zonelist.xml is valid
> INFO: The XML in /ods-data/etc/opendnssec/kasp.xml is valid
> Policy XXXTLD found
> 
> Generate new keys.
> 
> root at debugsigner002:~# ods-ksmutil key generate --policy XXXTLD
> --zonetotal 1 --interval P2Y
> Key sharing is Off
> Info: converting P2Y to seconds; M interpreted as 31 days, Y interpreted
> as 365 days
> HSM opened successfully.
> Info: 0 zone(s) found on policy "XXXTLD"
> Info: Keys will actually be generated for a total of 1 zone(s) as
> specified by zone total parameter
> 2 new KSK(s) (2048 bits) need to be created for policy XXXTLD:
> keys_to_generate(2) = keys_needed(2) - keys_available(0).
> 6 new ZSK(s) (1024 bits) need to be created for policy XXXTLD:
> keys_to_generate(6) = keys_needed(6) - keys_available(0).
> *WARNING* This will create 2 KSKs (2048 bits) and 6 ZSKs (1024 bits)
> Are you sure? [y/N] 
> y
> Created KSK size: 2048, alg: 8 with id: 39a954b0fccb0f5ed73614d5fc1a8144
> in repository: Keyper and database.
> Created KSK size: 2048, alg: 8 with id: 47dc08d7c5be2104b18a9f7a1702e6b0
> in repository: Keyper and database.
> Created ZSK size: 1024, alg: 8 with id: 64504804f1dc34cd44fa83cbede95275
> in repository: Keyper and database.
> Created ZSK size: 1024, alg: 8 with id: ec77d359ccdde3e38b222423a5d2075f
> in repository: Keyper and database.
> Created ZSK size: 1024, alg: 8 with id: 669a0a563fa03c62fc58d20e85628b35
> in repository: Keyper and database.
> Created ZSK size: 1024, alg: 8 with id: e799a40efda79c8e98a76adc72470f6d
> in repository: Keyper and database.
> Created ZSK size: 1024, alg: 8 with id: 0618eb27e1061e37df6bf6a055c85160
> in repository: Keyper and database.
> Created ZSK size: 1024, alg: 8 with id: 424fbb66fbaf3605b4d935b473d1be01
> in repository: Keyper and database.
> NOTE: keys generated in repository Keyper will not become active until
> they have been backed up
> all done! hsm_close result: 0
> 
> I also mark the keys as backed up.
> 
> root at debugsigner002:~# ods-ksmutil backup prepare
> Marked all repositories as pre-backed up at 2014-12-16 13:40:15
> root at debugsigner002:~# ods-ksmutil backup commit
> Marked all repositories as backed up at 2014-12-16 13:40:21
> 
> This time I stopped the signer and enforcer before setup, so I start them.
> 
> root at debugsigner002:~# ps auxww | grep ods
> opendns+ 14492  0.0  0.5 128840  5548 ?        Ss   13:42   0:00
> /ods-bin/sbin/ods-enforcerd
> opendns+ 14501  0.0  0.6 533744  7068 ?        Ssl  13:42   0:00
> /ods-bin/sbin/ods-signerd
> root     14514  0.0  0.0  11744   896 pts/1    S+   13:45   0:00 grep
> --color=auto ods
> 
> I added the zone, again by editing zonelist.xml and ...
> 
> root at debugsigner002:~# ods-ksmutil update zonelist
> zonelist filename set to /ods-data/etc/opendnssec/zonelist.xml.
> kasp filename set to /ods-data/etc/opendnssec/kasp.xml.
> Zone XXX found; policy set to XXXTLD
> Notifying enforcer of new database...
> 
> And I end up with the same problem.
> 
> Dec 16 13:46:08 debugsigner002 ods-signerd: [hsm] libhsm connection ok
> Dec 16 13:46:08 debugsigner002 ods-signerd: [hsm] unable to get key: key
> 39a954b0fccb0f5ed73614d5fc1a8144 not found
> Dec 16 13:46:08 debugsigner002 ods-signerd: [zone] unable to publish
> dnskeys for zone XXX: error creating dnskey
> Dec 16 13:46:08 debugsigner002 ods-signerd: [tools] unable to read zone
> XXX: failed to publish dnskeys (General error)
> Dec 16 13:46:08 debugsigner002 ods-signerd: [worker[1]] CRITICAL: failed
> to sign zone XXX: General error
> 
> And ods-ksmutil can still list the keys:
> 
> root at debugsigner002:~# ods-ksmutil key list -v
> Zone:                           Keytype:      State:    Date of next
> transition (to):  Size:   Algorithm:  CKA_ID:                          
> Repository:                       Keytag:
> XXX                              ZSK           active    2015-04-19
> 13:46:07 (retire)   1024    8           64504804f1dc34cd44fa83cbede95275
>  Keyper                            5680
> XXX                              KSK           publish   2014-12-16
> 17:51:07 (ready)    2048    8           39a954b0fccb0f5ed73614d5fc1a8144
>  Keyper                            6962
> 
> I'll send you the full log off-list.
> Thanks again.
> 
> Emil
> 
> On Tue, Dec 16, 2014 at 12:18 PM, Matthijs Mekking
> <matthijs at pletterpet.nl <mailto:matthijs at pletterpet.nl>> wrote:
> 
>     Hi Emil,
> 
>     Short: I tried to simulate your use case (with SoftHSM, on
>     ubuntu-trusty-64 VM), but it seems to work for me. Perhaps I used
>     slightly different commands? Can you share your used commands?
> 
>     Best regards,
>       Matthijs
> 
> 
>     Audit trail:
> 
>     I started with Keys:
>     Zone:                  Keytype:      State:    Date of next transition:
>     example.com <http://example.com>            KSK           publish 
>      2014-12-16 23:55:02
>     example.com <http://example.com>            ZSK           active   
>     2015-03-16 09:55:02
> 
>     On 16-12-14 08:54, Emil Natan wrote:
>     > Good morning,
>     >
>     > I have a test environment with ODS 1.4.6 and Keyper HSM where signing
>     > zones was working until I decided to remove all keys and start from scratch.
>     > I removed all keys with "ods-hsmutil purge"\
> 
>     $ sudo ods-hsmutil purge SoftHSM
>     Purging all keys from repository: SoftHSM
>     2 keys found.
> 
>     Are you sure you want to remove ALL keys from repository SoftHSM ?
>     (YES/NO) YES
> 
>     Starting purge...
>     Key remove successful: 816416e1255a1724021895b531c0e313
>     Key remove successful: 615ef6c218cc6bc6d714a0742a07617b
>     Purge done.
> 
> 
>     > reinitialized the HSM\
> 
>     Don't think this is necessary, but okay:
> 
>     $ sudo softhsm --init-token --slot 0 --label "OpenDNSSEC"
>     The SO PIN must have a length between 4 and 255 characters.
>     Enter SO PIN:
>     The user PIN must have a length between 4 and 255 characters.
>     Enter user PIN:
>     The token has been initialized.
> 
> 
>     > removed the single zone I used to sign\
> 
>     $ sudo ods-ksmutil zone delete --zone example.com <http://example.com>
>     zonelist filename set to /etc/opendnssec/zonelist.xml.
>     Zone list updated: 1 removed, 0 added, 0 updated.
> 
> 
>     > reinitialized the database "ods-ksmutil setup"\
> 
>     I think you should first stop the opendnssec service, but I will not do
>     that now:
> 
>     $ sudo ods-ksmutil setup
>     *WARNING* This will erase all data in the database; are you sure?
>     [y/N] y
>     fixing permissions on file /var/opendnssec/kasp.db
>     zonelist filename set to /etc/opendnssec/zonelist.xml.
>     kasp filename set to /etc/opendnssec/kasp.xml.
>     Repository SoftHSM found
>     No Maximum Capacity set.
>     RequireBackup NOT set; please make sure that you know the potential
>     problems of using keys which are not recoverable
>     INFO: The XML in /etc/opendnssec/conf.xml is valid
>     INFO: The XML in /etc/opendnssec/zonelist.xml is valid
>     INFO: The XML in /etc/opendnssec/kasp.xml is valid
>     WARNING: In policy default, Y used in duration field for Keys/KSK
>     Lifetime (P1Y) in /etc/opendnssec/kasp.xml - this will be interpreted as
>     365 days
>     WARNING: In policy lab, Y used in duration field for Keys/KSK Lifetime
>     (P1Y) in /etc/opendnssec/kasp.xml - this will be interpreted as 365 days
>     Policy default found
>     Info: converting P1Y to seconds; M interpreted as 31 days, Y interpreted
>     as 365 days
>     Policy lab found
>     Info: converting P1Y to seconds; M interpreted as 31 days, Y interpreted
>     as 365 days
> 
> 
>     > pregenerated new keys\
> 
>     But you have no zones currently (you removed the single zone)?
> 
>     $ sudo ods-ksmutil key generate --policy default --interval P1Y
>     Key sharing is Off
>     Info: converting P1Y to seconds; M interpreted as 31 days, Y interpreted
>     as 365 days
>     HSM opened successfully.
>     Info: 0 zone(s) found on policy "default"
>     No zones on policy default, skipping...
> 
> 
>     > added a zone\
> 
>     $ sudo ods-ksmutil zone add --zone example.com <http://example.com>
>     zonelist filename set to /etc/opendnssec/zonelist.xml.
>     Imported zone: example.com <http://example.com>
> 
> 
>     > updated, restarted all services.
> 
>     $ sudo ods-control stop
>     Stopping enforcer...
>     Stopping signer engine...
>     Engine shut down.
> 
>     $ sudo ods-control start
>     Starting enforcer...
>     OpenDNSSEC ods-enforcerd started (version 1.4.6), pid 28343
>     Starting signer engine...
>     OpenDNSSEC signer engine version 1.4.6
>     Engine running.
> 
>     > Everything seems to worked well, but the signer does not find one of the
>     > keys to sign the zone, more specifically the KSK. I went the above
>     > process few times, always ending with:
>     >
>     > Dec 16 09:40:27 debugsigner002 ods-signerd: [hsm] unable to get key: key
>     > f81e4b2cb33eec780320b6ceeb6f6bb8 not found
>     > Dec 16 09:40:27 debugsigner002 ods-signerd: [zone] unable to publish
>     > dnskeys for zone XXX: error creating dnskey
>     > Dec 16 09:40:27 debugsigner002 ods-signerd: [tools] unable to read zone
>     > XXX: failed to publish dnskeys (General error)
>     > Dec 16 09:40:27 debugsigner002 ods-signerd: [worker[4]] CRITICAL: failed
>     > to sign zone XXX: General error
> 
>     For me, it finds the old key in the
>     `/var/opendnssec/tmp/example.com.backup2` file and decides it is
>     corrupted:
> 
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [hsm] libhsm
>     connection opened succesfully
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [engine] signer
>     started (version 1.4.6), pid 28355
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [hsm] unable to
>     get key: key 615ef6c218cc6bc6d714a0742a07617b not found
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [zone] unable to
>     publish dnskeys for zone example.com <http://example.com>: error
>     creating dnskey
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [zone] corrupted
>     backup file zone example.com <http://example.com>: unable to publish
>     dnskeys (General error)
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [engine] unable to
>     recover zone example.com <http://example.com> from backup,
>     performing full sign
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [signconf] zone
>     example.com <http://example.com> signconf: RESIGN[PT7200S]
>     REFRESH[PT259200S]
>     VALIDITY[PT1209600S] DENIAL[PT1209600S] JITTER[PT43200S] OFFSET[PT3600S]
>     NSEC[50] DNSKEYTTL[PT3600S] SOATTL[PT3600S] MINIMUM[PT3600S]
>     SERIAL[unixtime]
>     Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [STATS]
>     example.com <http://example.com> 1418724479 RR[count=61 time=0(sec)]
>     NSEC3[count=60
>     time=0(sec)] RRSIG[new=112 reused=0 time=0(sec) avg=0(sig/sec)]
>     TOTAL[time=0(sec)]
> 
> 
>     > The key exist in both HSM and database. ods-hsmutil lists it:
>     >
>     > root at debugsigner002:~# ods-hsmutil list | grep
>     > f81e4b2cb33eec780320b6ceeb6f6bb8
>     > Keyper                f81e4b2cb33eec780320b6ceeb6f6bb8  RSA/2048
>     >
>     > ods-ksmutil shows it:
>     >
>     > root at debugsigner002:~# ods-ksmutil key list -v
>     > Keys:
>     > Zone:                           Keytype:      State:    Date of next
>     > transition (to):  Size:   Algorithm:  CKA_ID:
>     > Repository:                       Keytag:
>     > XXX                              KSK           active    2016-01-16
>     > 09:49:45 (retire)   2048    8           f81e4b2cb33eec780320b6ceeb6f6bb8
>     >  Keyper                            6061
>     > XXX                              ZSK           active    2015-04-18
>     > 22:40:55 (retire)   1024    8           d2aa0ba9af0f41429d23ea387abb836a
>     >  Keyper
>     >
>     > external tools - dnssec-keyfromlabel can use it.
>     > No other errors in the log.
>     >
>     > Any ideas what's wrong? Suggestions what else to try?
>     > Thanks.
>     >
>     > Emil
>     >
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > Opendnssec-user mailing list
>     > Opendnssec-user at lists.opendnssec.org
>     <mailto:Opendnssec-user at lists.opendnssec.org>
>     > https://lists.opendnssec.org/mailman/listinfo/opendnssec-user
>     >
> 
>     _______________________________________________
>     Opendnssec-user mailing list
>     Opendnssec-user at lists.opendnssec.org
>     <mailto:Opendnssec-user at lists.opendnssec.org>
>     https://lists.opendnssec.org/mailman/listinfo/opendnssec-user
> 
> 
> 
> _______________________________________________
> Opendnssec-user mailing list
> Opendnssec-user at lists.opendnssec.org
> https://lists.opendnssec.org/mailman/listinfo/opendnssec-user
> 

-- 
Sebastian Castro
Technical Research Manager
.nz Registry Services (New Zealand Domain Name Registry Limited)
desk: +64 4 495 2337
mobile: +64 21 400535



More information about the Opendnssec-user mailing list