[Opendnssec-develop] 5000 zones - almost possible

[Rickard Bellgrim]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi

Have done some testing with 5000 zones and the default policy + shared keys + NSEC. Using the ods-ksmutil to add zones.

The first 1000 zones are quickly added, but then it gets slower and slower. When you get to the 5000th zone, you can add around 1-2 zone per second.

When I have added the zones, I first start the Signer Engine. This takes around 3 seconds before I can type the next command. And then I start the Enforcer.

The Enforcer creates the KSK and ZSK + standby keys (total 4 keys). And after this, it start creating the zone configurations. But the Enforcer freezes on the first zone with this as the last line in the syslog "Config will be output to /home/rickard/opendnssec/config/1suffix.org.xml". Then after a minute or so, it starts to write the zone configurations.

First, only 10 configurations are created. Probably because the Signer Engine is stealing some CPU. But after a moment more are continuously created.

I am running MySQL, and the ods-ksmutil is very responsive. Even though the Enforcer is working with the zones. Which is great!

I forgot to disable the Auditor. So it was used before outputting the signed zones. This locked all of the worker threads and the auditor just eat up memory and CPU.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
24787 rickard   20   0  108m 105m 1476 R   25 10.4   4:14.19 ods-auditor
24948 rickard   20   0  108m 105m 1476 R   25 10.4   4:09.29 ods-auditor
24810 rickard   20   0  108m 105m 1476 R   25 10.4   4:13.12 ods-auditor
24852 rickard   20   0  108m 105m 1476 R   25 10.4   4:11.65 ods-auditor
24879 rickard   20   0  108m 105m 1476 R   25 10.4   4:11.55 ods-auditor
24902 rickard   20   0  108m 105m 1476 R   25 10.4   4:11.18 ods-auditor
24925 rickard   20   0  108m 105m 1476 R   25 10.4   4:10.52 ods-auditor
24856 rickard   20   0  108m 105m 1476 R   25 10.4   4:11.92 ods-auditor

Stop everything and restart with the Auditor disabled. All of the configurations are created after 40 minutes. And the Signer Engine is not so far behind, only 1 minute.

What I also can see is that ods-enforcerd is eating more and more memory. Memory leaks? From a small program to 526 MB in the memory.

Ok, now we have done the first round of generating zone configurations. Then the second round. It goes through the zones one by one. This takes around 4 minutes. But the memory usage was doubled to 1 GB.

What I can see is that OpenDNSSEC v1.0.0 is not suitable for handling a large number of zone, because of the time scales and memory problems.

Conclusions:
* MySQL makes the ods-ksmutil more responsive.
* The auditor should not be used when signing a large number of zones.
* ods-enforcerd has some problems with memory leaks.
* OpenDNSSEC v1.0.0 cannot be used for signing a large number of zones.

// Rickard

-----BEGIN PGP SIGNATURE-----
Version: 9.8.3 (Build 4028)
Charset: utf-8

wsBVAwUBSvqOiOCjgaNTdVjaAQjfhwf/dY/X59PRODEAWqAAsLlueQV0yGmWkz80
bljjEocK3a9ldhbFCMhBPEhRNReYHEOnHaNY2oQnM8EP1cw3hKMcvtgTRhRlmbvE
SNt2RX2Au8/pqj85bURa0Y0hd5d9CPb+QtOt8zS87znbA1fvKeV8moNfodhpLWSd
87W3brEdVDTGfwhNHFDlmegAA327dGlKxJgY504wollH+bAuyb/1OAgnZXOar++W
gSw1CfeMXKx5B54Z6GskxGZoL0wfL8IBWKpVV71xMGiWXClsUtpGlgTPWhdbuBlJ
R5BT2oCXR1SeevLMrkZyTeoibzfCL0bhztuA3hZ6s2PpUErfL6N8Pw==
=qOOv
-----END PGP SIGNATURE-----


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opendnssec.org/pipermail/opendnssec-develop/attachments/20091111/2c17347a/attachment.htm>