[Opendnssec-develop] interaction between the Signer and KASP

[Rick van Rein]

Hello Olaf,

About the data flow diagram I sent on the list yesterday:

> I think I need to understand what the ZoneDB stores.

The zoneDB stores an old AXFR plus a sequence of IXFR updates since
the AXFR version.  To avoid infinitely growing IXFR lists, the "IXFR
to AXFR smasher" process will construct a new AXFR that integrates
the old AXFR and one or more IXFR changes that were made to it.

The "zone publish" process will serve IXFR when asked and when available,
or AXFR otherwise.  It may have to construct an AXFR out of the older
AXFR and the sequence of IXFR following it.  Or it could serve the AXFR
and send a notify about any available IXFR.

The Q4 queue basically drops RRsets as a next IXFR into the zoneDB,
and commits the add as soon as a SOA has been added (the SOA being
the end-marker of a change to the data).

> In your picture I do not see any lines where when an IXFR is processed  
> information is collected from the ZoneDB while deletion or additions  
> of RRsets have impact on e.g. NSEC[3] RRs that exist in the database.  
> Those will need to be maintained.

The idea is that the lines marked "Diff" also support DNS queries
(I dubbed that "state-of-the-art DNS information in the text) that
comes from the zoneDB, unless one of the queues through which the
DNS queries travels has an answer.

Each process that modifies data (like "RRset signer") may remove records
(like RRSIG) from the RRset when responding to a "state-of-the-art DNS query"
in the reverse direction of the "Diff" arrows.  The result is that each
queue responds with the same kind of data that is also put into it;
for example, Q3 will contain (and respond with) NSEC3 data but Q2 has
no NSEC3 records in it.  So "state-of-the-art DNS info" requested
trough Q2 will not surrender NSEC3, but going through Q3 will.  If NSEC3
data is passed back from Q3 to Q2, the "add NSEC(3)" process will strip
that part because it is confusing the Q2 and any process before it.

It may well be that I have used clumsy terms, I couldn't find any better
when I prepared this documentation.  Suggestions for other terms are
welcomed.

> >This means that there is a transaction for each domain being signed,
> >maintained upon entry in the zoneDB.  Or in other words, data is
> >being collected in a pre-publishing file and later moved into the
> >publication area if zoneDB is implemented in files.
> 
> I do not understand the above paragraph.

The "Diff" arrows contain small bits of data to be signed, namely RRsets.
The zoneDB holds larger units, namely the chunksize that gets a serial
number and can be treated externally as a published version.

The zoneDB collects the RRsets when they drip in over the "Diff" arrow
from Q4, and calls it a complete version when the sequence of RRsets is
ended with the SOA end-marker.  As soon as the SOA comes in for the
zone, the zoneDB knows that a new version is complete, and the IXFR
for it moves to the publication area (or: the transaction representing
the version's entry in the zoneDB is committed).

> A ZoneDB implementation means that you have to do the NSEC3 generation  
> by seeking appropriate entries in the file.

As soon as a new NSEC3PARAM arrives in the "add NSEC(3)" process, it will
iterate all the names in the current file version.  Thanks to the
"state-of-the-art" DNS query mechanism, that includes any RRsets that may
just have been changed for the zone.  While iterating, all the hashes of
all the names are created and ordered to know which holes are to be
signed.

I think it is good to iterate over the domain because it makes the complexity
of ordering the zone (or: finding its holes) O(N*log(N)) -- using heapsort.
If NSEC3 were to develop along with single insertions and removals, the
creation of NSEC3 would end up costing O(N*N) as insertion-sort would be
the mechanism used.  N is the zone size, so that would be a problem for
the larger zones.

> >1a. Stuff comes in as an IXFR or AXFR, or
> >1b. Stuff is created in response to internal needs
> 
> I think I need to understand "internal needs". Internal to what?

1. DNSKEY rollover
2. NSEC3PARAM rollover (very rarely)
3. zone resigning

> There is not a one-to-one relation between NSEC3s and a processed  
> RRset. Besides this suggest SOA number increase with each RRset (or I  
> am misreading here, which is highly likely).

SOA serial numbers are increased when a SOA record arrives, and RRsets
are collected until that happens.  The sequence of RRsets, including
the SOA, form a new IXFR for the zone.

NSEC3 is not related to RR names, but to holes between them.  So if a
new RR name is introduced, two holes are created at its sides, and the old
one into which the new name falls is removed.  Similarly, if an RR name
is removed, its neighbouring two holes are removed and a single new one
is created to span the total range that used to be covered by the old
two holes.

It is immaterial AFAIK that a hash is applied to the name -- it merely
means that the order is different from what it would be if an alphebatical
ordering of names took place.  But there still is a highly deterministic
order that does not change if the NSEC3PARAM remains the same.


Hope this helps -- and otherwise keep asking!


Cheers,
 -Rick


P.S.

> NB: The street at which our offices are located has been
> renamed to the above.

I found that Google and bus drivers are not aware of this; it may be
helpful to mention Kruislaan for a bit longer.