[Opendnssec-develop] RE: 1.4 release - issue found with the multi-threaded enforcer

[Sara Dickinson]

All, 

Earlier this week a major issue was uncovered with the multi-threaded enforcer functionality that was added in 1.4. Jerry and Sion have investigated this and after a meeting today we all agreed that the best solution is to remove the multi-threaded code from the 1.4 code base. More details are given below, but obviously this is a significant change to make at this stage and will unfortunately slightly delay the release. However, we are convinced that this is the correct technical decision in the long run. 

We are proposing that the 1.4 release is briefly moved back to beta status while we run tests on the updated code. The code change is straightforward, and effectively reverts the enforcer component in trunk to the 1.3 code base with very minor 1.4 specific changes. As such the basic functionality is not in question, however since the rollback is a manual process we feel a full sanity check of the result is necessary. Once this testing is complete, we anticipate moving to a 1.4.0rc3 asap.

We are currently in the process of code reviewing the required changes and expect to commit them to trunk early next week. I will then email the users & maintainers list (but I will contact CIRA today). I plan to move the team meeting to Thursday next week so the team can review the decision to have a 1.4.0b3 release and the testing plan then.

Regards

Sara.

---------------------------
SUMMARY OF ISSUE:

Problem:
- a scenario has been found where key allocation is incorrectly done for multiple zones when more than 1 worker thread for the enforcer is enabled 
- a fix for this has been investigated, however the conclusion was that any fix would a) further increase the already complex code logic required to support multi-threading and b) largely eliminate any performance gain obtained by using the multi-threaded option (which was also only available with MySQL)
- as a result the mulit-threaded enforcer option cannot be implemented successfully on the 1.4 code base. 
- the threaded code is protected by #ifdef statements, but by default since the start of September all builds have used the new, threaded code with 1 worker thread. 

Solution:
- Whilst simply restricting the number of threads to 1 is an option, this leaves all the complex threading logic in place with no functional value to be gained from it. This code is likely to be harder to maintain and may be more error prone than the well tested, stable 1.3 code base. 
- Therefore the above approach of rolling back the threading changes is preferred