Software/Assessment/Documentation/EmergencyPatches hier beschreiben...


Software Development Emergency Patches

This page documents the proposed CAcert Emergency Patches strategy.

The current strategy of Software Development Update Cycle and git repository handling with bug branches allows us to process an emergency patch fast, bypassing other patches under testing and deployment.

Thus we have 3 emergency patch strategies in the following order of escalation:

  1. Emergency patches fast path thru the regular software development update cycle (preferred choice)
  2. Critical sysadmin applies a patch to the critical system given by a Software-Assessor x1

  3. Critical sysadmin gives remote access to a software-assessor or software-engineer with critical admin control x1

x1 this requires an arbitration, the dispute has to be filed for review after the emergency patch has happened

1. Emergency patches fast path strategy

This procedure is designed to quickly fix an emergency bug within a 24 hours period. It requires a working developers, Software-Assessors, testers and critical sysadmins team, so that when the event occurs, all parties can be contacted, all parties are doing their job and the patch passes the proposed Software Development Update Cycle within 24 hours (including documentation).

The teams

The involved teams and the number of team members required for the process

The process

  1. A "critical" issue has been identified (critical admins or software-assessors declare it as an emergency issue)
  2. One of the parties informs all teams about a critical issue by email, irc, phone or whatever is appropriate to contact the teams
  3. Each team reports back the person who will respond to the next working step to the initiator of the call with the information how to contact the person (by phone? by email?).
  4. The team contact goes into the standby mode.
  5. A patch is prepared by a software-assessor, a software-developer, a software-engineer
  6. The patch is reviewed by the 1st Software-Assessor and transferred to the cacert-devel repository and the testserver
  7. The testers team checks the patch and reports to the bug tracker
  8. A 2nd Software-Assessor reviews the patch the 2nd time and forwards the patch to the critical team
  9. Critical sysadmin deploys the patch to the critical system

all within approx 24 hours

2. Emergency patch application to critical system by critical sysadmin

If option 1 (Emergency patches fast path strategy) isn't fit for handling the current emergency issue or some of the required teams under option 1 cannot be reached, option 2 comes into place:

The teams

The involved teams and the number of team members required for the process

The process

  1. A "critical" issue has been identified (critical admin or software-assessor declares it as an emergency issue)
  2. A patch is prepared by a software-assessor, locally tested, transferred to a critical admin
  3. Critical sysadmin deploys the patch to the critical system
  4. Critical sysadmin documents the emergency patch procedure, files a dispute about an emergency issue handling under option 2
  5. The patch is entered into the update cycle process with the push replication of the critical system repository to the git repository under Software-Assessment team control
  6. The emergency patch process is reviewed by arbitration

3. Emergency patch appliance to critical system thru remote access under critical sysadmin control

If option 2 (Emergency patch applying to critical system by critical sysadmin) doesn't solve the problem or a software-engineer needs to analyse the "live" problem to identify the current emergency issue, option 3 comes into place:

The teams

The involved teams and the count of team members required for the process

The process

  1. A "critical" issue has been identified (critical admin or software-assessor declares it as an emergency issue)
  2. A critical admin and a software-assessor declare the current issue as unsolvable thru option 1 and option 2 without further detailed analysis at the critical system
  3. The critical admin (now in the role as an access-engineer) installs a remote access connection for the software-assessor for a onetime only connection
  4. The software-assessor or software-engineer connects thru the onetime-only connection onto a console, that is under permanent watch by the critical sysadmin
  5. The software-assessor or software-engineer tries to fix the problem
  6. The critical sysadmin terminates and disables the connection from the software-assessor or software-engineer
  7. The critical sysadmin documents the emergency access process, files a dispute about an emergency issue handling under option 3
  8. A patch will be entered into the update cycle process with the push replication of the critical system repository to the git repository under Software-Assessment team control 1.The emergency patch/access process will be reviewed by arbitration

4. Emergency patch logging

Actual use of the emergency patch procedure can be observed through SystemAdministration/EmergencyLogs


Software/Assessment/Documentation/EmergencyPatches (last edited 2015-01-19 09:55:39 by WytzevanderRaay)