Assurance Quality Assurance (sic!)

Although the system of CAcert Assurance is built to protect against any one failing, there will always be problems. For example, a given Assurer might not know enough or a given assurance might not work out as carefully as normal. It is desirable to find ways to keep the influence and frequency of these anomalies as low as possible.

As CAcert is a heterogeneous community of contributers, there exist different ideas as to how this goal can be achieved. This page collects all the ideas that might arise to ensure the quality of the assurances.

Assurance Challenge

At the moment, a User becomes an Assurer when they have gained 100 points. As this can be done by simply being assured by enough people, this is not a reliable indicator that the new Assurer knows in turn how to conduct the process.

Once a User gets to a certain number of points (100 or close to it), she should sit and pass the AssurerChallenge AssuranceTest. On completion of the AssurerChallenge, and on gaining 100 assurance points, she can become an Assurer.

The AssurerChallenge consists of many multiple-choice questions. The questions should be able to be answered after reading and understanding the AssuranceHandbook2.

Another view on the AssurerChallenge is to consider it less as a test and more as an interactive education program. The potential Assurer is asked a question, s/he tries to answer it and then the correct answer including an explanation why this answer is correct and the other answers are false is shown to him/her. So ideally s/he should develop a feeling of where the problems are and what to consider.

If s/he feels confident after answering several questions correctly s/he may make the "real" test.

The test can not exclude someone who is willing to cheat. It cannot ensure that everyone passing it is a good Assurer. But it can help those who are willing to become ones. And, it sets a minimum standard or baseline, which is a signal well-understood by the world at large.

Experience Points

Assurance and Experience have been mixed up in the past. From the Assurance Policy (POLICY), there will be two systems for each, being Assurance Points and Experience Points. Assurance points are only given for actual assurances done on the individual; Experience points reflect the number of Assurances an Assurer has done.

Delayed Assurance Points

At the moment, every Assurer gets 2 points immediately for each assurance. That means, that the (old) points from 100-150 reflect how many assurances someone already did as a form of experience-level. But if somebody is doing something wrong and nobody corrects the process, the assurer will continue with the bad habits. He will get experienced, but experienced doing it wrong.

Proposal: if you assure somebody with less than 50 points, you won't get immediately 2 points, but 1 point, when this assured person reaches 50 points itself, and another point when the person reaches 100 points. Why? This turns the scale from 100-150 from an pure experience level to a kind of likelihood, that you are a good assurer, because with each point you get, at least one other assurer assured the same person as you.

Complaint Button

If you want to assure somebody, it sometimes happens that the information in the CAcert database disagrees with the information on your own CAP form. Normally, you would then not complete the assurance process for that User. But sometimes the User has already gained sufficient points! This raises questions as to the quality of the assurance process carried out by prior assurers.

Proposal: Surface the mismatch by putting it into the CAcert DB with a kind of "Complaint" button. Some possible effects for each assurer of pressing the big red button could be:

(Or all of them.) An Assurer pressing the "Complaint" button has to be able to prove his complaint with the CAP form of course, if somebody asks. If you divide the "bad-assurer scale" by the number of assurances, you get a kind of quality value for each Assurer.

User Feedback & Evaluation

Sometimes assurances are done in a rushed fashion or sometimes the User knows that something is wrong. It would be useful to draw the User's observations and knowledge into the process.

Proposal: In order to get the points from an assurance, the assured person gives a rating (like 0-4) to the assurer. This could reflect how careful she/he thinks the assurer has checked her/his identity. The average rating of each assurer has an influence (as an factor e.g.), how much points he/she can give. This can even go down to zero. This is also good for newcomers, who should start with zero, so they have to do some "good" assurances before they can issue points. Based on these values, the new assurer can build up a quality ranking as a motivation.

Who can see the feedback? Points could be revealed to the Assurer, comments could be sent to the Quality Assurance department, averages could be posted on the site? How is it processed?

Mentor Assurer

Once the 100 points are gained, there is no real sense that an assurer knows what to do (see Assurer Test above).

Proposal: For each new Assurer, appoint a experienced Senior Assurer as a mentor to guide the Junior Assurer through the first N assurances.

Please add further ideas and pro's or con's.

AssuranceQuality (last edited 2009-10-28 13:21:28 by UlrichSchroeter)