Before: Arbitrator EvaStöwe (A), Respondent: CAcert (R), Claimant: Michael T (C) Ron C (C2), Case: a20140408.1

The dispute of C2 was joined into this case at 2014-04-11 because it was a response to the mass-mail which was send to inform the members based on the ruling.

History Log

Private Part

EOT Private Part

original Dispute

Hi,

Just now the Heartbleed attack was discovered http://heartbleed.com/. This
a) affects some of our infrastructure systems so users of those should
know that they might need to change their credentials
b) probably affects a lot of our server certificate users so they should
be notified how they should react (also because they need to comply with
the CCA and protect their keys, so we should inform them how they can do
that)

I hereby file dispute against CAcert to notify members of the potential
issue.

I accepted DRP and CCA.

second dispute filed by C2 recieved and merged into this case at 2014-04-11

This was a response to the mail send in this case.

This "dispute" resolved itself easily later. The claimant had no plans to file a lawsuit but wanted us to warn that others may file a lawsuit.

No further discovery, action or decision necessary, here.

Discovery

A critical bug within openssl (heartbleed) was discovered.

The critical systems from CAcert were not affected. But some of the infrastructure servers were affected. For those servers the openssl-version and the certificates have to be replaced.

Also a lot of our users could be affected.

As our principles state that we care about the security of our members, CAcert should inform the possibly affected members about

Beside of placing this information internally at many places (different mailing lists, blog, PR-post), this is critical to send mails to the primary email addresses of the members.

This should be done for all possibly affected members, which are all who have issued a server certificate in the time frame where the bug was within openssl.

There are also members who flagged their account with a request to be informed about general topics. This topic is critical enough so that those members also should be informed, as they quite likely at least want to know that our critical systems were not affected.

Ruling

I hereby come to the following ruling:

* Software team should provide a script that allows to send a mail to
all current members, who have either have had an active
server-certificate since 2011-12-01 or have activated the "general
announcement"-flag. (This is not issued since then, but active at this
date or later.)

* The arbitrator or the case manager should take care, that a
mail-template to inform those members about the Heartbleed Bug and in
what regard they may be affected, is provided.

* As soon as both (script and mail-template) are available they should
be send to critical team, which should execute the script to send the
mail to said users.

* Support should be aware, that there may be a lot of mails and returns
because of this action.

2014-04-08, Cologne

Execution

(needs to be updated, mail.sending comleted)

Execution details

Mostly documented in history log.

Further information can be found at:

Further details of emergency action and other links in this context

The script is posted in the following mail, this includes the texts which were send: Mail from cacert-systemlog

Request for executing mail script from software assessor team to critical team

Dear critical,

Please execute the ruling of a20140408.1 and execute the script
scripts/send_heartbleed.php in branch bug-1265 (Revision e5c83c7a2d97b5990bc1ebc5c941638a33233dce).

Cf.
https://git.cacert.org/gitweb/?p=cacert-devel.git;a=blob;f=scripts/send_heartbleed.php;h=6bf0f5f8fee63bbd66cccc50975162f704d5f1f2;hb=e5c83c7a2d97b5990bc1ebc5c941638a33233dce

Report results and status updates in the bugtracker at
https://bugs.cacert.org/view.php?id=1265

Please perform this operation ASAP.

Report from critical team about mail sending

The script has been running from April 9, 10:45 until April 10, 18:37 CEST. 
A total of 168977 messages has been sent out, for a total userid base of 290146 entries.
According to the postfix mail statistics, a total of 170213 e-mails were sent during this period (including regular webdb service mails). For 22414 e-mails out of these delivery problems were reported.
At this moment (April 11, 11:30 CEST) there are still some 3700 e-mails queued for possible delivery later (the regular queue size is more like 50 - 100 e-mails).

For future mass-mailings like this, it is recommended to increase the sending rate from 1 msg/second to 10 msg/second, so the available server resources are used better and the mailing can complete within a number of hours rather than days. We applied this change for this mail shot after 28 hours (during which around 99000 msgs were sent out), and after that, the remaining shot of some 70.000 mails completed within 4 hours.

It's difficult to give an exact percentage, but from browsing through the error messages it looks like spam filtering at the mail server level is only a minor cause (less than 10%) for the delivery failures we've experienced.
Much more common are mail addresses which don't exist anymore, mis-configured mail servers and similar issues. Greylisting has contributed a bit to slowing down the message delivery, but it's not a real problem.

Similiar Cases

none known


Arbitrations/a20140408.1 (last edited 2016-06-30 20:09:19 by EvaStöwe)