CAcert Regression Test

We are currently building up a regression test system, to automatically test the functionality of the CAcert website.

/!\ Regression tests against the production system must be authorized by at least 2 people from the core-team, and monitored by at least one core-team member. /!\ Regression code that runs against the production system must not be commited to the SVN, or otherwise publizised

There is a new regression testing system that we should try out: Selenium

Where to start?

You can get our current regression tests from a subversion repostory at: http://svn.cacert.org/CAcert/Regression/

A live site running the tests software is available at http://test.chost.de/Regression/ - username and password is both cacert.

The tests are currently not testing the main CAcert website but https://www.test1.cacert.at/

Several tests have been written so far. The testing framework we use is SimpleTest. Basically a test is a script simulating the user/admin/etc input to CAcert web site and checking the results.

Please protect your test installation and framework from indexing by a search engine. You can do this by using a "robots.txt" in your web directory.

Tests Already Implemented

general-website.php

login.php

lostpassword.php

join.php

newemail.php

change-details-password.php

additional tests

Planned Tests

The following scenarios should be covered by the Regression Test system. If you start working on a scenario, please add your name besides the scenario on this page, to make sure, that somebody else isn't doing the same thing at the same time.

Please see UseCases

Organisation account test

History

Requirements

At the beginning, we started collecting the requirements we have for a CAcert regression-test framework:

Requirements:

Frameworks evaluated

Then we evaluated the following software packages:

SimpleTest

This is the currently used framework.

FunkLoad

Looks good. Python based

http://www.bash.org/

Bash. It´s just a shell, and not very useful for regression testing

HTTP::ApacheBench

Not HTTPS capable

CGI::Test

Only works for CGI, forget it

Puffin

Needs python module xml.dom.ext

JMeter

More for load/stress testing

Test::Builder, WWW::Mechanize, Perl

Junit+HTTPUnit

Didn´t work properly

TestNG

Not tested yet

AutoIt

Great for MS Windows Automation

RegressionTest (last edited 2009-09-13 00:29:19 by PhilippGuehring)