View source for Manual:PHP unit testing/Writing unit tests

<languages/>
{{Manual:PHP unit testing/navigation}}
<translate>
== General advice == <!--T:79-->
</translate>
<translate><!--T:32--> The [<tvar name=url>https://docs.phpunit.de/en/9.6/index.html</tvar> PHPUnit Manual] provides good instructions for understanding and developing unit tests.</translate>
<translate><!--T:33--> Pay particularly close attention to the sections on [<tvar name=url>https://docs.phpunit.de/en/9.6/writing-tests-for-phpunit.html</tvar> writing] and [<tvar name=url2>https://docs.phpunit.de/en/9.6/organizing-tests.html</tvar> organizing] tests.</translate>

<translate>
=== Developing best practices === <!--T:20-->

<!--T:34-->
Developers new to unit testing in MediaWiki should use <tvar name=1>{{git file|project=mediawiki/core|branch=HEAD|file=tests/phpunit/includes/SampleTest.php|text=SampleTest.php}}</tvar> as a starting point – it contains helpful comments that will ease the process of learning to write unit tests.

<!--T:35-->
Another resource is the [<tvar name=url>https://web.archive.org/web/20160303184343/http://assets.en.oreilly.com/1/event/45/PHPUnit%20Best%20Practices%20Presentation.pdf</tvar> slides] from the [<tvar name=url2>https://web.archive.org/web/20150925211211/http://www.oscon.com/oscon2010/public/schedule/detail/12510</tvar> PHPUnit Best Practices] talk that Sebastian Bergmann gave at OSCON 2010.

<!--T:21-->
Developers should avoid inventing new conventions or borrowing conventions from other frameworks; using the already well-established PHPUnit conventions will serve to make MediaWiki's tests both useful and usable.

<!--T:40-->
Unit tests should follow ''[<tvar name=rules>http://www.artima.com/weblogs/viewpost.jsp?thread=126923</tvar> A Set of Unit Testing Rules]'' by Michael Feathers.

=== Write testable code === <!--T:1-->
</translate>
{{ll|Manual:Writing testable PHP code|2=<translate><!--T:2--> Please try to write testable code.</translate>}}

<translate><!--T:3--> MediaWiki was not written with the objective of being testable.</translate>
<translate><!--T:4--> It uses global variables all over the place and static methods in many places.</translate>
<translate><!--T:5--> This is a legacy that we have to accept, but try not to introduce these things into new code, and [[<tvar name=1>Requests for comment/Configuration database</tvar>|try to change going forward]].</translate>

<translate><!--T:36--> A good resource might be Miško Hevery's [<tvar name=url>https://web.archive.org/web/20200219224539/http://misko.hevery.com/2009/03/09/guide-to-testability-is-now-downloadable/</tvar> Guide to Testability].</translate>
<translate><!--T:37--> ([<tvar name=url>https://web.archive.org/web/20200829024409/http://misko.hevery.com/about/</tvar> Miško Hevery] is [one of?] Google's Agile Coaches.)</translate>

<translate>
=== Test conventions === <!--T:6-->
</translate>
<translate><!--T:38--> The filename must end with <tvar name=1><code>Test.php</code></tvar>.</translate>
<translate><!--T:39--> Use <tvar name=1>{{git file|project=mediawiki/core|branch=HEAD|file=tests/phpunit/includes/SampleTest.php|text=SampleTest.php}}</tvar> as a starting point.</translate>

<translate>
=== <tvar name="1">setUp()</tvar> and <tvar name="2">tearDown()</tvar> === <!--T:7-->
</translate>
* <translate><!--T:41--> Must be <tvar name=1><code>protected</code></tvar> functions.</translate>
* <translate><!--T:42--> <tvar name=1>tearDown()</tvar> should be in the opposite order of setUp().</translate>
* <translate><!--T:43--> <tvar name=1>setUp()</tvar> starts with calling its parent.</translate>
* <translate><!--T:44--> <tvar name=1>tearDown()</tvar> ends with calling its parent.</translate>

<translate>
==== Assertion functions ==== <!--T:8-->
</translate>
* <translate><!--T:45--> Must be <tvar name=1><code>public</code></tvar> functions.</translate>
* <translate><!--T:46--> The name of the function should be in lowerCamelCase and begin with the word <tvar name=1><code>test</code></tvar>; e.g., <tvar name=2><code>function testFooBar</code></tvar>.</translate>
* <translate><!--T:47--> Whenever possible, refer to the most important method being tested; e.g., <tvar name=1>{{phpi|Html::expandAttributes}}</tvar> is tested in <tvar name=2>{{phpi|HtmlTest::testExpandAttributes}}</tvar>.</translate>

<translate>
==== Data providers ==== <!--T:9-->
</translate>
* <translate><!--T:48--> Must be <tvar name=1><code>public</code></tvar> functions.</translate>
* <translate><!--T:49--> The name of the data provider should be in lowerCamelCase and begin with the word <tvar name=1><code>provide</code></tvar>; e.g., <tvar name=2><code>provideHtml5InputTypes</code></tvar>.</translate>
* <translate><!--T:50--> Should not instantiate MediaWiki Services, since data providers are called before <tvar name=1><code>setUpBeforeClass</code></tvar>.</translate> <translate><!--T:51--> For example, instead of <tvar name=1>{{phpi|Title::newFromText}}</tvar>, use <tvar name=2><code>new TitleValue</code></tvar> to avoid creating a <tvar name=3><code>TitleParser</code></tvar>.</translate>

<translate>
=== Telling a story === <!--T:10-->
</translate>
<translate><!--T:11--> Test output should tell a story.</translate>
<translate><!--T:52--> The <tvar name=1><code>--testdox</code></tvar> output format is a good way to view this story: a test suite execution is displayed as a set of statements about the test classes, along with whether or not they have passed.</translate>
<translate><!--T:53--> The statements (unless customized) are the test method names with corrected capitalization and spacing.</translate>

<translate><!--T:54--> The <tvar name=1><code>@testdox</code></tvar> annotation can be used to customize the message that is displayed.</translate>
<translate><!--T:55--> It is not currently used in the MediaWiki codebase.</translate>

<translate>
<!--T:56-->
See [<tvar name=url>https://phpunit.readthedocs.io/es/latest/other-uses-for-tests.html</tvar> "Other Uses for Tests"] for more information.

=== Number of assertions === <!--T:12-->

<!--T:13-->
Only one assertion per test unless there is a good reason (expensive tests may need to be grouped).

=== Failing === <!--T:30-->

<!--T:78-->
Usually tests code shouldn't do <tvar name=1>{{phpi|die("Error")}}</tvar>, but use the phpunit fail method:
</translate>

<syntaxhighlight lang="PHP">
$this->fail( 'A useful error message' )
</syntaxhighlight>

<translate>
<!--T:31-->
This would show up as a failure in the testing summary rather than bringing the whole test suite down.

== Specific to MediaWiki == <!--T:80-->

=== Grouping tests === <!--T:14-->
</translate>
<translate><!--T:15--> PHPUnit allows tests to be put into arbitrary groups.</translate>
<translate><!--T:57--> Groups of tests can be selected for execution or excluded from execution when the test suite is run (see the [<tvar name=url>https://phpunit.readthedocs.io/en/8.0/appendixes.annotations.html#appendixes.annotations.group</tvar> @group annotation], [<tvar name=url2>https://phpunit.readthedocs.io/en/8.0/textui.html</tvar> The Command-Line Test Runner] and [<tvar name=url3>https://phpunit.readthedocs.io/en/8.0/appendixes.configuration.html</tvar> XML Configuration File] documentation in the PHPUnit manual for additional details.)</translate>

<translate><!--T:58--> To add a test (or class) to a group, use the <tvar name=1><code>@group</code></tvar> annotation in the docblock preceding the code.</translate>
<translate><!--T:16--> For example:</translate>

<syntaxhighlight lang="php">
/**
 * @group Broken
 * @group Internationalization
 */
class HttpTest extends MediaWikiTestCase {
    ...
</syntaxhighlight>

<translate>
<!--T:17-->
Several functional groups are currently used in MediaWiki unit tests:
</translate>

* '''API''' - <translate><!--T:81--> Tests that exercise the [[<tvar name=1>Special:MyLanguage/API</tvar>|MediaWiki API]].</translate>
* '''Broken''' - <translate><!--T:82--> Put broken tests into group <tvar name=1>Broken</tvar>.</translate> <translate><!--T:83--> Tests in this group will not be run (as is configured in <tvar name=1><code>{{git file|project=mediawiki/core|branch=HEAD|file=tests/phpunit/suite.xml|text=tests/phpunit/suite.xml}}</code></tvar>).</translate>
* '''Database''' - <translate><!--T:84--> Tests that require database connectivity should be put into group <tvar name=1>Database</tvar>.</translate>
: {{note|1=<translate><!--T:85--> This causes temporary tables to be overlayed over the real wiki database, so test cases can perform database operations without changing the actual wiki.</translate>}}
* '''Destructive''' - <translate><!--T:86--> Tests that alter or destroy data should be put into group <tvar name=1>Destructive</tvar>.</translate>
* '''Search''' - <translate><!--T:87--> Tests that use MediaWiki's built-in search put into group <tvar name=1>Search</tvar>.</translate>
* '''SeleniumFramework''' - <translate><!--T:88--> Tests that require [[SeleniumFramework]] to be installed should be put in group <tvar name=1>SeleniumFramework</tvar>.</translate>
* '''Stub''' - <translate><!--T:89--> Put test stubs into group <tvar name=1>Stub</tvar>.</translate> <translate><!--T:90--> Tests in this group will not be run (as is configured in <tvar name=1><code>{{git file|project=mediawiki/core|branch=HEAD|file=tests/phpunit/suite.xml|text=tests/phpunit/suite.xml}}</code></tvar>).</translate>
* '''Sqlite''' - <translate><!--T:91--> Tests that use SQLite should be put into group <tvar name=1>Sqlite</tvar>.</translate>
* '''Standalone''' - <translate><!--T:112--> Very slow tests that should not be run in [[<tvar name=1>Special:MyLanguage/Continuous integration/Tutorials/Adding your extension to the shared gate job</tvar>|the gate]], to help keep tests fast for other gated extensions.</translate>
* '''Upload''' - <translate><!--T:92--> Tests that upload files should be put into group <tvar name=1>Upload</tvar>.</translate>
* '''Utility''' - <translate><!--T:93--> Currently unused by any test.</translate> <translate><!--T:94--> Tests in this group will be not be run (as is configured in <tvar name=1><code>{{git file|project=mediawiki/core|branch=HEAD|file=tests/phpunit/suite.xml|text=tests/phpunit/suite.xml}}</code></tvar>).</translate>

<translate>
<!--T:18-->
In addition, tests may also be grouped based on development team:
</translate>

* <translate><!--T:95--> '''Fundraising'''</translate>
* <translate><!--T:96--> '''EditorEngagement'''</translate>
* <translate><!--T:97--> '''Internationalization'''</translate>
* <translate><!--T:113--> etc.</translate>

<translate><!--T:59--> To test only a particular group, use the <tvar name=1><code>--group</code></tvar> flag from the command line:</translate>

 composer phpunit:entrypoint -- --group Search

<translate><!--T:60--> or if you use the Makefile in <tvar name=1>core/tests/phpunit</tvar>:</translate>

 make FLAGS="--group Search" ''target''

<translate>
<!--T:61-->
where target can be phpunit, safe, etc.

=== Coverage === <!--T:19-->
</translate>
<translate><!--T:62--> The [<tvar name=url>https://phpunit.readthedocs.io/en/8.0/code-coverage-analysis.html</tvar> PHPUnit documentation has a chapter about coverage].</translate>
<translate><!--T:63--> There is a [<tvar name=url>https://doc.wikimedia.org/cover/mediawiki-core/</tvar> coverage report for MediaWiki core] generated twice a day.</translate>
<translate><!--T:64--> As the [<tvar name=url>http://phpunit.de/manual/current/en/appendixes.configuration.html</tvar> <tvar name=1>forceCoversAnnotation</tvar> option] should be in effect the test should be marked with [<tvar name=url2>http://phpunit.de/manual/current/en/appendixes.annotations.html#appendixes.annotations.covers</tvar> @covers annotations] to express which parts of the code the test actually checks (as opposed to code that is just run but whose results are never tested for with assertions).</translate>

<translate>
<!--T:65-->
Note that <tvar name=1><code>@covers</code></tvar> requires fully-qualified class names (unlike <tvar name=2>{{ll|Doxygen|Doxygen}}</tvar> annotations such as <tvar name=3><code>@param</code></tvar>).

=== Class === <!--T:98-->
</translate>
<translate><!--T:99--> You will want to extend one of the MediaWiki test classes.</translate>

<syntaxhighlight lang="php">
class HttpTest extends MediaWikiTestCase {
    ...
</syntaxhighlight>

<translate><!--T:100--> The following are some common MediaWiki test classes to extend.</translate>
<translate><!--T:101--> Bullets at a lower level extend their parent.</translate>

* <code>TestCase</code> - <translate><!--T:102--> PHPUnit's test class</translate>
** <code>MediaWikiUnitTestCase</code> - <translate><!--T:103--> For unit tests of methods with no dependencies, or methods whose dependencies are completely mocked.</translate> <translate><!--T:104--> These should go in their own subfolder called <tvar name=1><code>/unit/</code></tvar>, so that <tvar name=2><code>phpunit:unit</tvar></code></code> works correctly. </translate>
*** <code>HookRunnerTestBase</code> - Tests that all arguments passed into a HookRunner class are passed along to a HookContainer.
** <code>MediaWikiIntegrationTestCase</code> - <translate><!--T:105--> Assists with testing classes that access global variables, methods, services, or a storage backend.</translate> <translate><!--T:106--> Prevents sending actual emails.</translate> <translate><!--T:107--> These should go in their own subfolder called <tvar name=1><code>/integration/</code></tvar>, so that <tvar name=2><code>phpunit:integration</tvar></code></code> works correctly.</translate> <translate><!--T:108--> Can safely test the SQL database if you add <tvar name=1><code>@group Database</code></tvar> to your tests.</translate> Changes to database data are reset at the beginning of each test method.
*** <code>MediaWikiLangTestCase</code> - <translate><!--T:109--> Does some language setup such as <tvar name=1><code>$this->setUserLang( 'en' );</code></tvar> and <tvar name=2><code>$this->setContentLang( 'en' );</code></tvar> and does <tvar name=3><code>$services->getMessageCache()->disable()</code></tvar></translate>
**** <code>ApiTestCase</code> - <translate><!--T:110--> Has some extra methods for testing the <tvar name=4>{{ll|Action API}}</tvar>, such as <tvar name=1><code>doApiRequest()</code></tvar>, <tvar name=2><code>doApiRequestWithToken()</code></tvar>, <tvar name=3><code>buildFauxRequest()</code></tvar>, etc.</translate>
*** <code>MaintenanceBaseTestCase</code> - <translate><!--T:114--> For testing MediaWiki maintenance classes.</translate>
*** <code>SpecialPageTestBase</code> - <translate><!--T:115--> For testing MediaWiki special pages.</translate>

<translate>
=== Databases === <!--T:22-->
</translate>
<translate><!--T:66--> When testing database-dependent code, you should put your test case in the ''<tvar name=1>Database</tvar>'' group (see [[<tvar name=2>#Grouping tests</tvar>|above]]).</translate>
<translate><!--T:67--> That tells <tvar name=3>MediaWikiTestCase</tvar> to setup a <tvar name=1><code>DB_PRIMARY</code></tvar> database connection for you to use in <tvar name=2><code>$this->db</code></tvar>.</translate>
<translate><!--T:68--> Normally, this uses a separate temporary database, with certain limited data prefilled by <tvar name=1><code>addCoreDBData</code></tvar>, including a '<tvar name=2>UTSysop</tvar>' user and a '<tvar name=3>UTPage</tvar>' title.</translate>
<translate><!--T:111--> (The temporary database is created with <tvar name=1><code>[https://dev.mysql.com/doc/refman/8.0/en/create-temporary-table.html CREATE TEMPORARY TABLE]</code></tvar>, and the tables are prefixed with <tvar name=2><code>unittest_</code></tvar>.)</translate>
<translate><!--T:116--> You can force <tvar name=1>PHPUnit</tvar> to not use temporary tables (for example, if you want to step debug and look in the database using a database viewer) by setting the .env variable <tvar name=2><code>PHPUNIT_USE_NORMAL_TABLES=1</code></tvar>.</translate>
<translate><!--T:69--> A test case can add additional data to the database by overriding <tvar name=1><code>addDBData</code></tvar> (which by default does nothing).</translate>

<translate>
<!--T:24-->
Please note that only the table schema is copied over from your actual database, not the existing data in that table.

<!--T:71-->
You can directly test the current contents of the database with <tvar name=1>{{method doclink |class=MediaWikiTestCase |method=assertSelect() |anchor=ac52071ce3aa849874d6531edccb7328a}}</tvar>.
</translate>

<syntaxhighlight lang="php">
$this->assertSelect(
	'test', // Table
	[ 'first_name', 'last_name', 'street' ], // Fields to select
	[], // Conditions
	[ [ 'Jane', 'Doe', 'Broadway' ] ] // Expected values
);
</syntaxhighlight>

<translate>
<!--T:25-->
Here are some examples from extensions that you can look at for reference:
</translate>

* {{git file |project=mediawiki/extensions/BounceHandler |file=tests/phpunit/PruneOldBounceRecordsTest.php |text=PruneOldBounceRecordsTest }} - <translate><!--T:72--> from <tvar name=1>{{ll|Extension:BounceHandler}}</tvar></translate>
* {{git file |project=mediawiki/extensions/PageAssessments |file=tests/phpunit/PageAssessmentsDAOTest.php |text=PageAssessmentsDAOTest }} - <translate><!--T:73--> from <tvar name=1>{{ll|Extension:PageAssessments}}</tvar></translate>

<translate><!--T:74--> Tests that are not in the ''<tvar name=1>Database</tvar>'' group still run against a temporary cloned database (even if they ignore <tvar name=2>{{phpi|$this->db}}</tvar> and instead use e.g. <tvar name=3>{{phpi|wfGetDB()}}</tvar> directly); however, this database is only set up once for the whole test run, and not reset between test runs.</translate>
<translate><!--T:26--> Tests should avoid relying on this safety feature if possible.</translate>

<translate>
=== Maintenance scripts === <!--T:27-->

<!--T:75-->
Test cases for {{<tvar name=1>ll|Manual:Maintenance scripts</tvar>|maintenance scripts}} should inherit from <tvar name=2>{{class doclink |class=MediaWiki\Tests\Maintenance\MaintenanceBaseTestCase }}</tvar>, to handle the different output channels used by maintenance scripts.

<!--T:76-->
The base test case <tvar name=1>{{phpi|setUp()}}</tvar> method will instantiate your <tvar name=2><code>Maintenance</code></tvar> object for you, if you specify the class to construct by providing the mandatory <tvar name=3>{{phpi|getMaintenanceClass()}}</tvar> in your subclass:
</translate>

<syntaxhighlight lang="php" line="1">
    public function getMaintenanceClass() {
        return PurgeScoreCache::class;
    }
</syntaxhighlight>

<translate><!--T:77--> In the unlikely event that you want to do something special to instantiate the class under test, you can override the <tvar name=1>{{phpi|createMaintenance()}}</tvar> method, but hopefully this isn't needed.</translate>

<translate><!--T:28--> By default, the maintenance script output will be suppressed and ignored.</translate>
<translate><!--T:29--> If you wish to test output (this is a good idea), use code like:</translate>

<syntaxhighlight lang="php" line="1">
use MediaWiki\Tests\Maintenance\MaintenanceBaseTestCase;

class PurgeScoreCacheTest extends MaintenanceBaseTestCase {
    public function testNotAThing() {
        $this->maintenance->loadWithArgv( [ '--model', 'not_a_thing' ] );
        $this->maintenance->execute();

        $this->expectOutputRegex( '/skipping \'not_a_thing\' model/' );
    }
}
</syntaxhighlight>

{{Conventions navigation}}