Refactor parser tests
authorTim Starling <tstarling@wikimedia.org>
Thu, 8 Sep 2016 01:25:22 +0000 (11:25 +1000)
committerTim Starling <tstarling@wikimedia.org>
Mon, 12 Sep 2016 06:11:42 +0000 (16:11 +1000)
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.

Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.

General:

* Split out the pre-Setup.php global variable configuration from
  phpunit.php into a new class called TestSetup, also called it from
  parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
  Maintenance.
* In Setup.php improved "caches" debug output.

PHPUnit frontend:

* Delete the entire contents of NewParserTest and replace it with a
  small wrapper around ParserTestRunner. It doesn't inherit from
  MediaWikiTestCase anymore since integrating the setup code was an
  unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
  instantiable TestSuite class instead of just a static method. Got rid
  of the eval(), just construct TestCase objects directly with a
  specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
  support them anymore. Note that --filter now works just as well as
  --regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
  for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
  once, as is implied by the documentation.

parserTests.php frontend:

* Made parserTests.php into a Maintenance subclass, moved CLI-specific
  code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
  backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
  to the old default behaviour

Test file reading and interpretation:

* Rename TestFileIterator to TestFileReader, and make it read and buffer
  an entire file, instead of iterating.
* The previous code had an associative array representation of test
  specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
  unnecessary now that all extensions use ParserFirstCallInit. Resurrect
  an old interpretation of the feature which was accidentally broken: if
  a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
  human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
  articles in them. This greatly speeds up execution time when --regex
  matches a small number of tests. It may possibly break extensions, but
  they would have been randomly broken anyway since there is no
  guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
  link formatting, life is complicated enough already. It can go in
  OutputPageTest if that's a thing we really need.

Result recording and display:

* Make TestRecorder into a generic plugin interface for progress output
  etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
  a long inheritance chain. All test recorders now directly inherit from
  TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
  frontend. Most events are ignored since they are never emitted in the
  PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.

Setup and teardown:

* Introduce a new API for setup/teardown where setup functions return a
  ScopedCallback object which automatically performs the corresponding
  teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
  here which was simply copied from Setup.php without review, and had
  nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
  performance, give staticSetup() precedence in cases where they were
  both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
  from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
  don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
  I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
  customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
  handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
  now! Also, the relevant parser tests were removed from the Math
  extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
  allows setup/teardown to be done once for each batch of articles
  instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
  copied in from Setup.php in 2010, and are redundant since we do
  actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
  this alone.

Fuzz tests:

* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
  where they used to reside.

Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a

30 files changed:
includes/Hooks.php
includes/Setup.php
maintenance/Maintenance.php
maintenance/checkLess.php
tests/common/TestSetup.php [new file with mode: 0644]
tests/common/TestsAutoLoader.php
tests/parser/DbTestPreviewer.php
tests/parser/DbTestRecorder.php
tests/parser/DelayedParserTest.php [deleted file]
tests/parser/ITestRecorder.php [deleted file]
tests/parser/MultiTestRecorder.php [new file with mode: 0644]
tests/parser/ParserTestPrinter.php [new file with mode: 0644]
tests/parser/ParserTestResult.php
tests/parser/ParserTestRunner.php
tests/parser/PhpunitTestRecorder.php [new file with mode: 0644]
tests/parser/README
tests/parser/TestFileDataProvider.php [deleted file]
tests/parser/TestFileReader.php
tests/parser/TestRecorder.php
tests/parser/fuzzTest.php
tests/parser/parserTests.php
tests/parser/parserTests.txt
tests/phpunit/Makefile
tests/phpunit/MediaWikiTestCase.php
tests/phpunit/includes/parser/ParserIntegrationTest.php
tests/phpunit/phpunit.php
tests/phpunit/suite.xml
tests/phpunit/suites/CoreParserTestSuite.php [new file with mode: 0644]
tests/phpunit/suites/ParserTestFileSuite.php [new file with mode: 0644]
tests/phpunit/suites/ParserTestTopLevelSuite.php

index b6c194c..511781d 100644 (file)
@@ -64,7 +64,7 @@ class Hooks {
         * @throws MWException If not in testing mode.
         */
        public static function clear( $name ) {
-               if ( !defined( 'MW_PHPUNIT_TEST' ) ) {
+               if ( !defined( 'MW_PHPUNIT_TEST' ) && !defined( 'MW_PARSER_TEST' ) ) {
                        throw new MWException( 'Cannot reset hooks in operation.' );
                }
 
index 97cba25..2f462b8 100644 (file)
@@ -674,7 +674,7 @@ $parserMemc = wfGetParserCacheStorage();
 
 wfDebugLog( 'caches',
        'cluster: ' . get_class( $wgMemc ) .
-       ', WAN: ' . $wgMainWANCache .
+       ', WAN: ' . ( $wgMainWANCache === CACHE_NONE ? 'CACHE_NONE' : $wgMainWANCache ) .
        ', stash: ' . $wgMainStash .
        ', message: ' . get_class( $messageMemc ) .
        ', parser: ' . get_class( $parserMemc ) .
index 6e1f741..2216de1 100644 (file)
@@ -1488,6 +1488,14 @@ abstract class Maintenance {
 
                return fgets( STDIN, 1024 );
        }
+
+       /**
+        * Call this to set up the autoloader to allow classes to be used from the
+        * tests directory.
+        */
+       public static function requireTestsAutoloader() {
+               require_once __DIR__ . '/../tests/common/TestsAutoLoader.php';
+       }
 }
 
 /**
index df1868e..8416c8a 100644 (file)
@@ -40,7 +40,7 @@ class CheckLess extends Maintenance {
                // NOTE (phuedx, 2014-03-26) wgAutoloadClasses isn't set up
                // by either of the dependencies at the top of the file, so
                // require it here.
-               require_once __DIR__ . '/../tests/common/TestsAutoLoader.php';
+               self::requireTestsAutoloader();
 
                // If phpunit isn't available by autoloader try pulling it in
                if ( !class_exists( 'PHPUnit_Framework_TestCase' ) ) {
diff --git a/tests/common/TestSetup.php b/tests/common/TestSetup.php
new file mode 100644 (file)
index 0000000..6c3ad07
--- /dev/null
@@ -0,0 +1,111 @@
+<?php
+
+use MediaWiki\MediaWikiServices;
+
+/**
+ * Common code for test environment initialisation and teardown
+ */
+class TestSetup {
+       /**
+        * This should be called before Setup.php, e.g. from the finalSetup() method
+        * of a Maintenance subclass
+        */
+       public static function applyInitialConfig() {
+               global $wgMainCacheType, $wgMessageCacheType, $wgParserCacheType, $wgMainWANCache;
+               global $wgMainStash;
+               global $wgLanguageConverterCacheType, $wgUseDatabaseMessages;
+               global $wgLocaltimezone, $wgLocalisationCacheConf;
+               global $wgDevelopmentWarnings;
+               global $wgSessionProviders, $wgSessionPbkdf2Iterations;
+               global $wgJobTypeConf;
+               global $wgAuthManagerConfig, $wgAuth;
+
+               // wfWarn should cause tests to fail
+               $wgDevelopmentWarnings = true;
+
+               // Make sure all caches and stashes are either disabled or use
+               // in-process cache only to prevent tests from using any preconfigured
+               // cache meant for the local wiki from outside the test run.
+               // See also MediaWikiTestCase::run() which mocks CACHE_DB and APC.
+
+               // Disabled in DefaultSettings, override local settings
+               $wgMainWANCache =
+               $wgMainCacheType = CACHE_NONE;
+               // Uses CACHE_ANYTHING in DefaultSettings, use hash instead of db
+               $wgMessageCacheType =
+               $wgParserCacheType =
+               $wgSessionCacheType =
+               $wgLanguageConverterCacheType = 'hash';
+               // Uses db-replicated in DefaultSettings
+               $wgMainStash = 'hash';
+               // Use memory job queue
+               $wgJobTypeConf = [
+                       'default' => [ 'class' => 'JobQueueMemory', 'order' => 'fifo' ],
+               ];
+
+               $wgUseDatabaseMessages = false; # Set for future resets
+
+               // Assume UTC for testing purposes
+               $wgLocaltimezone = 'UTC';
+
+               $wgLocalisationCacheConf['storeClass'] = 'LCStoreNull';
+
+               // Generic MediaWiki\Session\SessionManager configuration for tests
+               // We use CookieSessionProvider because things might be expecting
+               // cookies to show up in a FauxRequest somewhere.
+               $wgSessionProviders = [
+                       [
+                               'class' => MediaWiki\Session\CookieSessionProvider::class,
+                               'args' => [ [
+                                       'priority' => 30,
+                                       'callUserSetCookiesHook' => true,
+                               ] ],
+                       ],
+               ];
+
+               // Single-iteration PBKDF2 session secret derivation, for speed.
+               $wgSessionPbkdf2Iterations = 1;
+
+               // Generic AuthManager configuration for testing
+               $wgAuthManagerConfig = [
+                       'preauth' => [],
+                       'primaryauth' => [
+                               [
+                                       'class' => MediaWiki\Auth\TemporaryPasswordPrimaryAuthenticationProvider::class,
+                                       'args' => [ [
+                                               'authoritative' => false,
+                                       ] ],
+                               ],
+                               [
+                                       'class' => MediaWiki\Auth\LocalPasswordPrimaryAuthenticationProvider::class,
+                                       'args' => [ [
+                                               'authoritative' => true,
+                                       ] ],
+                               ],
+                       ],
+                       'secondaryauth' => [],
+               ];
+               $wgAuth = new MediaWiki\Auth\AuthManagerAuthPlugin();
+
+               // Bug 44192 Do not attempt to send a real e-mail
+               Hooks::clear( 'AlternateUserMailer' );
+               Hooks::register(
+                       'AlternateUserMailer',
+                       function () {
+                               return false;
+                       }
+               );
+               // xdebug's default of 100 is too low for MediaWiki
+               ini_set( 'xdebug.max_nesting_level', 1000 );
+
+               // Bug T116683 serialize_precision of 100
+               // may break testing against floating point values
+               // treated with PHP's serialize()
+               ini_set( 'serialize_precision', 17 );
+
+               // TODO: we should call MediaWikiTestCase::prepareServices( new GlobalVarConfig() ) here.
+               // But PHPUnit may not be loaded yet, so we have to wait until just
+               // before PHPUnit_TextUI_Command::main() is executed.
+       }
+
+}
index 7de3394..2a985fe 100644 (file)
  */
 
 global $wgAutoloadClasses;
-$testDir = __DIR__ . '/..';
+$testDir = __DIR__ . "/..";
 
 $wgAutoloadClasses += [
 
+       # tests/common
+       'TestSetup' => "$testDir/common/TestSetup.php",
+
+       # tests/parser
+       'DbTestPreviewer' => "$testDir/parser/DbTestPreviewer.php",
+       'DbTestRecorder' => "$testDir/parser/DbTestRecorder.php",
+       'DjVuSupport' => "$testDir/parser/DjVuSupport.php",
+       'TestRecorder' => "$testDir/parser/TestRecorder.php",
+       'MultiTestRecorder' => "$testDir/parser/MultiTestRecorder.php",
+       'ParserTestRunner' => "$testDir/parser/ParserTestRunner.php",
+       'ParserTestParserHook' => "$testDir/parser/ParserTestParserHook.php",
+       'ParserTestPrinter' => "$testDir/parser/ParserTestPrinter.php",
+       'ParserTestResult' => "$testDir/parser/ParserTestResult.php",
+       'ParserTestResultNormalizer' => "$testDir/parser/ParserTestResultNormalizer.php",
+       'PhpunitTestRecorder' => "$testDir/parser/PhpunitTestRecorder.php",
+       'TestFileReader' => "$testDir/parser/TestFileReader.php",
+       'TestRecorder' => "$testDir/parser/TestRecorder.php",
+       'TidySupport' => "$testDir/parser/TidySupport.php",
+
        # tests/phpunit
        'MediaWikiTestCase' => "$testDir/phpunit/MediaWikiTestCase.php",
        'MediaWikiPHPUnitTestListener' => "$testDir/phpunit/MediaWikiPHPUnitTestListener.php",
@@ -85,6 +104,9 @@ $wgAutoloadClasses += [
        # tests/phpunit/includes/page
        'WikiPageTest' => "$testDir/phpunit/includes/page/WikiPageTest.php",
 
+       # tests/phpunit/includes/parser
+       'ParserIntegrationTest' => "$testDir/phpunit/includes/parser/ParserIntegrationTest.php",
+
        # tests/phpunit/includes/password
        'PasswordTestCase' => "$testDir/phpunit/includes/password/PasswordTestCase.php",
 
@@ -98,6 +120,13 @@ $wgAutoloadClasses += [
        'MediaWiki\\Session\\TestBagOStuff' => "$testDir/phpunit/includes/session/TestBagOStuff.php",
        'MediaWiki\\Session\\TestUtils' => "$testDir/phpunit/includes/session/TestUtils.php",
 
+       # tests/phpunit/includes/site
+       'SiteTest' => "$testDir/phpunit/includes/site/SiteTest.php",
+       'TestSites' => "$testDir/phpunit/includes/site/TestSites.php",
+
+       # tests/phpunit/includes/specialpage
+       'SpecialPageTestHelper' => "$testDir/phpunit/includes/specialpage/SpecialPageTestHelper.php",
+
        # tests/phpunit/includes/specials
        'SpecialPageTestBase' => "$testDir/phpunit/includes/specials/SpecialPageTestBase.php",
        'SpecialPageExecutor' => "$testDir/phpunit/includes/specials/SpecialPageExecutor.php",
@@ -129,29 +158,7 @@ $wgAutoloadClasses += [
                => "$testDir/phpunit/mocks/session/DummySessionBackend.php",
        'DummySessionProvider' => "$testDir/phpunit/mocks/session/DummySessionProvider.php",
 
-       # tests/parser
-       'DbTestPreviewer' => "$testDir/parser/DbTestPreviewer.php",
-       'DbTestRecorder' => "$testDir/parser/DbTestRecorder.php",
-       'DelayedParserTest' => "$testDir/parser/DelayedParserTest.php",
-       'DjVuSupport' => "$testDir/parser/DjVuSupport.php",
-       'ITestRecorder' => "$testDir/parser/ITestRecorder.php",
-       'ParserIntegrationTest' => "$testDir/phpunit/includes/parser/ParserIntegrationTest.php",
-       'ParserTestRunner' => "$testDir/parser/ParserTestRunner.php",
-       'ParserTestParserHook' => "$testDir/parser/ParserTestParserHook.php",
-       'ParserTestResult' => "$testDir/parser/ParserTestResult.php",
-       'ParserTestResultNormalizer' => "$testDir/parser/ParserTestResultNormalizer.php",
-       'TestFileDataProvider' => "$testDir/parser/TestFileDataProvider.php",
-       'TestFileReader' => "$testDir/parser/TestFileReader.php",
-       'TestRecorder' => "$testDir/parser/TestRecorder.php",
-       'TidySupport' => "$testDir/parser/TidySupport.php",
-
-       # tests/phpunit/includes/site
-       'SiteTest' => "$testDir/phpunit/includes/site/SiteTest.php",
-       'TestSites' => "$testDir/phpunit/includes/site/TestSites.php",
-
-       # tests/phpunit/includes/specialpage
-       'SpecialPageTestHelper' => "$testDir/phpunit/includes/specialpage/SpecialPageTestHelper.php",
-
-       # tests/phpunit/suites
+       # tests/suites
+       'ParserTestFileSuite' => "$testDir/phpunit/suites/ParserTestFileSuite.php",
        'ParserTestTopLevelSuite' => "$testDir/phpunit/suites/ParserTestTopLevelSuite.php",
 ];
index 2412254..7809ab3 100644 (file)
@@ -20,6 +20,7 @@
  */
 
 class DbTestPreviewer extends TestRecorder {
+       protected $filter; // /< Test name filter callback
        protected $lb; // /< Database load balancer
        protected $db; // /< Database connection to the main DB
        protected $curRun; // /< run ID number for the current run
@@ -28,14 +29,10 @@ class DbTestPreviewer extends TestRecorder {
 
        /**
         * This should be called before the table prefix is changed
-        * @param TestRecorder $parent
         */
-       function __construct( $parent ) {
-               parent::__construct( $parent );
-
-               $this->lb = wfGetLBFactory()->newMainLB();
-               // This connection will have the wiki's table prefix, not parsertest_
-               $this->db = $this->lb->getConnection( DB_MASTER );
+       function __construct( $db, $filter = false ) {
+               $this->db = $db;
+               $this->filter = $filter;
        }
 
        /**
@@ -43,8 +40,6 @@ class DbTestPreviewer extends TestRecorder {
         * and all that fun stuff
         */
        function start() {
-               parent::start();
-
                if ( !$this->db->tableExists( 'testrun', __METHOD__ )
                        || !$this->db->tableExists( 'testitem', __METHOD__ )
                ) {
@@ -58,17 +53,8 @@ class DbTestPreviewer extends TestRecorder {
                $this->results = [];
        }
 
-       function getName( $test, $subtest ) {
-               if ( $subtest ) {
-                       return "$test subtest #$subtest";
-               } else {
-                       return $test;
-               }
-       }
-
-       function record( $test, $subtest, $result ) {
-               parent::record( $test, $subtest, $result );
-               $this->results[ $this->getName( $test, $subtest ) ] = $result;
+       function record( $test, ParserTestResult $result ) {
+               $this->results[$test['desc']] = $result->isSuccess() ? 1 : 0;
        }
 
        function report() {
@@ -90,11 +76,10 @@ class DbTestPreviewer extends TestRecorder {
 
                        $res = $this->db->select( 'testitem', [ 'ti_name', 'ti_success' ],
                                [ 'ti_run' => $this->prevRun ], __METHOD__ );
+                       $filter = $this->filter;
 
                        foreach ( $res as $row ) {
-                               if ( !$this->parent->regex
-                                       || preg_match( "/{$this->parent->regex}/i", $row->ti_name )
-                               ) {
+                               if ( !$filter || $filter( $row->ti_name ) ) {
                                        $prevResults[$row->ti_name] = $row->ti_success;
                                }
                        }
@@ -143,7 +128,6 @@ class DbTestPreviewer extends TestRecorder {
                }
 
                print "\n";
-               parent::report();
        }
 
        /**
@@ -216,13 +200,5 @@ class DbTestPreviewer extends TestRecorder {
                        . date( "d-M-Y H:i:s", strtotime( $pre->tr_date ) ) . ", " . $pre->tr_mw_version
                        . " and $postDate";
        }
-
-       /**
-        * Close the DB connection
-        */
-       function end() {
-               $this->lb->closeAll();
-               parent::end();
-       }
 }
 
index 26aef97..0e94301 100644 (file)
  * @ingroup Testing
  */
 
-class DbTestRecorder extends DbTestPreviewer {
+class DbTestRecorder extends TestRecorder {
        public $version;
+       private $db;
+
+       public function __construct( IDatabase $db ) {
+               $this->db = $db;
+       }
 
        /**
         * Set up result recording; insert a record for the run with the date
@@ -37,8 +42,6 @@ class DbTestRecorder extends DbTestPreviewer {
                        echo "OK, resuming.\n";
                }
 
-               parent::start();
-
                $this->db->insert( 'testrun',
                        [
                                'tr_date' => $this->db->timestamp(),
@@ -58,17 +61,15 @@ class DbTestRecorder extends DbTestPreviewer {
        /**
         * Record an individual test item's success or failure to the db
         *
-        * @param string $test
-        * @param bool $result
+        * @param array $test
+        * @param ParserTestResult $result
         */
-       function record( $test, $subtest, $result ) {
-               parent::record( $test, $subtest, $result );
-
+       function record( $test, ParserTestResult $result ) {
                $this->db->insert( 'testitem',
                        [
                                'ti_run' => $this->curRun,
-                               'ti_name' => $this->getName( $test, $subtest ),
-                               'ti_success' => $result ? 1 : 0,
+                               'ti_name' => $test['desc'],
+                               'ti_success' => $result->isSuccess() ? 1 : 0,
                        ],
                        __METHOD__ );
        }
@@ -78,7 +79,6 @@ class DbTestRecorder extends DbTestPreviewer {
         */
        function end() {
                $this->db->commit( __METHOD__ );
-               parent::end();
        }
 }
 
diff --git a/tests/parser/DelayedParserTest.php b/tests/parser/DelayedParserTest.php
deleted file mode 100644 (file)
index f9ece92..0000000
+++ /dev/null
@@ -1,118 +0,0 @@
-<?php
-
-/**
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
- * http://www.gnu.org/copyleft/gpl.html
- *
- * @file
- * @ingroup Testing
- */
-
-/**
- * A class to delay execution of a parser test hooks.
- */
-class DelayedParserTest {
-
-       /** Initialized on construction */
-       private $hooks;
-       private $fnHooks;
-       private $transparentHooks;
-
-       public function __construct() {
-               $this->reset();
-       }
-
-       /**
-        * Init/reset or forgot about the current delayed test.
-        * Call to this will erase any hooks function that were pending.
-        */
-       public function reset() {
-               $this->hooks = [];
-               $this->fnHooks = [];
-               $this->transparentHooks = [];
-       }
-
-       /**
-        * Called whenever we actually want to run the hook.
-        * Should be the case if we found the parserTest is not disabled
-        * @param ParserTestRunner|ParserIntegrationTest $parserTest
-        * @return bool
-        * @throws MWException
-        */
-       public function unleash( &$parserTest ) {
-               if ( !( $parserTest instanceof ParserTestRunner
-                       || $parserTest instanceof ParserIntegrationTest )
-               ) {
-                       throw new MWException( __METHOD__ . " must be passed an instance of " .
-                               "ParserTestRunner or ParserIntegrationTest classes\n" );
-               }
-
-               # Trigger delayed hooks. Any failure will make us abort
-               foreach ( $this->hooks as $hook ) {
-                       $ret = $parserTest->requireHook( $hook );
-                       if ( !$ret ) {
-                               return false;
-                       }
-               }
-
-               # Trigger delayed function hooks. Any failure will make us abort
-               foreach ( $this->fnHooks as $fnHook ) {
-                       $ret = $parserTest->requireFunctionHook( $fnHook );
-                       if ( !$ret ) {
-                               return false;
-                       }
-               }
-
-               # Trigger delayed transparent hooks. Any failure will make us abort
-               foreach ( $this->transparentHooks as $hook ) {
-                       $ret = $parserTest->requireTransparentHook( $hook );
-                       if ( !$ret ) {
-                               return false;
-                       }
-               }
-
-               # Delayed execution was successful.
-               return true;
-       }
-
-       /**
-        * Similar to ParserTestRunner object but does not run anything
-        * Use unleash() to really execute the hook
-        * @param string $hook
-        */
-       public function requireHook( $hook ) {
-               $this->hooks[] = $hook;
-       }
-
-       /**
-        * Similar to ParserTestRunner object but does not run anything
-        * Use unleash() to really execute the hook function
-        * @param string $fnHook
-        */
-       public function requireFunctionHook( $fnHook ) {
-               $this->fnHooks[] = $fnHook;
-       }
-
-       /**
-        * Similar to ParserTestRunner object but does not run anything
-        * Use unleash() to really execute the hook function
-        * @param string $hook
-        */
-       public function requireTransparentHook( $hook ) {
-               $this->transparentHooks[] = $hook;
-       }
-
-}
-
diff --git a/tests/parser/ITestRecorder.php b/tests/parser/ITestRecorder.php
deleted file mode 100644 (file)
index 5a78beb..0000000
+++ /dev/null
@@ -1,61 +0,0 @@
-<?php
-/**
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
- * http://www.gnu.org/copyleft/gpl.html
- *
- * @file
- * @ingroup Testing
- */
-
-/**
- * Interface to record parser test results.
- *
- * The ITestRecorder is a very simple interface to record the result of
- * MediaWiki parser tests. One should call start() before running the
- * full parser tests and end() once all the tests have been finished.
- * After each test, you should use record() to keep track of your tests
- * results. Finally, report() is used to generate a summary of your
- * test run, one could dump it to the console for human consumption or
- * register the result in a database for tracking purposes.
- *
- * @since 1.22
- */
-interface ITestRecorder {
-
-       /**
-        * Called at beginning of the parser test run
-        */
-       public function start();
-
-       /**
-        * Called after each test
-        * @param string $test
-        * @param integer $subtest
-        * @param bool $result
-        */
-       public function record( $test, $subtest, $result );
-
-       /**
-        * Called before finishing the test run
-        */
-       public function report();
-
-       /**
-        * Called at the end of the parser test run
-        */
-       public function end();
-
-}
-
diff --git a/tests/parser/MultiTestRecorder.php b/tests/parser/MultiTestRecorder.php
new file mode 100644 (file)
index 0000000..5fbfecf
--- /dev/null
@@ -0,0 +1,55 @@
+<?php
+
+/**
+ * This is a TestRecorder representing a collection of other TestRecorders.
+ * It proxies calls to all constituent objects.
+ */
+class MultiTestRecorder extends TestRecorder {
+       private $recorders = [];
+
+       public function addRecorder( TestRecorder $recorder ) {
+               $this->recorders[] = $recorder;
+       }
+
+       private function proxy( $funcName, $args ) {
+               foreach ( $this->recorders as $recorder ) {
+                       call_user_func_array( [ $recorder, $funcName ], $args );
+               }
+       }
+
+       public function start() {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function startTest( $test ) {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function startSuite( $path ) {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function endSuite( $path ) {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function record( $test, ParserTestResult $result ) {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function warning( $message ) {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function skipped( $test, $subtest ) {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function report() {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+
+       public function end() {
+               $this->proxy( __FUNCTION__, func_get_args() );
+       }
+}
diff --git a/tests/parser/ParserTestPrinter.php b/tests/parser/ParserTestPrinter.php
new file mode 100644 (file)
index 0000000..cad3a53
--- /dev/null
@@ -0,0 +1,326 @@
+<?php
+/**
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ * http://www.gnu.org/copyleft/gpl.html
+ *
+ * @file
+ * @ingroup Testing
+ */
+
+/**
+ * This is a TestRecorder responsible for printing information about progress,
+ * success and failure to the console. It is specific to the parserTests.php
+ * frontend.
+ */
+class ParserTestPrinter extends TestRecorder {
+       private $total;
+       private $success;
+       private $skipped;
+       private $term;
+       private $showDiffs;
+       private $showProgress;
+       private $showFailure;
+       private $showOutput;
+       private $useDwdiff;
+       private $markWhitespace;
+       private $xmlError;
+
+       function __construct( $term, $options ) {
+               $this->term = $term;
+               $options += [
+                       'showDiffs' => true,
+                       'showProgress' => true,
+                       'showFailure' => true,
+                       'showOutput' => false,
+                       'useDwdiff' => false,
+                       'markWhitespace' => false,
+               ];
+               $this->showDiffs = $options['showDiffs'];
+               $this->showProgress = $options['showProgress'];
+               $this->showFailure = $options['showFailure'];
+               $this->showOutput = $options['showOutput'];
+               $this->useDwdiff = $options['useDwdiff'];
+               $this->markWhitespace = $options['markWhitespace'];
+       }
+
+       public function start() {
+               $this->total = 0;
+               $this->success = 0;
+               $this->skipped = 0;
+       }
+
+       public function startTest( $test ) {
+               if ( $this->showProgress ) {
+                       $this->showTesting( $test['desc'] );
+               }
+       }
+
+       private function showTesting( $desc ) {
+               print "Running test $desc... ";
+       }
+
+       /**
+        * Show "Reading tests from ..."
+        *
+        * @param string $path
+        */
+       public function startSuite( $path ) {
+               print $this->term->color( 1 ) .
+                       "Running parser tests from \"$path\"..." .
+                       $this->term->reset() .
+                       "\n";
+       }
+
+       public function endSuite( $path ) {
+               print "\n";
+       }
+
+       public function record( $test, ParserTestResult $result ) {
+               $this->total++;
+               $this->success += ( $result->isSuccess() ? 1 : 0 );
+
+               if ( $result->isSuccess() ) {
+                       $this->showSuccess( $result );
+               } else {
+                       $this->showFailure( $result );
+               }
+       }
+
+       /**
+        * Print a happy success message.
+        *
+        * @param ParserTestResult $testResult
+        * @return bool
+        */
+       private function showSuccess( ParserTestResult $testResult ) {
+               if ( $this->showProgress ) {
+                       print $this->term->color( '1;32' ) . 'PASSED' . $this->term->reset() . "\n";
+               }
+       }
+
+       /**
+        * Print a failure message and provide some explanatory output
+        * about what went wrong if so configured.
+        *
+        * @param ParserTestResult $testResult
+        * @return bool
+        */
+       private function showFailure( ParserTestResult $testResult ) {
+               if ( $this->showFailure ) {
+                       if ( !$this->showProgress ) {
+                               # In quiet mode we didn't show the 'Testing' message before the
+                               # test, in case it succeeded. Show it now:
+                               $this->showTesting( $testResult->getDescription() );
+                       }
+
+                       print $this->term->color( '31' ) . 'FAILED!' . $this->term->reset() . "\n";
+
+                       if ( $this->showOutput ) {
+                               print "--- Expected ---\n{$testResult->expected}\n";
+                               print "--- Actual ---\n{$testResult->actual}\n";
+                       }
+
+                       if ( $this->showDiffs ) {
+                               print $this->quickDiff( $testResult->expected, $testResult->actual );
+                               if ( !$this->wellFormed( $testResult->actual ) ) {
+                                       print "XML error: $this->xmlError\n";
+                               }
+                       }
+               }
+
+               return false;
+       }
+
+       /**
+        * Run given strings through a diff and return the (colorized) output.
+        * Requires writable /tmp directory and a 'diff' command in the PATH.
+        *
+        * @param string $input
+        * @param string $output
+        * @param string $inFileTail Tailing for the input file name
+        * @param string $outFileTail Tailing for the output file name
+        * @return string
+        */
+       private function quickDiff( $input, $output,
+               $inFileTail = 'expected', $outFileTail = 'actual'
+       ) {
+               if ( $this->markWhitespace ) {
+                       $pairs = [
+                               "\n" => '¶',
+                               ' ' => '·',
+                               "\t" => '→'
+                       ];
+                       $input = strtr( $input, $pairs );
+                       $output = strtr( $output, $pairs );
+               }
+
+               # Windows, or at least the fc utility, is retarded
+               $slash = wfIsWindows() ? '\\' : '/';
+               $prefix = wfTempDir() . "{$slash}mwParser-" . mt_rand();
+
+               $infile = "$prefix-$inFileTail";
+               $this->dumpToFile( $input, $infile );
+
+               $outfile = "$prefix-$outFileTail";
+               $this->dumpToFile( $output, $outfile );
+
+               $shellInfile = wfEscapeShellArg( $infile );
+               $shellOutfile = wfEscapeShellArg( $outfile );
+
+               global $wgDiff3;
+               // we assume that people with diff3 also have usual diff
+               if ( $this->useDwdiff ) {
+                       $shellCommand = 'dwdiff -Pc';
+               } else {
+                       $shellCommand = ( wfIsWindows() && !$wgDiff3 ) ? 'fc' : 'diff -au';
+               }
+
+               $diff = wfShellExec( "$shellCommand $shellInfile $shellOutfile" );
+
+               unlink( $infile );
+               unlink( $outfile );
+
+               if ( $this->useDwdiff ) {
+                       return $diff;
+               } else {
+                       return $this->colorDiff( $diff );
+               }
+       }
+
+       /**
+        * Write the given string to a file, adding a final newline.
+        *
+        * @param string $data
+        * @param string $filename
+        */
+       private function dumpToFile( $data, $filename ) {
+               $file = fopen( $filename, "wt" );
+               fwrite( $file, $data . "\n" );
+               fclose( $file );
+       }
+
+       /**
+        * Colorize unified diff output if set for ANSI color output.
+        * Subtractions are colored blue, additions red.
+        *
+        * @param string $text
+        * @return string
+        */
+       private function colorDiff( $text ) {
+               return preg_replace(
+                       [ '/^(-.*)$/m', '/^(\+.*)$/m' ],
+                       [ $this->term->color( 34 ) . '$1' . $this->term->reset(),
+                               $this->term->color( 31 ) . '$1' . $this->term->reset() ],
+                       $text );
+       }
+
+       private function wellFormed( $text ) {
+               $html =
+                       Sanitizer::hackDocType() .
+                               '<html>' .
+                               $text .
+                               '</html>';
+
+               $parser = xml_parser_create( "UTF-8" );
+
+               # case folding violates XML standard, turn it off
+               xml_parser_set_option( $parser, XML_OPTION_CASE_FOLDING, false );
+
+               if ( !xml_parse( $parser, $html, true ) ) {
+                       $err = xml_error_string( xml_get_error_code( $parser ) );
+                       $position = xml_get_current_byte_index( $parser );
+                       $fragment = $this->extractFragment( $html, $position );
+                       $this->xmlError = "$err at byte $position:\n$fragment";
+                       xml_parser_free( $parser );
+
+                       return false;
+               }
+
+               xml_parser_free( $parser );
+
+               return true;
+       }
+
+       private function extractFragment( $text, $position ) {
+               $start = max( 0, $position - 10 );
+               $before = $position - $start;
+               $fragment = '...' .
+                       $this->term->color( 34 ) .
+                       substr( $text, $start, $before ) .
+                       $this->term->color( 0 ) .
+                       $this->term->color( 31 ) .
+                       $this->term->color( 1 ) .
+                       substr( $text, $position, 1 ) .
+                       $this->term->color( 0 ) .
+                       $this->term->color( 34 ) .
+                       substr( $text, $position + 1, 9 ) .
+                       $this->term->color( 0 ) .
+                       '...';
+               $display = str_replace( "\n", ' ', $fragment );
+               $caret = '   ' .
+                       str_repeat( ' ', $before ) .
+                       $this->term->color( 31 ) .
+                       '^' .
+                       $this->term->color( 0 );
+
+               return "$display\n$caret";
+       }
+
+       /**
+        * Show a warning to the user
+        */
+       public function warning( $message ) {
+               echo "$message\n";
+       }
+
+       /**
+        * Mark a test skipped
+        */
+       public function skipped( $test, $subtest ) {
+               if ( $this->showProgress ) {
+                       print $this->term->color( '1;33' ) . 'SKIPPED' . $this->term->reset() . "\n";
+               }
+               $this->skipped++;
+       }
+
+       public function report() {
+               if ( $this->total > 0 ) {
+                       $this->reportPercentage( $this->success, $this->total );
+               } else {
+                       print $this->term->color( 31 ) . "No tests found." . $this->term->reset() . "\n";
+               }
+       }
+
+       private function reportPercentage( $success, $total ) {
+               $ratio = wfPercent( 100 * $success / $total );
+               print $this->term->color( 1 ) . "Passed $success of $total tests ($ratio)";
+               if ( $this->skipped ) {
+                       print ", skipped {$this->skipped}";
+               }
+               print "... ";
+
+               if ( $success == $total ) {
+                       print $this->term->color( 32 ) . "ALL TESTS PASSED!";
+               } else {
+                       $failed = $total - $success;
+                       print $this->term->color( 31 ) . "$failed tests failed!";
+               }
+
+               print $this->term->reset() . "\n";
+
+               return ( $success == $total );
+       }
+}
+
index a7b3672..6396a01 100644 (file)
  * @since 1.22
  */
 class ParserTestResult {
-       /**
-        * Description of the parser test.
-        *
-        * This is usually the text used to describe a parser test in the .txt
-        * files.  It is initialized on a construction and you most probably
-        * never want to change it.
-        */
-       public $description;
+       /** The test info array */
+       public $test;
        /** Text that was expected */
        public $expected;
        /** Actual text rendered */
        public $actual;
 
        /**
-        * @param string $description A short text describing the parser test
-        *   usually the text in the parser test .txt file.  The description
-        *   is later available using the property $description.
+        * @param array $test The test info array from TestIterator
+        * @param string $expected The normalized expected output
+        * @param string $actual The actual output
         */
-       public function __construct( $description ) {
-               $this->description = $description;
+       public function __construct( $test, $expected, $actual ) {
+               $this->test = $test;
+               $this->expected = $expected;
+               $this->actual = $actual;
        }
 
        /**
@@ -41,4 +37,8 @@ class ParserTestResult {
        public function isSuccess() {
                return $this->expected === $this->actual;
        }
+
+       public function getDescription() {
+               return $this->test['desc'];
+       }
 }
index be3f0f7..ba7f8f8 100644 (file)
@@ -1,8 +1,7 @@
 <?php
 /**
- * Helper code for the MediaWiki parser test suite. Some code is duplicated
- * in PHPUnit's ParserIntegrationTest.php, so you'll probably want to update both
- * at the same time.
+ * Generic backend for the MediaWiki parser test suite, used by both the
+ * standalone parserTests.php and the PHPUnit "parsertests" suite.
  *
  * Copyright © 2004, 2010 Brion Vibber <brion@pobox.com>
  * https://www.mediawiki.org/
@@ -23,7 +22,6 @@
  * http://www.gnu.org/copyleft/gpl.html
  *
  * @todo Make this more independent of the configuration (and if possible the database)
- * @todo document
  * @file
  * @ingroup Testing
  */
@@ -33,25 +31,21 @@ use MediaWiki\MediaWikiServices;
  * @ingroup Testing
  */
 class ParserTestRunner {
-       /**
-        * @var bool $color whereas output should be colorized
-        */
-       private $color;
-
-       /**
-        * @var bool $showOutput Show test output
-        */
-       private $showOutput;
-
        /**
         * @var bool $useTemporaryTables Use temporary tables for the temporary database
         */
        private $useTemporaryTables = true;
 
        /**
-        * @var bool $databaseSetupDone True if the database has been set up
+        * @var array $setupDone The status of each setup function
         */
-       private $databaseSetupDone = false;
+       private $setupDone = [
+               'staticSetup' => false,
+               'perTestSetup' => false,
+               'setupDatabase' => false,
+               'setDatabase' => false,
+               'setupUploads' => false,
+       ];
 
        /**
         * Our connection to the database
@@ -76,189 +70,417 @@ class ParserTestRunner {
        private $tidySupport;
 
        /**
-        * @var ITestRecorder
+        * @var TidyDriverBase
+        */
+       private $tidyDriver = null;
+
+       /**
+        * @var TestRecorder
         */
        private $recorder;
 
+       /**
+        * The upload directory, or null to not set up an upload directory
+        *
+        * @var string|null
+        */
        private $uploadDir = null;
 
-       public $regex = "";
-       private $savedGlobals = [];
-       private $useDwdiff = false;
-       private $markWhitespace = false;
-       private $normalizationFunctions = [];
-
        /**
-        * Sets terminal colorization and diff/quick modes depending on OS and
-        * command-line options (--color and --quick).
-        * @param array $options
+        * The name of the file backend to use, or null to use MockFileBackend.
+        * @var string|null
         */
-       public function __construct( $options = [] ) {
-               # Only colorize output if stdout is a terminal.
-               $this->color = !wfIsWindows() && Maintenance::posix_isatty( 1 );
-
-               if ( isset( $options['color'] ) ) {
-                       switch ( $options['color'] ) {
-                               case 'no':
-                                       $this->color = false;
-                                       break;
-                               case 'yes':
-                               default:
-                                       $this->color = true;
-                                       break;
-                       }
-               }
+       private $fileBackendName;
 
-               $this->term = $this->color
-                       ? new AnsiTermColorer()
-                       : new DummyTermColorer();
+       /**
+        * A complete regex for filtering tests.
+        * @var string
+        */
+       private $regex;
 
-               $this->showDiffs = !isset( $options['quick'] );
-               $this->showProgress = !isset( $options['quiet'] );
-               $this->showFailure = !(
-                       isset( $options['quiet'] )
-                               && ( isset( $options['record'] )
-                               || isset( $options['compare'] ) ) ); // redundant output
+       /**
+        * A list of normalization functions to apply to the expected and actual
+        * output.
+        * @var array
+        */
+       private $normalizationFunctions = [];
 
-               $this->showOutput = isset( $options['show-output'] );
-               $this->useDwdiff = isset( $options['dwdiff'] );
-               $this->markWhitespace = isset( $options['mark-ws'] );
+       /**
+        * @param TestRecorder $recorder
+        * @param array $options
+        */
+       public function __construct( TestRecorder $recorder, $options = [] ) {
+               $this->recorder = $recorder;
 
                if ( isset( $options['norm'] ) ) {
-                       foreach ( explode( ',', $options['norm'] ) as $func ) {
+                       foreach ( $options['norm'] as $func ) {
                                if ( in_array( $func, [ 'removeTbody', 'trimWhitespace' ] ) ) {
                                        $this->normalizationFunctions[] = $func;
                                } else {
-                                       echo "Warning: unknown normalization option \"$func\"\n";
+                                       $this->recorder->warning(
+                                               "Warning: unknown normalization option \"$func\"\n" );
                                }
                        }
                }
 
-               if ( isset( $options['filter'] ) ) {
-                       $options['regex'] = $options['filter'];
-               }
-
-               if ( isset( $options['regex'] ) ) {
-                       if ( isset( $options['record'] ) ) {
-                               echo "Warning: --record cannot be used with --regex, disabling --record\n";
-                               unset( $options['record'] );
-                       }
+               if ( isset( $options['regex'] ) && $options['regex'] !== false ) {
                        $this->regex = $options['regex'];
                } else {
                        # Matches anything
-                       $this->regex = '';
+                       $this->regex = '//';
                }
 
-               $this->setupRecorder( $options );
-               $this->keepUploads = isset( $options['keep-uploads'] );
+               $this->keepUploads = !empty( $options['keep-uploads'] );
 
-               if ( $this->keepUploads ) {
-                       $this->uploadDir = wfTempDir() . '/mwParser-images';
-               } else {
-                       $this->uploadDir = wfTempDir() . "/mwParser-" . mt_rand() . "-images";
-               }
+               $this->fileBackendName = isset( $options['file-backend'] ) ?
+                       $options['file-backend'] : false;
 
-               $this->runDisabled = isset( $options['run-disabled'] );
-               $this->runParsoid = isset( $options['run-parsoid'] );
+               $this->runDisabled = !empty( $options['run-disabled'] );
+               $this->runParsoid = !empty( $options['run-parsoid'] );
 
                $this->djVuSupport = new DjVuSupport();
-               $this->tidySupport = new TidySupport( isset( $options['use-tidy-config'] ) );
+               $this->tidySupport = new TidySupport( !empty( $options['use-tidy-config'] ) );
                if ( !$this->tidySupport->isEnabled() ) {
-                       echo "Warning: tidy is not installed, skipping some tests\n";
+                       $this->recorder->warning(
+                               "Warning: tidy is not installed, skipping some tests\n" );
                }
 
-               $this->hooks = [];
-               $this->functionHooks = [];
-               $this->transparentHooks = [];
-               $this->setUp();
+               if ( isset( $options['upload-dir'] ) ) {
+                       $this->uploadDir = $options['upload-dir'];
+               }
        }
 
-       function setUp() {
-               global $wgParser, $wgParserConf, $IP, $messageMemc, $wgMemc,
-                       $wgUser, $wgLang, $wgOut, $wgRequest, $wgStyleDirectory,
-                       $wgExtraNamespaces, $wgNamespaceAliases, $wgNamespaceProtection, $wgLocalFileRepo,
-                       $wgExtraInterlanguageLinkPrefixes, $wgLocalInterwikis,
-                       $parserMemc, $wgThumbnailScriptPath, $wgScriptPath, $wgResourceBasePath,
-                       $wgArticlePath, $wgScript, $wgStylePath, $wgExtensionAssetsPath,
-                       $wgMainCacheType, $wgMessageCacheType, $wgParserCacheType, $wgLockManagers;
-
-               $wgScriptPath = '';
-               $wgScript = '/index.php';
-               $wgStylePath = '/skins';
-               $wgResourceBasePath = '';
-               $wgExtensionAssetsPath = '/extensions';
-               $wgArticlePath = '/wiki/$1';
-               $wgThumbnailScriptPath = false;
-               $wgLockManagers = [ [
+       public function getRecorder() {
+               return $this->recorder;
+       }
+
+       /**
+        * Do any setup which can be done once for all tests, independent of test
+        * options, except for database setup.
+        *
+        * Public setup functions in this class return a ScopedCallback object. When
+        * this object is destroyed by going out of scope, teardown of the
+        * corresponding test setup is performed.
+        *
+        * Teardown objects may be chained by passing a ScopedCallback from a
+        * previous setup stage as the $nextTeardown parameter. This enforces the
+        * convention that teardown actions are taken in reverse order to the
+        * corresponding setup actions. When $nextTeardown is specified, a
+        * ScopedCallback will be returned which first tears down the current
+        * setup stage, and then tears down the previous setup stage which was
+        * specified by $nextTeardown.
+        *
+        * @param ScopedCallback|null $nextTeardown
+        * @return ScopedCallback
+        */
+       public function staticSetup( $nextTeardown = null ) {
+               // A note on coding style:
+
+               // The general idea here is to keep setup code together with
+               // corresponding teardown code, in a fine-grained manner. We have two
+               // arrays: $setup and $teardown. The code snippets in the $setup array
+               // are executed at the end of the method, before it returns, and the
+               // code snippets in the $teardown array are executed in reverse order
+               // when the ScopedCallback object is consumed.
+
+               // Because it is a common operation to save, set and restore global
+               // variables, we have an additional convention: when the array key of
+               // $setup is a string, the string is taken to be the name of the global
+               // variable, and the element value is taken to be the desired new value.
+
+               // It's acceptable to just do the setup immediately, instead of adding
+               // a closure to $setup, except when the setup action depends on global
+               // variable initialisation being done first. In this case, you have to
+               // append a closure to $setup after the global variable is appended.
+
+               // When you add to setup functions in this class, please keep associated
+               // setup and teardown actions together in the source code, and please
+               // add comments explaining why the setup action is necessary.
+
+               $setup = [];
+               $teardown = [];
+
+               $teardown[] = $this->markSetupDone( 'staticSetup' );
+
+               // Some settings which influence HTML output
+               $setup['wgSitename'] = 'MediaWiki';
+               $setup['wgServer'] = 'http://example.org';
+               $setup['wgServerName'] = 'example.org';
+               $setup['wgScriptPath'] = '';
+               $setup['wgScript'] = '/index.php';
+               $setup['wgResourceBasePath'] = '';
+               $setup['wgStylePath'] = '/skins';
+               $setup['wgExtensionAssetsPath'] = '/extensions';
+               $setup['wgArticlePath'] = '/wiki/$1';
+               $setup['wgActionPaths'] = [];
+               $setup['wgVariantArticlePath'] = false;
+               $setup['wgUploadNavigationUrl'] = false;
+               $setup['wgCapitalLinks'] = true;
+               $setup['wgNoFollowLinks'] = true;
+               $setup['wgNoFollowDomainExceptions'] = [ 'no-nofollow.org' ];
+               $setup['wgExternalLinkTarget'] = false;
+               $setup['wgExperimentalHtmlIds'] = false;
+               $setup['wgLocaltimezone'] = 'UTC';
+               $setup['wgHtml5'] = true;
+               $setup['wgDisableLangConversion'] = false;
+               $setup['wgDisableTitleConversion'] = false;
+
+               // "extra language links"
+               // see https://gerrit.wikimedia.org/r/111390
+               $setup['wgExtraInterlanguageLinkPrefixes'] = [ 'mul' ];
+
+               // All FileRepo changes should be done here by injecting services,
+               // there should be no need to change global variables.
+               RepoGroup::setSingleton( $this->createRepoGroup() );
+               $teardown[] = function () {
+                       RepoGroup::destroySingleton();
+               };
+
+               // Set up null lock managers
+               $setup['wgLockManagers'] = [ [
                        'name' => 'fsLockManager',
-                       'class' => 'FSLockManager',
-                       'lockDirectory' => $this->uploadDir . '/lockdir',
+                       'class' => 'NullLockManager',
                ], [
                        'name' => 'nullLockManager',
                        'class' => 'NullLockManager',
                ] ];
-               $wgLocalFileRepo = [
-                       'class' => 'LocalRepo',
-                       'name' => 'local',
-                       'url' => 'http://example.com/images',
-                       'hashLevels' => 2,
-                       'transformVia404' => false,
-                       'backend' => new FSFileBackend( [
+               $reset = function() {
+                       LockManagerGroup::destroySingletons();
+               };
+               $setup[] = $reset;
+               $teardown[] = $reset;
+
+               // This allows article insertion into the prefixed DB
+               $setup['wgDefaultExternalStore'] = false;
+
+               // This might slightly reduce memory usage
+               $setup['wgAdaptiveMessageCache'] = true;
+
+               // This is essential and overrides disabling of database messages in TestSetup
+               $setup['wgUseDatabaseMessages'] = true;
+               $reset = function () {
+                       MessageCache::destroyInstance();
+               };
+               $setup[] = $reset;
+               $teardown[] = $reset;
+
+               // It's not necessary to actually convert any files
+               $setup['wgSVGConverter'] = 'null';
+               $setup['wgSVGConverters'] = [ 'null' => 'echo "1">$output' ];
+
+               // Fake constant timestamp
+               Hooks::register( 'ParserGetVariableValueTs', 'ParserTestRunner::getFakeTimestamp' );
+               $teardown[] = function () {
+                       Hooks::clear( 'ParserGetVariableValueTs' );
+               };
+
+               $this->appendNamespaceSetup( $setup, $teardown );
+
+               // Set up interwikis and append teardown function
+               $teardown[] = $this->setupInterwikis();
+
+               // This affects title normalization in links. It invalidates
+               // MediaWikiTitleCodec objects.
+               $setup['wgLocalInterwikis'] = [ 'local', 'mi' ];
+               $reset = function () {
+                       $this->resetTitleServices();
+               };
+               $setup[] = $reset;
+               $teardown[] = $reset;
+
+               // Set up a mock MediaHandlerFactory
+               MediaWikiServices::getInstance()->disableService( 'MediaHandlerFactory' );
+               MediaWikiServices::getInstance()->redefineService(
+                       'MediaHandlerFactory',
+                       function() {
+                               return new MockMediaHandlerFactory();
+                       }
+               );
+               $teardown[] = function () {
+                       MediaWikiServices::getInstance()->resetServiceForTesting( 'MediaHandlerFactory' );
+               };
+
+               // SqlBagOStuff broke when using temporary tables on r40209 (bug 15892).
+               // It seems to have been fixed since (r55079?), but regressed at some point before r85701.
+               // This works around it for now...
+               global $wgObjectCaches;
+               $setup['wgObjectCaches'] = [ CACHE_DB => $wgObjectCaches['hash'] ] + $wgObjectCaches;
+               if ( isset( ObjectCache::$instances[CACHE_DB] ) ) {
+                       $savedCache = ObjectCache::$instances[CACHE_DB];
+                       ObjectCache::$instances[CACHE_DB] = new HashBagOStuff;
+                       $teardown[] = function () use ( $savedCache ) {
+                               ObjectCache::$instances[CACHE_DB] = $savedCache;
+                       };
+               }
+
+               $teardown[] = $this->executeSetupSnippets( $setup );
+
+               // Schedule teardown snippets in reverse order
+               return $this->createTeardownObject( $teardown, $nextTeardown );
+       }
+
+       private function appendNamespaceSetup( &$setup, &$teardown ) {
+               // Add a namespace shadowing a interwiki link, to test
+               // proper precedence when resolving links. (bug 51680)
+               $setup['wgExtraNamespaces'] = [
+                       100 => 'MemoryAlpha',
+                       101 => 'MemoryAlpha_talk'
+               ];
+               // Changing wgExtraNamespaces invalidates caches in MWNamespace and
+               // any live Language object, both on setup and teardown
+               $reset = function () {
+                       MWNamespace::getCanonicalNamespaces( true );
+                       $GLOBALS['wgContLang']->resetNamespaces();
+               };
+               $setup[] = $reset;
+               $teardown[] = $reset;
+       }
+
+       /**
+        * Create a RepoGroup object appropriate for the current configuration
+        * @return RepoGroup
+        */
+       protected function createRepoGroup() {
+               if ( $this->uploadDir ) {
+                       if ( $this->fileBackendName ) {
+                               throw new MWException( 'You cannot specify both use-filebackend and upload-dir' );
+                       }
+                       $backend = new FSFileBackend( [
                                'name' => 'local-backend',
                                'wikiId' => wfWikiID(),
-                               'containerPaths' => [
-                                       'local-public' => $this->uploadDir . '/public',
-                                       'local-thumb' => $this->uploadDir . '/thumb',
-                                       'local-temp' => $this->uploadDir . '/temp',
-                                       'local-deleted' => $this->uploadDir . '/deleted',
-                               ]
-                       ] )
-               ];
-               $wgNamespaceProtection[NS_MEDIAWIKI] = 'editinterface';
-               $wgNamespaceAliases['Image'] = NS_FILE;
-               $wgNamespaceAliases['Image_talk'] = NS_FILE_TALK;
-               # add a namespace shadowing a interwiki link, to test
-               # proper precedence when resolving links. (bug 51680)
-               $wgExtraNamespaces[100] = 'MemoryAlpha';
-               $wgExtraNamespaces[101] = 'MemoryAlpha talk';
-
-               // XXX: tests won't run without this (for CACHE_DB)
-               if ( $wgMainCacheType === CACHE_DB ) {
-                       $wgMainCacheType = CACHE_NONE;
-               }
-               if ( $wgMessageCacheType === CACHE_DB ) {
-                       $wgMessageCacheType = CACHE_NONE;
+                               'basePath' => $this->uploadDir
+                       ] );
+               } elseif ( $this->fileBackendName ) {
+                       global $wgFileBackends;
+                       $name = $this->fileBackendName;
+                       $useConfig = false;
+                       foreach ( $wgFileBackends as $conf ) {
+                               if ( $conf['name'] === $name ) {
+                                       $useConfig = $conf;
+                               }
+                       }
+                       if ( $useConfig === false ) {
+                               throw new MWException( "Unable to find file backend \"$name\"" );
+                       }
+                       $useConfig['name'] = 'local-backend'; // swap name
+                       unset( $useConfig['lockManager'] );
+                       unset( $useConfig['fileJournal'] );
+                       $class = $useConfig['class'];
+                       $backend = new $class( $useConfig );
+               } else {
+                       # Replace with a mock. We do not care about generating real
+                       # files on the filesystem, just need to expose the file
+                       # informations.
+                       $backend = new MockFileBackend( [
+                               'name' => 'local-backend',
+                               'wikiId' => wfWikiID()
+                       ] );
                }
-               if ( $wgParserCacheType === CACHE_DB ) {
-                       $wgParserCacheType = CACHE_NONE;
+
+               return new RepoGroup(
+                       [
+                               'class' => 'LocalRepo',
+                               'name' => 'local',
+                               'url' => 'http://example.com/images',
+                               'hashLevels' => 2,
+                               'transformVia404' => false,
+                               'backend' => $backend
+                       ],
+                       []
+               );
+       }
+
+       /**
+        * Execute an array in which elements with integer keys are taken to be
+        * callable objects, and other elements are taken to be global variable
+        * set operations, with the key giving the variable name and the value
+        * giving the new global variable value. A closure is returned which, when
+        * executed, sets the global variables back to the values they had before
+        * this function was called.
+        *
+        * @see staticSetup
+        *
+        * @param array $setup
+        * @return closure
+        */
+       protected function executeSetupSnippets( $setup ) {
+               $saved = [];
+               foreach ( $setup as $name => $value ) {
+                       if ( is_int( $name ) ) {
+                               $value();
+                       } else {
+                               $saved[$name] = isset( $GLOBALS[$name] ) ? $GLOBALS[$name] : null;
+                               $GLOBALS[$name] = $value;
+                       }
                }
+               return function () use ( $saved ) {
+                       $this->executeSetupSnippets( $saved );
+               };
+       }
 
-               DeferredUpdates::clearPendingUpdates();
-               $wgMemc = wfGetMainCache(); // checks $wgMainCacheType
-               $messageMemc = wfGetMessageCacheStorage();
-               $parserMemc = wfGetParserCacheStorage();
-
-               RequestContext::resetMain();
-               $context = new RequestContext;
-               $wgUser = new User;
-               $wgLang = $context->getLanguage();
-               $wgOut = $context->getOutput();
-               $wgRequest = $context->getRequest();
-               $wgParser = new StubObject( 'wgParser', $wgParserConf['class'], [ $wgParserConf ] );
-
-               if ( $wgStyleDirectory === false ) {
-                       $wgStyleDirectory = "$IP/skins";
+       /**
+        * Take a setup array in the same format as the one given to
+        * executeSetupSnippets(), and return a ScopedCallback which, when consumed,
+        * executes the snippets in the setup array in reverse order. This is used
+        * to create "teardown objects" for the public API.
+        *
+        * @see staticSetup
+        *
+        * @param array $teardown The snippet array
+        * @param ScopedCallback|null A ScopedCallback to consume
+        * @return ScopedCallback
+        */
+       protected function createTeardownObject( $teardown, $nextTeardown ) {
+               return new ScopedCallback( function() use ( $teardown, $nextTeardown ) {
+                       // Schedule teardown snippets in reverse order
+                       $teardown = array_reverse( $teardown );
+
+                       $this->executeSetupSnippets( $teardown );
+                       if ( $nextTeardown ) {
+                               ScopedCallback::consume( $nextTeardown );
+                       }
+               } );
+       }
+
+       /**
+        * Set a setupDone flag to indicate that setup has been done, and return
+        * the teardown closure. If the flag was already set, throw an exception.
+        *
+        * @param string $funcName The setup function name
+        * @return closure
+        */
+       protected function markSetupDone( $funcName ) {
+               if ( $this->setupDone[$funcName] ) {
+                       throw new MWException( "$funcName is already done" );
                }
+               $this->setupDone[$funcName] = true;
+               return function () use ( $funcName ) {
+                       wfDebug( "markSetupDone unmarked $funcName" );
+                       $this->setupDone[$funcName] = false;
+               };
+       }
 
-               self::setupInterwikis();
-               $wgLocalInterwikis = [ 'local', 'mi' ];
-               // "extra language links"
-               // see https://gerrit.wikimedia.org/r/111390
-               array_push( $wgExtraInterlanguageLinkPrefixes, 'mul' );
+       /**
+        * Ensure a given setup stage has been done, throw an exception if it has
+        * not.
+        */
+       protected function checkSetupDone( $funcName, $funcName2 = null ) {
+               if ( !$this->setupDone[$funcName]
+                       && ( $funcName === null || !$this->setupDone[$funcName2] )
+               ) {
+                       throw new MWException( "$funcName must be called before calling " .
+                               wfGetCaller() );
+               }
+       }
 
-               // Reset namespace cache
-               MWNamespace::getCanonicalNamespaces( true );
-               Language::factory( 'en' )->resetNamespaces();
+       /**
+        * Determine whether a particular setup function has been run
+        *
+        * @param string $funcName
+        * @return boolean
+        */
+       public function isSetupDone( $funcName ) {
+               return isset( $this->setupDone[$funcName] ) ? $this->setupDone[$funcName] : false;
        }
 
        /**
@@ -269,8 +491,10 @@ class ParserTestRunner {
         * the interwiki cache by using the 'InterwikiLoadPrefix' hook.
         * Since we are not interested in looking up interwikis in the database,
         * the hook completely replace the existing mechanism (hook returns false).
+        *
+        * @return closure for teardown
         */
-       public static function setupInterwikis() {
+       private function setupInterwikis() {
                # Hack: insert a few Wikipedia in-project interwiki prefixes,
                # for testing inter-language links
                Hooks::register( 'InterwikiLoadPrefix', function ( $prefix, &$iwData ) {
@@ -333,20 +557,18 @@ class ParserTestRunner {
                        // We only want to rely on the above fixtures
                        return false;
                } );// hooks::register
-       }
 
-       /**
-        * Remove the hardcoded interwiki lookup table.
-        */
-       public static function tearDownInterwikis() {
-               Hooks::clear( 'InterwikiLoadPrefix' );
+               return function () {
+                       // Tear down
+                       Hooks::clear( 'InterwikiLoadPrefix' );
+               };
        }
 
        /**
         * Reset the Title-related services that need resetting
         * for each test
         */
-       public static function resetTitleServices() {
+       private function resetTitleServices() {
                $services = MediaWikiServices::getInstance();
                $services->resetServiceForTesting( 'TitleFormatter' );
                $services->resetServiceForTesting( 'TitleParser' );
@@ -355,18 +577,6 @@ class ParserTestRunner {
                $services->resetServiceForTesting( 'LinkRendererFactory' );
        }
 
-       public function setupRecorder( $options ) {
-               if ( isset( $options['record'] ) ) {
-                       $this->recorder = new DbTestRecorder( $this );
-                       $this->recorder->version = isset( $options['setversion'] ) ?
-                               $options['setversion'] : SpecialVersion::getVersion();
-               } elseif ( isset( $options['compare'] ) ) {
-                       $this->recorder = new DbTestPreviewer( $this );
-               } else {
-                       $this->recorder = new TestRecorder( $this );
-               }
-       }
-
        /**
         * Remove last character if it is a newline
         * @group utility
@@ -389,50 +599,111 @@ class ParserTestRunner {
         * Prints status updates on stdout and counts up the total
         * number and percentage of passed tests.
         *
+        * Handles all setup and teardown.
+        *
         * @param array $filenames Array of strings
         * @return bool True if passed all tests, false if any tests failed.
         */
        public function runTestsFromFiles( $filenames ) {
                $ok = false;
 
-               // be sure, ParserTestRunner::addArticle has correct language set,
-               // so that system messages gets into the right language cache
-               $GLOBALS['wgLanguageCode'] = 'en';
-               $GLOBALS['wgContLang'] = Language::factory( 'en' );
+               $teardownGuard = $this->staticSetup();
+               $teardownGuard = $this->setupDatabase( $teardownGuard );
+               $teardownGuard = $this->setupUploads( $teardownGuard );
 
                $this->recorder->start();
                try {
-                       $this->setupDatabase();
                        $ok = true;
 
                        foreach ( $filenames as $filename ) {
-                               echo "Running parser tests from: $filename\n";
-                               $tests = new TestFileReader( $filename, $this );
-                               $ok = $this->runTests( $tests ) && $ok;
+                               $testFileInfo = TestFileReader::read( $filename, [
+                                       'runDisabled' => $this->runDisabled,
+                                       'runParsoid' => $this->runParsoid,
+                                       'regex' => $this->regex ] );
+
+                               // Don't start the suite if there are no enabled tests in the file
+                               if ( !$testFileInfo['tests'] ) {
+                                       continue;
+                               }
+
+                               $this->recorder->startSuite( $filename );
+                               $ok = $this->runTests( $testFileInfo ) && $ok;
+                               $this->recorder->endSuite( $filename );
                        }
 
-                       $this->teardownDatabase();
                        $this->recorder->report();
                } catch ( DBError $e ) {
-                       echo $e->getMessage();
+                       $this->recorder->warning( $e->getMessage() );
                }
                $this->recorder->end();
 
+               ScopedCallback::consume( $teardownGuard );
+
                return $ok;
        }
 
-       function runTests( $tests ) {
+       /**
+        * Determine whether the current parser has the hooks registered in it
+        * that are required by a file read by TestFileReader.
+        */
+       public function meetsRequirements( $requirements ) {
+               foreach ( $requirements as $requirement ) {
+                       switch ( $requirement['type'] ) {
+                       case 'hook':
+                               $ok = $this->requireHook( $requirement['name'] );
+                               break;
+                       case 'functionHook':
+                               $ok = $this->requireFunctionHook( $requirement['name'] );
+                               break;
+                       case 'transparentHook':
+                               $ok = $this->requireTransparentHook( $requirement['name'] );
+                               break;
+                       }
+                       if ( !$ok ) {
+                               return false;
+                       }
+               }
+               return true;
+       }
+
+       /**
+        * Run the tests from a single file. staticSetup() and setupDatabase()
+        * must have been called already.
+        *
+        * @param array $testFileInfo Parsed file info returned by TestFileReader
+        * @return bool True if passed all tests, false if any tests failed.
+        */
+       public function runTests( $testFileInfo ) {
                $ok = true;
 
-               foreach ( $tests as $t ) {
-                       $result =
-                               $this->runTest( $t['test'], $t['input'], $t['result'], $t['options'], $t['config'] );
-                       $ok = $ok && $result;
-                       $this->recorder->record( $t['test'], $t['subtest'], $result );
+               $this->checkSetupDone( 'staticSetup' );
+
+               // Don't add articles from the file if there are no enabled tests from the file
+               if ( !$testFileInfo['tests'] ) {
+                       return true;
                }
 
-               if ( $this->showProgress ) {
-                       print "\n";
+               // If any requirements are not met, mark all tests from the file as skipped
+               if ( !$this->meetsRequirements( $testFileInfo['requirements'] ) ) {
+                       foreach ( $testFileInfo['tests'] as $test ) {
+                               $this->recorder->startTest( $test );
+                               $this->recorder->skipped( $test, 'required extension not enabled' );
+                       }
+                       return true;
+               }
+
+               // Add articles
+               $this->addArticles( $testFileInfo['articles'] );
+
+               // Run tests
+               foreach ( $testFileInfo['tests'] as $test ) {
+                       $this->recorder->startTest( $test );
+                       $result =
+                               $this->runTest( $test );
+                       if ( $result !== false ) {
+                               $ok = $ok && $result->isSuccess();
+                               $this->recorder->record( $test, $result );
+                       }
                }
 
                return $ok;
@@ -449,21 +720,7 @@ class ParserTestRunner {
 
                $class = $wgParserConf['class'];
                $parser = new $class( [ 'preprocessorClass' => $preprocessor ] + $wgParserConf );
-
-               foreach ( $this->hooks as $tag => $callback ) {
-                       $parser->setHook( $tag, $callback );
-               }
-
-               foreach ( $this->functionHooks as $tag => $bits ) {
-                       list( $callback, $flags ) = $bits;
-                       $parser->setFunctionHook( $tag, $callback, $flags );
-               }
-
-               foreach ( $this->transparentHooks as $tag => $callback ) {
-                       $parser->setTransparentTagHook( $tag, $callback );
-               }
-
-               Hooks::run( 'ParserTestParser', [ &$parser ] );
+               ParserTestParserHook::setup( $parser );
 
                return $parser;
        }
@@ -473,33 +730,39 @@ class ParserTestRunner {
         * and compare the output against the expected results.
         * Prints status and explanatory messages to stdout.
         *
-        * @param string $desc Test's description
-        * @param string $input Wikitext to try rendering
-        * @param string $result Result to output
-        * @param array $opts Test's options
-        * @param string $config Overrides for global variables, one per line
-        * @return bool
+        * staticSetup() and setupWikiData() must be called before this function
+        * is entered.
+        *
+        * @param array $test The test parameters:
+        *  - test: The test name
+        *  - desc: The subtest description
+        *  - input: Wikitext to try rendering
+        *  - options: Array of test options
+        *  - config: Overrides for global variables, one per line
+        *
+        * @return ParserTestResult or false if skipped
         */
-       public function runTest( $desc, $input, $result, $opts, $config ) {
-               if ( $this->showProgress ) {
-                       $this->showTesting( $desc );
-               }
-
-               $opts = $this->parseOptions( $opts );
-               $context = $this->setupGlobals( $opts, $config );
+       public function runTest( $test ) {
+               wfDebug( __METHOD__.": running {$test['desc']}" );
+               $opts = $this->parseOptions( $test['options'] );
+               $teardownGuard = $this->perTestSetup( $test );
 
+               $context = RequestContext::getMain();
                $user = $context->getUser();
                $options = ParserOptions::newFromContext( $context );
 
                if ( isset( $opts['djvu'] ) ) {
                        if ( !$this->djVuSupport->isEnabled() ) {
-                               return $this->showSkipped();
+                               $this->recorder->skipped( $test,
+                                       'djvu binaries do not exist or are not executable' );
+                               return false;
                        }
                }
 
                if ( isset( $opts['tidy'] ) ) {
                        if ( !$this->tidySupport->isEnabled() ) {
-                               return $this->showSkipped();
+                               $this->recorder->skipped( $test, 'tidy extension is not installed' );
+                               return false;
                        } else {
                                $options->setTidy( true );
                        }
@@ -511,29 +774,28 @@ class ParserTestRunner {
                        $titleText = 'Parser test';
                }
 
-               ObjectCache::getMainWANInstance()->clearProcessCache();
                $local = isset( $opts['local'] );
                $preprocessor = isset( $opts['preprocessor'] ) ? $opts['preprocessor'] : null;
                $parser = $this->getParser( $preprocessor );
                $title = Title::newFromText( $titleText );
 
                if ( isset( $opts['pst'] ) ) {
-                       $out = $parser->preSaveTransform( $input, $title, $user, $options );
+                       $out = $parser->preSaveTransform( $test['input'], $title, $user, $options );
                } elseif ( isset( $opts['msg'] ) ) {
-                       $out = $parser->transformMsg( $input, $options, $title );
+                       $out = $parser->transformMsg( $test['input'], $options, $title );
                } elseif ( isset( $opts['section'] ) ) {
                        $section = $opts['section'];
-                       $out = $parser->getSection( $input, $section );
+                       $out = $parser->getSection( $test['input'], $section );
                } elseif ( isset( $opts['replace'] ) ) {
                        $section = $opts['replace'][0];
                        $replace = $opts['replace'][1];
-                       $out = $parser->replaceSection( $input, $section, $replace );
+                       $out = $parser->replaceSection( $test['input'], $section, $replace );
                } elseif ( isset( $opts['comment'] ) ) {
-                       $out = Linker::formatComment( $input, $title, $local );
+                       $out = Linker::formatComment( $test['input'], $title, $local );
                } elseif ( isset( $opts['preload'] ) ) {
-                       $out = $parser->getPreloadText( $input, $title, $options );
+                       $out = $parser->getPreloadText( $test['input'], $title, $options );
                } else {
-                       $output = $parser->parse( $input, $title, $options, true, true, 1337 );
+                       $output = $parser->parse( $test['input'], $title, $options, true, true, 1337 );
                        $output->setTOCEnabled( !isset( $opts['notoc'] ) );
                        $out = $output->getText();
                        if ( isset( $opts['tidy'] ) ) {
@@ -559,45 +821,27 @@ class ParserTestRunner {
                        if ( isset( $opts['ill'] ) ) {
                                $out = implode( ' ', $output->getLanguageLinks() );
                        } elseif ( isset( $opts['cat'] ) ) {
-                               $outputPage = $context->getOutput();
-                               $outputPage->addCategoryLinks( $output->getCategories() );
-                               $cats = $outputPage->getCategoryLinks();
-
-                               if ( isset( $cats['normal'] ) ) {
-                                       $out = implode( ' ', $cats['normal'] );
-                               } else {
-                                       $out = '';
+                               $out = '';
+                               foreach ( $output->getCategories() as $name => $sortkey ) {
+                                       if ( $out !== '' ) {
+                                               $out .= "\n";
+                                       }
+                                       $out .= "cat=$name sort=$sortkey";
                                }
                        }
                }
 
-               $this->teardownGlobals();
+               ScopedCallback::consume( $teardownGuard );
 
+               $expected = $test['result'];
                if ( count( $this->normalizationFunctions ) ) {
-                       $result = ParserTestResultNormalizer::normalize( $result, $this->normalizationFunctions );
+                       $expected = ParserTestResultNormalizer::normalize(
+                               $test['expected'], $this->normalizationFunctions );
                        $out = ParserTestResultNormalizer::normalize( $out, $this->normalizationFunctions );
                }
 
-               $testResult = new ParserTestResult( $desc );
-               $testResult->expected = $result;
-               $testResult->actual = $out;
-
-               return $this->showTestResult( $testResult );
-       }
-
-       /**
-        * Refactored in 1.22 to use ParserTestResult
-        * @param ParserTestResult $testResult
-        * @return bool
-        */
-       function showTestResult( ParserTestResult $testResult ) {
-               if ( $testResult->isSuccess() ) {
-                       $this->showSuccess( $testResult );
-                       return true;
-               } else {
-                       $this->showFailure( $testResult );
-                       return false;
-               }
+               $testResult = new ParserTestResult( $test, $expected, $out );
+               return $testResult;
        }
 
        /**
@@ -617,6 +861,13 @@ class ParserTestRunner {
                }
        }
 
+       /**
+        * Given the options string, return an associative array of options.
+        * @todo Move this to TestFileReader
+        *
+        * @param string $instring
+        * @return array
+        */
        private function parseOptions( $instring ) {
                $opts = [];
                // foo
@@ -705,15 +956,25 @@ class ParserTestRunner {
        }
 
        /**
-        * Set up the global variables for a consistent environment for each test.
-        * Ideally this should replace the global configuration entirely.
-        * @param string $opts
-        * @param string $config
-        * @return RequestContext
+        * Do any required setup which is dependent on test options.
+        *
+        * @see staticSetup() for more information about setup/teardown
+        *
+        * @param array $test Test info supplied by TestFileReader
+        * @param callable|null $nextTeardown
+        * @return ScopedCallback
         */
-       public function setupGlobals( $opts = '', $config = '' ) {
-               # Find out values for some special options.
-               $lang =
+       public function perTestSetup( $test, $nextTeardown = null ) {
+               $teardown = [];
+
+               $this->checkSetupDone( 'setupDatabase', 'setDatabase' );
+               $teardown[] = $this->markSetupDone( 'perTestSetup' );
+
+               $opts = $this->parseOptions( $test['options'] );
+               $config = $test['config'];
+
+               // Find out values for some special options.
+               $langCode =
                        self::getOptionValue( 'language', $opts, 'en' );
                $variant =
                        self::getOptionValue( 'variant', $opts, false );
@@ -722,131 +983,79 @@ class ParserTestRunner {
                $linkHolderBatchSize =
                        self::getOptionValue( 'wgLinkHolderBatchSize', $opts, 1000 );
 
-               $settings = [
-                       'wgServer' => 'http://example.org',
-                       'wgServerName' => 'example.org',
-                       'wgScript' => '/index.php',
-                       'wgScriptPath' => '',
-                       'wgArticlePath' => '/wiki/$1',
-                       'wgActionPaths' => [],
-                       'wgLockManagers' => [ [
-                               'name' => 'fsLockManager',
-                               'class' => 'FSLockManager',
-                               'lockDirectory' => $this->uploadDir . '/lockdir',
-                       ], [
-                               'name' => 'nullLockManager',
-                               'class' => 'NullLockManager',
-                       ] ],
-                       'wgLocalFileRepo' => [
-                               'class' => 'LocalRepo',
-                               'name' => 'local',
-                               'url' => 'http://example.com/images',
-                               'hashLevels' => 2,
-                               'transformVia404' => false,
-                               'backend' => new FSFileBackend( [
-                                       'name' => 'local-backend',
-                                       'wikiId' => wfWikiID(),
-                                       'containerPaths' => [
-                                               'local-public' => $this->uploadDir,
-                                               'local-thumb' => $this->uploadDir . '/thumb',
-                                               'local-temp' => $this->uploadDir . '/temp',
-                                               'local-deleted' => $this->uploadDir . '/delete',
-                                       ]
-                               ] )
-                       ],
+               $setup = [
                        'wgEnableUploads' => self::getOptionValue( 'wgEnableUploads', $opts, true ),
-                       'wgUploadNavigationUrl' => false,
-                       'wgStylePath' => '/skins',
-                       'wgSitename' => 'MediaWiki',
-                       'wgLanguageCode' => $lang,
-                       'wgDBprefix' => $this->db->getType() != 'oracle' ? 'parsertest_' : 'pt_',
+                       'wgLanguageCode' => $langCode,
                        'wgRawHtml' => self::getOptionValue( 'wgRawHtml', $opts, false ),
-                       'wgLang' => null,
-                       'wgContLang' => null,
                        'wgNamespacesWithSubpages' => [ 0 => isset( $opts['subpage'] ) ],
                        'wgMaxTocLevel' => $maxtoclevel,
-                       'wgCapitalLinks' => true,
-                       'wgNoFollowLinks' => true,
-                       'wgNoFollowDomainExceptions' => [ 'no-nofollow.org' ],
-                       'wgThumbnailScriptPath' => false,
-                       'wgUseImageResize' => true,
-                       'wgSVGConverter' => 'null',
-                       'wgSVGConverters' => [ 'null' => 'echo "1">$output' ],
-                       'wgLocaltimezone' => 'UTC',
                        'wgAllowExternalImages' => self::getOptionValue( 'wgAllowExternalImages', $opts, true ),
                        'wgThumbLimits' => [ self::getOptionValue( 'thumbsize', $opts, 180 ) ],
                        'wgDefaultLanguageVariant' => $variant,
-                       'wgVariantArticlePath' => false,
-                       'wgGroupPermissions' => [ '*' => [
-                               'createaccount' => true,
-                               'read' => true,
-                               'edit' => true,
-                               'createpage' => true,
-                               'createtalk' => true,
-                       ] ],
-                       'wgNamespaceProtection' => [ NS_MEDIAWIKI => 'editinterface' ],
-                       'wgDefaultExternalStore' => [],
-                       'wgForeignFileRepos' => [],
                        'wgLinkHolderBatchSize' => $linkHolderBatchSize,
-                       'wgExperimentalHtmlIds' => false,
-                       'wgExternalLinkTarget' => false,
-                       'wgHtml5' => true,
-                       'wgAdaptiveMessageCache' => true,
-                       'wgDisableLangConversion' => false,
-                       'wgDisableTitleConversion' => false,
-                       // Tidy options.
-                       'wgUseTidy' => false,
-                       'wgTidyConfig' => isset( $opts['tidy'] ) ? $this->tidySupport->getConfig() : null
                ];
 
                if ( $config ) {
                        $configLines = explode( "\n", $config );
 
                        foreach ( $configLines as $line ) {
-                               list( $var, $value ) = explode( '=', $line, 2 );
-
-                               $settings[$var] = eval( "return $value;" );
+                               list( $var, $value )  = explode( '=', $line, 2 );
+                               $setup[$var] = eval( "return $value;" );
                        }
                }
 
-               $this->savedGlobals = [];
-
                /** @since 1.20 */
-               Hooks::run( 'ParserTestGlobals', [ &$settings ] );
+               Hooks::run( 'ParserTestGlobals', [ &$setup ] );
 
-               foreach ( $settings as $var => $val ) {
-                       if ( array_key_exists( $var, $GLOBALS ) ) {
-                               $this->savedGlobals[$var] = $GLOBALS[$var];
+               // Create tidy driver
+               if ( isset( $opts['tidy'] ) ) {
+                       // Cache a driver instance
+                       if ( $this->tidyDriver === null ) {
+                               $this->tidyDriver = MWTidy::factory( $this->tidySupport->getConfig() );
                        }
-
-                       $GLOBALS[$var] = $val;
+                       $tidy = $this->tidyDriver;
+               } else {
+                       $tidy = false;
                }
-
-               // Must be set before $context as user language defaults to $wgContLang
-               $GLOBALS['wgContLang'] = Language::factory( $lang );
-               $GLOBALS['wgMemc'] = new EmptyBagOStuff;
-
-               RequestContext::resetMain();
-               $context = RequestContext::getMain();
-               $GLOBALS['wgLang'] = $context->getLanguage();
-               $GLOBALS['wgOut'] = $context->getOutput();
-               $GLOBALS['wgUser'] = $context->getUser();
+               MWTidy::setInstance( $tidy );
+               $teardown[] = function () {
+                       MWTidy::destroySingleton();
+               };
+
+               // Set content language. This invalidates the magic word cache and title services
+               wfDebug( "Setting up language $langCode" );
+               $lang = Language::factory( $langCode );
+               $setup['wgContLang'] = $lang;
+               $reset = function () {
+                       MagicWord::clearCache();
+                       $this->resetTitleServices();
+               };
+               $setup[] = $reset;
+               $teardown[] = $reset;
+
+               // Make a user object with the same language
+               $user = new User;
+               $user->setOption( 'language', $langCode );
+               $setup['wgLang'] = $lang;
 
                // We (re)set $wgThumbLimits to a single-element array above.
-               $context->getUser()->setOption( 'thumbsize', 0 );
-
-               global $wgHooks;
+               $user->setOption( 'thumbsize', 0 );
 
-               $wgHooks['ParserTestParser'][] = 'ParserTestParserHook::setup';
-               $wgHooks['ParserGetVariableValueTs'][] = 'ParserTestRunner::getFakeTimestamp';
+               $setup['wgUser'] = $user;
 
-               MagicWord::clearCache();
-               MWTidy::destroySingleton();
-               RepoGroup::destroySingleton();
+               // And put both user and language into the context
+               $context = RequestContext::getMain();
+               $context->setUser( $user );
+               $context->setLanguage( $lang );
+               $teardown[] = function () use ( $context ) {
+                       // Reset context to the restored globals
+                       $context->setUser( $GLOBALS['wgUser'] );
+                       $context->setLanguage( $GLOBALS['wgContLang'] );
+               };
 
-               self::resetTitleServices();
+               $teardown[] = $this->executeSetupSnippets( $setup );
 
-               return $context;
+               return $this->createTeardownObject( $teardown, $nextTeardown );
        }
 
        /**
@@ -876,31 +1085,46 @@ class ParserTestRunner {
                return $tables;
        }
 
+       public function setDatabase( IDatabase $db ) {
+               $this->db = $db;
+               $this->setupDone['setDatabase'] = true;
+       }
+
        /**
-        * Set up a temporary set of wiki tables to work with for the tests.
-        * Currently this will only be done once per run, and any changes to
-        * the db will be visible to later tests in the run.
+        * Set up temporary DB tables.
+        *
+        * For best performance, call this once only for all tests. However, it can
+        * be called at the start of each test if more isolation is desired.
+        *
+        * @todo: This is basically an unrefactored copy of
+        * MediaWikiTestCase::setupAllTestDBs. They should be factored out somehow.
+        *
+        * Do not call this function from a MediaWikiTestCase subclass, since
+        * MediaWikiTestCase does its own DB setup. Instead use setDatabase().
+        *
+        * @see staticSetup() for more information about setup/teardown
+        *
+        * @param ScopedCallback|null $nextTeardown The next teardown object
+        * @return ScopedCallback The teardown object
         */
-       public function setupDatabase() {
+       public function setupDatabase( $nextTeardown = null ) {
                global $wgDBprefix;
 
-               if ( $this->databaseSetupDone ) {
-                       return;
-               }
-
                $this->db = wfGetDB( DB_MASTER );
                $dbType = $this->db->getType();
 
-               if ( $wgDBprefix === 'parsertest_' || ( $dbType == 'oracle' && $wgDBprefix === 'pt_' ) ) {
-                       throw new MWException( 'setupDatabase should be called before setupGlobals' );
+               if ( $dbType == 'oracle' ) {
+                       $suspiciousPrefixes = [ 'pt_', MediaWikiTestCase::ORA_DB_PREFIX ];
+               } else {
+                       $suspiciousPrefixes = [ 'parsertest_', MediaWikiTestCase::DB_PREFIX ];
+               }
+               if ( in_array( $wgDBprefix, $suspiciousPrefixes ) ) {
+                       throw new MWException( "\$wgDBprefix=$wgDBprefix suggests DB setup is already done" );
                }
 
-               $this->databaseSetupDone = true;
+               $teardown = [];
 
-               # SqlBagOStuff broke when using temporary tables on r40209 (bug 15892).
-               # It seems to have been fixed since (r55079?), but regressed at some point before r85701.
-               # This works around it for now...
-               ObjectCache::$instances[CACHE_DB] = new HashBagOStuff;
+               $teardown[] = $this->markSetupDone( 'setupDatabase' );
 
                # CREATE TEMPORARY TABLE breaks if there is more than one server
                if ( wfGetLB()->getServerCount() != 1 ) {
@@ -924,20 +1148,45 @@ class ParserTestRunner {
                                'user_name' => 'Anonymous' ] );
                }
 
-               # Update certain things in site_stats
-               $this->db->insert( 'site_stats',
-                       [ 'ss_row_id' => 1, 'ss_images' => 2, 'ss_good_articles' => 1 ] );
+               $teardown[] = function () {
+                       $this->teardownDatabase();
+               };
+
+               // Wipe some DB query result caches on setup and teardown
+               $reset = function () {
+                       LinkCache::singleton()->clear();
+
+                       // Clear the message cache
+                       MessageCache::singleton()->clear();
+               };
+               $reset();
+               $teardown[] = $reset;
+               return $this->createTeardownObject( $teardown, $nextTeardown );
+       }
 
-               # Reinitialise the LocalisationCache to match the database state
-               Language::getLocalisationCache()->unloadAll();
+       /**
+        * Add data about uploads to the new test DB, and set up the upload
+        * directory. This should be called after either setDatabase() or
+        * setupDatabase().
+        *
+        * @param ScopedCallback|null $nextTeardown The next teardown object
+        * @return ScopedCallback The teardown object
+        */
+       public function setupUploads( $nextTeardown = null ) {
+               $teardown = [];
+
+               $this->checkSetupDone( 'setupDatabase', 'setDatabase' );
+               $teardown[] = $this->markSetupDone( 'setupUploads' );
 
-               # Clear the message cache
-               MessageCache::singleton()->clear();
+               // Create the files in the upload directory (or pretend to create them
+               // in a MockFileBackend). Append teardown callback.
+               $teardown[] = $this->setupUploadBackend();
 
-               // Remember to update newParserTests.php after changing the below
-               // (and it uses a slightly different syntax just for teh lulz)
-               $this->setupUploadDir();
+               // Create a user
                $user = User::createNew( 'WikiSysop' );
+
+               // Register the uploads in the database
+
                $image = wfLocalFile( Title::makeTitle( NS_FILE, 'Foobar.jpg' ) );
                # note that the size/width/height/bits/etc of the file
                # are actually set by inspecting the file itself; the arguments
@@ -1060,14 +1309,18 @@ class ParserTestRunner {
                        'sha1' => Wikimedia\base_convert( '', 16, 36, 31 ),
                        'fileExists' => true
                ], $this->db->timestamp( '20010115123600' ), $user );
+
+               return $this->createTeardownObject( $teardown, $nextTeardown );
        }
 
-       public function teardownDatabase() {
-               if ( !$this->databaseSetupDone ) {
-                       $this->teardownGlobals();
-                       return;
-               }
-               $this->teardownUploadDir( $this->uploadDir );
+       /**
+        * Helper for database teardown, called from the teardown closure. Destroy
+        * the database clone and fix up some things that CloneDatabase doesn't fix.
+        *
+        * @todo Move most things here to CloneDatabase
+        */
+       private function teardownDatabase() {
+               $this->checkSetupDone( 'setupDatabase' );
 
                $this->dbClone->destroy();
                $this->databaseSetupDone = false;
@@ -1081,7 +1334,6 @@ class ParserTestRunner {
                                $this->db->query( "DROP TABLE `parsertest_searchindex`" );
                        }
                        # Don't need to do anything
-                       $this->teardownGlobals();
                        return;
                }
 
@@ -1098,375 +1350,179 @@ class ParserTestRunner {
                if ( $this->db->getType() == 'oracle' ) {
                        $this->db->query( 'BEGIN FILL_WIKI_INFO; END;' );
                }
-
-               $this->teardownGlobals();
        }
 
        /**
-        * Create a dummy uploads directory which will contain a couple
-        * of files in order to pass existence tests.
+        * Upload test files to the backend created by createRepoGroup().
         *
-        * @return string The directory
+        * @return callable The teardown callback
         */
-       private function setupUploadDir() {
+       private function setupUploadBackend() {
                global $IP;
 
-               $dir = $this->uploadDir;
-               if ( $this->keepUploads && is_dir( $dir ) ) {
-                       return;
-               }
-
-               // wfDebug( "Creating upload directory $dir\n" );
-               if ( file_exists( $dir ) ) {
-                       wfDebug( "Already exists!\n" );
-                       return;
-               }
-
-               wfMkdirParents( $dir . '/3/3a', null, __METHOD__ );
-               copy( "$IP/tests/phpunit/data/parser/headbg.jpg", "$dir/3/3a/Foobar.jpg" );
-               wfMkdirParents( $dir . '/e/ea', null, __METHOD__ );
-               copy( "$IP/tests/phpunit/data/parser/wiki.png", "$dir/e/ea/Thumb.png" );
-               wfMkdirParents( $dir . '/0/09', null, __METHOD__ );
-               copy( "$IP/tests/phpunit/data/parser/headbg.jpg", "$dir/0/09/Bad.jpg" );
-               wfMkdirParents( $dir . '/f/ff', null, __METHOD__ );
-               file_put_contents( "$dir/f/ff/Foobar.svg",
-                       '<?xml version="1.0" encoding="utf-8"?>' .
+               $repo = RepoGroup::singleton()->getLocalRepo();
+               $base = $repo->getZonePath( 'public' );
+               $backend = $repo->getBackend();
+               $backend->prepare( [ 'dir' => "$base/3/3a" ] );
+               $backend->store( [
+                       'src' => "$IP/tests/phpunit/data/parser/headbg.jpg",
+                       'dst' => "$base/3/3a/Foobar.jpg"
+               ] );
+               $backend->prepare( [ 'dir' => "$base/e/ea" ] );
+               $backend->store( [
+                       'src' => "$IP/tests/phpunit/data/parser/wiki.png",
+                       'dst' => "$base/e/ea/Thumb.png"
+               ] );
+               $backend->prepare( [ 'dir' => "$base/0/09" ] );
+               $backend->store( [
+                       'src' => "$IP/tests/phpunit/data/parser/headbg.jpg",
+                       'dst' => "$base/0/09/Bad.jpg"
+               ] );
+               $backend->prepare( [ 'dir' => "$base/5/5f" ] );
+               $backend->store( [
+                       'src' => "$IP/tests/phpunit/data/parser/LoremIpsum.djvu",
+                       'dst' => "$base/5/5f/LoremIpsum.djvu"
+               ] );
+
+               // No helpful SVG file to copy, so make one ourselves
+               $data = '<?xml version="1.0" encoding="utf-8"?>' .
                        '<svg xmlns="http://www.w3.org/2000/svg"' .
-                       ' version="1.1" width="240" height="180"/>' );
-               wfMkdirParents( $dir . '/5/5f', null, __METHOD__ );
-               copy( "$IP/tests/phpunit/data/parser/LoremIpsum.djvu", "$dir/5/5f/LoremIpsum.djvu" );
-               wfMkdirParents( $dir . '/0/00', null, __METHOD__ );
-               copy( "$IP/tests/phpunit/data/parser/320x240.ogv", "$dir/0/00/Video.ogv" );
-               wfMkdirParents( $dir . '/4/41', null, __METHOD__ );
-               copy( "$IP/tests/phpunit/data/media/say-test.ogg", "$dir/4/41/Audio.oga" );
-
-               return;
-       }
+                       ' version="1.1" width="240" height="180"/>';
 
-       /**
-        * Restore default values and perform any necessary clean-up
-        * after each test runs.
-        */
-       public function teardownGlobals() {
-               RepoGroup::destroySingleton();
-               FileBackendGroup::destroySingleton();
-               LockManagerGroup::destroySingletons();
-               LinkCache::singleton()->clear();
-               MWTidy::destroySingleton();
-
-               foreach ( $this->savedGlobals as $var => $val ) {
-                       $GLOBALS[$var] = $val;
-               }
+               $backend->prepare( [ 'dir' => "$base/f/ff" ] );
+               $backend->quickCreate( [
+                       'content' => $data, 'dst' => "$base/f/ff/Foobar.svg"
+               ] );
+
+               return function () use ( $backend ) {
+                       if ( $backend instanceof MockFileBackend ) {
+                               // In memory backend, so dont bother cleaning them up.
+                               return;
+                       }
+                       $this->teardownUploadBackend();
+               };
        }
 
        /**
         * Remove the dummy uploads directory
-        * @param string $dir
         */
-       private function teardownUploadDir( $dir ) {
+       private function teardownUploadBackend() {
                if ( $this->keepUploads ) {
                        return;
                }
 
-               // delete the files first, then the dirs.
-               self::deleteFiles(
-                       [
-                               "$dir/3/3a/Foobar.jpg",
-                               "$dir/thumb/3/3a/Foobar.jpg/*.jpg",
-                               "$dir/e/ea/Thumb.png",
-                               "$dir/0/09/Bad.jpg",
-                               "$dir/5/5f/LoremIpsum.djvu",
-                               "$dir/thumb/5/5f/LoremIpsum.djvu/*-LoremIpsum.djvu.jpg",
-                               "$dir/f/ff/Foobar.svg",
-                               "$dir/thumb/f/ff/Foobar.svg/*-Foobar.svg.png",
-                               "$dir/math/f/a/5/fa50b8b616463173474302ca3e63586b.png",
-                               "$dir/0/00/Video.ogv",
-                               "$dir/thumb/0/00/Video.ogv/120px--Video.ogv.jpg",
-                               "$dir/thumb/0/00/Video.ogv/180px--Video.ogv.jpg",
-                               "$dir/thumb/0/00/Video.ogv/240px--Video.ogv.jpg",
-                               "$dir/thumb/0/00/Video.ogv/320px--Video.ogv.jpg",
-                               "$dir/thumb/0/00/Video.ogv/270px--Video.ogv.jpg",
-                               "$dir/thumb/0/00/Video.ogv/320px-seek=2-Video.ogv.jpg",
-                               "$dir/thumb/0/00/Video.ogv/320px-seek=3.3666666666667-Video.ogv.jpg",
-                               "$dir/4/41/Audio.oga",
-                       ]
-               );
+               $repo = RepoGroup::singleton()->getLocalRepo();
+               $public = $repo->getZonePath( 'public' );
 
-               self::deleteDirs(
+               $this->deleteFiles(
                        [
-                               "$dir/3/3a",
-                               "$dir/3",
-                               "$dir/thumb/3/3a/Foobar.jpg",
-                               "$dir/thumb/3/3a",
-                               "$dir/thumb/3",
-                               "$dir/e/ea",
-                               "$dir/e",
-                               "$dir/f/ff/",
-                               "$dir/f/",
-                               "$dir/thumb/f/ff/Foobar.svg",
-                               "$dir/thumb/f/ff/",
-                               "$dir/thumb/f/",
-                               "$dir/0/00/",
-                               "$dir/0/09/",
-                               "$dir/0/",
-                               "$dir/5/5f",
-                               "$dir/5",
-                               "$dir/thumb/0/00/Video.ogv",
-                               "$dir/thumb/0/00",
-                               "$dir/thumb/0",
-                               "$dir/thumb/5/5f/LoremIpsum.djvu",
-                               "$dir/thumb/5/5f",
-                               "$dir/thumb/5",
-                               "$dir/thumb",
-                               "$dir/4/41",
-                               "$dir/4",
-                               "$dir/math/f/a/5",
-                               "$dir/math/f/a",
-                               "$dir/math/f",
-                               "$dir/math",
-                               "$dir/lockdir",
-                               "$dir",
+                               "$public/3/3a/Foobar.jpg",
+                               "$public/e/ea/Thumb.png",
+                               "$public/0/09/Bad.jpg",
+                               "$public/5/5f/LoremIpsum.djvu",
+                               "$public/f/ff/Foobar.svg",
+                               "$public/0/00/Video.ogv",
+                               "$public/4/41/Audio.oga",
                        ]
                );
        }
 
        /**
-        * Delete the specified files, if they exist.
-        * @param array $files Full paths to files to delete.
-        */
-       private static function deleteFiles( $files ) {
-               foreach ( $files as $pattern ) {
-                       foreach ( glob( $pattern ) as $file ) {
-                               if ( file_exists( $file ) ) {
-                                       unlink( $file );
-                               }
-                       }
-               }
-       }
-
-       /**
-        * Delete the specified directories, if they exist. Must be empty.
-        * @param array $dirs Full paths to directories to delete.
+        * Delete the specified files and their parent directories
+        * @param array $files File backend URIs mwstore://...
         */
-       private static function deleteDirs( $dirs ) {
-               foreach ( $dirs as $dir ) {
-                       if ( is_dir( $dir ) ) {
-                               rmdir( $dir );
-                       }
+       private function deleteFiles( $files ) {
+               // Delete the files
+               $backend = RepoGroup::singleton()->getLocalRepo()->getBackend();
+               foreach ( $files as $file ) {
+                       $backend->delete( [ 'src' => $file ], [ 'force' => 1 ] );
                }
-       }
 
-       /**
-        * "Running test $desc..."
-        * @param string $desc
-        */
-       protected function showTesting( $desc ) {
-               print "Running test $desc... ";
-       }
-
-       /**
-        * Print a happy success message.
-        *
-        * Refactored in 1.22 to use ParserTestResult
-        *
-        * @param ParserTestResult $testResult
-        * @return bool
-        */
-       protected function showSuccess( ParserTestResult $testResult ) {
-               if ( $this->showProgress ) {
-                       print $this->term->color( '1;32' ) . 'PASSED' . $this->term->reset() . "\n";
-               }
-
-               return true;
-       }
-
-       /**
-        * Print a failure message and provide some explanatory output
-        * about what went wrong if so configured.
-        *
-        * Refactored in 1.22 to use ParserTestResult
-        *
-        * @param ParserTestResult $testResult
-        * @return bool
-        */
-       protected function showFailure( ParserTestResult $testResult ) {
-               if ( $this->showFailure ) {
-                       if ( !$this->showProgress ) {
-                               # In quiet mode we didn't show the 'Testing' message before the
-                               # test, in case it succeeded. Show it now:
-                               $this->showTesting( $testResult->description );
-                       }
-
-                       print $this->term->color( '31' ) . 'FAILED!' . $this->term->reset() . "\n";
-
-                       if ( $this->showOutput ) {
-                               print "--- Expected ---\n{$testResult->expected}\n";
-                               print "--- Actual ---\n{$testResult->actual}\n";
-                       }
-
-                       if ( $this->showDiffs ) {
-                               print $this->quickDiff( $testResult->expected, $testResult->actual );
-                               if ( !$this->wellFormed( $testResult->actual ) ) {
-                                       print "XML error: $this->mXmlError\n";
+               // Delete the parent directories
+               foreach ( $files as $file ) {
+                       $tmp = FileBackend::parentStoragePath( $file );
+                       while ( $tmp ) {
+                               if ( !$backend->clean( [ 'dir' => $tmp ] )->isOK() ) {
+                                       break;
                                }
+                               $tmp = FileBackend::parentStoragePath( $tmp );
                        }
                }
-
-               return false;
        }
 
        /**
-        * Print a skipped message.
+        * Add articles to the test DB.
         *
-        * @return bool
+        * @param $articles Article info array from TestFileReader
         */
-       protected function showSkipped() {
-               if ( $this->showProgress ) {
-                       print $this->term->color( '1;33' ) . 'SKIPPED' . $this->term->reset() . "\n";
+       public function addArticles( $articles ) {
+               global $wgContLang;
+               $setup = [];
+               $teardown = [];
+
+               // Be sure ParserTestRunner::addArticle has correct language set,
+               // so that system messages get into the right language cache
+               if ( $wgContLang->getCode() !== 'en' ) {
+                       $setup['wgLanguageCode'] = 'en';
+                       $setup['wgContLang'] = Language::factory( 'en' );
                }
 
-               return true;
-       }
+               // Add special namespaces, in case that hasn't been done by staticSetup() yet
+               $this->appendNamespaceSetup( $setup, $teardown );
 
-       /**
-        * Run given strings through a diff and return the (colorized) output.
-        * Requires writable /tmp directory and a 'diff' command in the PATH.
-        *
-        * @param string $input
-        * @param string $output
-        * @param string $inFileTail Tailing for the input file name
-        * @param string $outFileTail Tailing for the output file name
-        * @return string
-        */
-       protected function quickDiff( $input, $output,
-               $inFileTail = 'expected', $outFileTail = 'actual'
-       ) {
-               if ( $this->markWhitespace ) {
-                       $pairs = [
-                               "\n" => '¶',
-                               ' ' => '·',
-                               "\t" => '→'
-                       ];
-                       $input = strtr( $input, $pairs );
-                       $output = strtr( $output, $pairs );
-               }
-
-               # Windows, or at least the fc utility, is retarded
-               $slash = wfIsWindows() ? '\\' : '/';
-               $prefix = wfTempDir() . "{$slash}mwParser-" . mt_rand();
-
-               $infile = "$prefix-$inFileTail";
-               $this->dumpToFile( $input, $infile );
-
-               $outfile = "$prefix-$outFileTail";
-               $this->dumpToFile( $output, $outfile );
-
-               $shellInfile = wfEscapeShellArg( $infile );
-               $shellOutfile = wfEscapeShellArg( $outfile );
+               // wgCapitalLinks obviously needs initialisation
+               $setup['wgCapitalLinks'] = true;
 
-               global $wgDiff3;
-               // we assume that people with diff3 also have usual diff
-               if ( $this->useDwdiff ) {
-                       $shellCommand = 'dwdiff -Pc';
-               } else {
-                       $shellCommand = ( wfIsWindows() && !$wgDiff3 ) ? 'fc' : 'diff -au';
-               }
-
-               $diff = wfShellExec( "$shellCommand $shellInfile $shellOutfile" );
+               $teardown[] = $this->executeSetupSnippets( $setup );
 
-               unlink( $infile );
-               unlink( $outfile );
-
-               if ( $this->useDwdiff ) {
-                       return $diff;
-               } else {
-                       return $this->colorDiff( $diff );
+               foreach ( $articles as $info ) {
+                       $this->addArticle( $info['name'], $info['text'], $info['file'], $info['line'] );
                }
-       }
-
-       /**
-        * Write the given string to a file, adding a final newline.
-        *
-        * @param string $data
-        * @param string $filename
-        */
-       private function dumpToFile( $data, $filename ) {
-               $file = fopen( $filename, "wt" );
-               fwrite( $file, $data . "\n" );
-               fclose( $file );
-       }
 
-       /**
-        * Colorize unified diff output if set for ANSI color output.
-        * Subtractions are colored blue, additions red.
-        *
-        * @param string $text
-        * @return string
-        */
-       protected function colorDiff( $text ) {
-               return preg_replace(
-                       [ '/^(-.*)$/m', '/^(\+.*)$/m' ],
-                       [ $this->term->color( 34 ) . '$1' . $this->term->reset(),
-                               $this->term->color( 31 ) . '$1' . $this->term->reset() ],
-                       $text );
-       }
+               // Wipe WANObjectCache process cache, which is invalidated by article insertion
+               // due to T144706
+               ObjectCache::getMainWANInstance()->clearProcessCache();
 
-       /**
-        * Show "Reading tests from ..."
-        *
-        * @param string $path
-        */
-       public function showRunFile( $path ) {
-               print $this->term->color( 1 ) .
-                       "Reading tests from \"$path\"..." .
-                       $this->term->reset() .
-                       "\n";
+               $this->executeSetupSnippets( $teardown );
        }
 
        /**
         * Insert a temporary test article
         * @param string $name The title, including any prefix
         * @param string $text The article text
+        * @param string $file The input file name
         * @param int|string $line The input line number, for reporting errors
-        * @param bool|string $ignoreDuplicate Whether to silently ignore duplicate pages
         * @throws Exception
         * @throws MWException
         */
-       public static function addArticle( $name, $text, $line = 'unknown', $ignoreDuplicate = '' ) {
-               global $wgCapitalLinks;
-
-               $oldCapitalLinks = $wgCapitalLinks;
-               $wgCapitalLinks = true; // We only need this from SetupGlobals() See r70917#c8637
-
+       private function addArticle( $name, $text, $file, $line ) {
                $text = self::chomp( $text );
                $name = self::chomp( $name );
 
                $title = Title::newFromText( $name );
+               wfDebug( __METHOD__ . ": adding $name" );
 
                if ( is_null( $title ) ) {
-                       throw new MWException( "invalid title '$name' at line $line\n" );
+                       throw new MWException( "invalid title '$name' at $file:$line\n" );
                }
 
                $page = WikiPage::factory( $title );
                $page->loadPageData( 'fromdbmaster' );
 
                if ( $page->exists() ) {
-                       if ( $ignoreDuplicate == 'ignoreduplicate' ) {
-                               return;
-                       } else {
-                               throw new MWException( "duplicate article '$name' at line $line\n" );
-                       }
+                       throw new MWException( "duplicate article '$name' at $file:$line\n" );
                }
 
                $page->doEditContent( ContentHandler::makeContent( $text, $title ), '', EDIT_NEW );
 
-               $wgCapitalLinks = $oldCapitalLinks;
+               // The RepoGroup cache is invalidated by the creation of file redirects
+               if ( $title->getNamespace() === NS_IMAGE ) {
+                       RepoGroup::singleton()->clearCache( $title );
+               }
        }
 
        /**
-        * Steal a callback function from the primary parser, save it for
-        * application to our scary parser. If the hook is not installed,
-        * abort processing of this file.
+        * Check if a hook is installed
         *
         * @param string $name
         * @return bool True if tag hook is present
@@ -1475,21 +1531,17 @@ class ParserTestRunner {
                global $wgParser;
 
                $wgParser->firstCallInit(); // make sure hooks are loaded.
-
                if ( isset( $wgParser->mTagHooks[$name] ) ) {
-                       $this->hooks[$name] = $wgParser->mTagHooks[$name];
+                       return true;
                } else {
-                       echo "   This test suite requires the '$name' hook extension, skipping.\n";
+                       $this->recorder->warning( "   This test suite requires the '$name' hook " .
+                               "extension, skipping." );
                        return false;
                }
-
-               return true;
        }
 
        /**
-        * Steal a callback function from the primary parser, save it for
-        * application to our scary parser. If the hook is not installed,
-        * abort processing of this file.
+        * Check if a function hook is installed
         *
         * @param string $name
         * @return bool True if function hook is present
@@ -1500,19 +1552,16 @@ class ParserTestRunner {
                $wgParser->firstCallInit(); // make sure hooks are loaded.
 
                if ( isset( $wgParser->mFunctionHooks[$name] ) ) {
-                       $this->functionHooks[$name] = $wgParser->mFunctionHooks[$name];
+                       return true;
                } else {
-                       echo "   This test suite requires the '$name' function hook extension, skipping.\n";
+                       $this->recorder->warning( "   This test suite requires the '$name' function " .
+                               "hook extension, skipping." );
                        return false;
                }
-
-               return true;
        }
 
        /**
-        * Steal a callback function from the primary parser, save it for
-        * application to our scary parser. If the hook is not installed,
-        * abort processing of this file.
+        * Check if a transparent tag hook is installed
         *
         * @param string $name
         * @return bool True if function hook is present
@@ -1523,67 +1572,18 @@ class ParserTestRunner {
                $wgParser->firstCallInit(); // make sure hooks are loaded.
 
                if ( isset( $wgParser->mTransparentTagHooks[$name] ) ) {
-                       $this->transparentHooks[$name] = $wgParser->mTransparentTagHooks[$name];
+                       return true;
                } else {
-                       echo "   This test suite requires the '$name' transparent hook extension, skipping.\n";
-                       return false;
-               }
-
-               return true;
-       }
-
-       private function wellFormed( $text ) {
-               $html =
-                       Sanitizer::hackDocType() .
-                               '<html>' .
-                               $text .
-                               '</html>';
-
-               $parser = xml_parser_create( "UTF-8" );
-
-               # case folding violates XML standard, turn it off
-               xml_parser_set_option( $parser, XML_OPTION_CASE_FOLDING, false );
-
-               if ( !xml_parse( $parser, $html, true ) ) {
-                       $err = xml_error_string( xml_get_error_code( $parser ) );
-                       $position = xml_get_current_byte_index( $parser );
-                       $fragment = $this->extractFragment( $html, $position );
-                       $this->mXmlError = "$err at byte $position:\n$fragment";
-                       xml_parser_free( $parser );
-
+                       $this->recorder->warning( "   This test suite requires the '$name' transparent " .
+                               "hook extension, skipping.\n" );
                        return false;
                }
-
-               xml_parser_free( $parser );
-
-               return true;
-       }
-
-       private function extractFragment( $text, $position ) {
-               $start = max( 0, $position - 10 );
-               $before = $position - $start;
-               $fragment = '...' .
-                       $this->term->color( 34 ) .
-                       substr( $text, $start, $before ) .
-                       $this->term->color( 0 ) .
-                       $this->term->color( 31 ) .
-                       $this->term->color( 1 ) .
-                       substr( $text, $position, 1 ) .
-                       $this->term->color( 0 ) .
-                       $this->term->color( 34 ) .
-                       substr( $text, $position + 1, 9 ) .
-                       $this->term->color( 0 ) .
-                       '...';
-               $display = str_replace( "\n", ' ', $fragment );
-               $caret = '   ' .
-                       str_repeat( ' ', $before ) .
-                       $this->term->color( 31 ) .
-                       '^' .
-                       $this->term->color( 0 );
-
-               return "$display\n$caret";
        }
 
+       /**
+        * The ParserGetVariableValueTs hook, used to make sure time-related parser
+        * functions give a persistent value.
+        */
        static function getFakeTimestamp( &$parser, &$ts ) {
                $ts = 123; // parsed as '1970-01-01T00:02:03Z'
                return true;
diff --git a/tests/parser/PhpunitTestRecorder.php b/tests/parser/PhpunitTestRecorder.php
new file mode 100644 (file)
index 0000000..238d018
--- /dev/null
@@ -0,0 +1,16 @@
+<?php
+
+class PhpunitTestRecorder extends TestRecorder {
+       private $testCase;
+
+       public function setTestCase( PHPUnit_Framework_TestCase $testCase ) {
+               $this->testCase = $testCase;
+       }
+
+       /**
+        * Mark a test skipped
+        */
+       public function skipped( $test, $reason ) {
+               $this->testCase->markTestSkipped( "SKIPPED: $reason" );
+       }
+}
index 8b41337..f1a82ee 100644 (file)
@@ -1,8 +1,12 @@
-Parser tests are run using our PHPUnit test suite in tests/phpunit:
+Parser tests can be run either via PHPUnit or by using the standalone
+parserTests.php in this directory. The standalone version provides more
+options.
+
+To run parser tests via PHPUnit:
 
  $ cd tests/phpunit
- ./phpunit.php --group Parser
+ ./phpunit.php --testsuite parsertests
 
-You can optionally filter by title using --regex. I.e. :
+You can optionally filter by title using --filter, e.g.
 
- ./phpunit.php --group Parser --regex="Bug 6200"
+ ./phpunit.php --testsuite parsertests --filter="Bug 6200"
diff --git a/tests/parser/TestFileDataProvider.php b/tests/parser/TestFileDataProvider.php
deleted file mode 100644 (file)
index 5528605..0000000
+++ /dev/null
@@ -1,42 +0,0 @@
-<?php
-/**
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
- * http://www.gnu.org/copyleft/gpl.html
- *
- * @file
- * @ingroup Testing
- */
-
-/**
- * An iterator for use as a phpunit data provider. Provides the test arguments
- * in the order expected by ParserIntegrationTest::testParserTest().
- */
-class TestFileDataProvider extends TestFileReader {
-       function current() {
-               $test = parent::current();
-               if ( $test ) {
-                       return [
-                               $test['test'],
-                               $test['input'],
-                               $test['result'],
-                               $test['options'],
-                               $test['config'],
-                       ];
-               } else {
-                       return $test;
-               }
-       }
-}
-
index 18e09cc..a1a8d19 100644 (file)
  * @ingroup Testing
  */
 
-class TestFileReader implements Iterator {
+class TestFileReader {
        private $file;
        private $fh;
-       /**
-        * @var ParserTestRunner|ParserTestTopLevelSuite An instance of ParserTestRunner
-        * (parserTests.php) or ParserTestTopLevelSuite (phpunit)
-        */
-       private $parserTest;
-       private $index = 0;
-       private $test;
        private $section = null;
        /** String|null: current test section being analyzed */
        private $sectionData = [];
-       private $lineNum;
-       private $eof;
-       # Create a fake parser tests which never run anything unless
-       # asked to do so. This will avoid running hooks for a disabled test
-       private $delayedParserTest;
-       private $nextSubTest = 0;
-
-       function __construct( $file, $parserTest ) {
-               $this->file = $file;
-               $this->fh = fopen( $this->file, "rt" );
-
-               if ( !$this->fh ) {
-                       throw new MWException( "Couldn't open file '$file'\n" );
-               }
-
-               $this->parserTest = $parserTest;
-               $this->delayedParserTest = new DelayedParserTest();
-
-               $this->lineNum = $this->index = 0;
-       }
-
-       function rewind() {
-               if ( fseek( $this->fh, 0 ) ) {
-                       throw new MWException( "Couldn't fseek to the start of '$this->file'\n" );
+       private $lineNum = 0;
+       private $runDisabled;
+       private $runParsoid;
+       private $regex;
+
+       private $articles = [];
+       private $requirements = [];
+       private $tests = [];
+
+       public static function read( $file, array $options = [] ) {
+               $reader = new self( $file, $options );
+               $reader->execute();
+
+               $requirements = [];
+               foreach ( $reader->requirements as $type => $reqsOfType ) {
+                       foreach ( $reqsOfType as $name => $unused ) {
+                               $requirements[] = [
+                                       'type' => $type,
+                                       'name' => $name
+                               ];
+                       }
                }
 
-               $this->index = -1;
-               $this->lineNum = 0;
-               $this->eof = false;
-               $this->next();
-
-               return true;
-       }
-
-       function current() {
-               return $this->test;
+               return [
+                       'requirements' => $requirements,
+                       'tests' => $reader->tests,
+                       'articles' => $reader->articles
+               ];
        }
 
-       function key() {
-               return $this->index;
-       }
+       private function __construct( $file, $options ) {
+               $this->file = $file;
+               $this->fh = fopen( $this->file, "rt" );
 
-       function next() {
-               if ( $this->readNextTest() ) {
-                       $this->index++;
-                       return true;
-               } else {
-                       $this->eof = true;
+               if ( !$this->fh ) {
+                       throw new MWException( "Couldn't open file '$file'\n" );
                }
-       }
 
-       function valid() {
-               return $this->eof != true;
+               $options = $options + [
+                       'runDisabled' => false,
+                       'runParsoid' => false,
+                       'regex' => '//',
+               ];
+               $this->runDisabled = $options['runDisabled'];
+               $this->runParsoid = $options['runParsoid'];
+               $this->regex = $options['regex'];
        }
 
-       function setupCurrentTest() {
+       private function addCurrentTest() {
                // "input" and "result" are old section names allowed
                // for backwards-compatibility.
                $input = $this->checkSection( [ 'wikitext', 'input' ], false );
                $result = $this->checkSection( [ 'html/php', 'html/*', 'html', 'result' ], false );
-               // some tests have "with tidy" and "without tidy" variants
+               // Some tests have "with tidy" and "without tidy" variants
                $tidy = $this->checkSection( [ 'html/php+tidy', 'html+tidy' ], false );
-               if ( $tidy != false ) {
-                       if ( $this->nextSubTest == 0 ) {
-                               if ( $result != false ) {
-                                       $this->nextSubTest = 1; // rerun non-tidy variant later
-                               }
-                               $result = $tidy;
-                       } else {
-                               $this->nextSubTest = 0; // go on to next test after this
-                               $tidy = false;
-                       }
-               }
 
                if ( !isset( $this->sectionData['options'] ) ) {
                        $this->sectionData['options'] = '';
@@ -115,50 +90,35 @@ class TestFileReader implements Iterator {
                }
 
                $isDisabled = preg_match( '/\\bdisabled\\b/i', $this->sectionData['options'] ) &&
-                       !$this->parserTest->runDisabled;
+                       !$this->runDisabled;
                $isParsoidOnly = preg_match( '/\\bparsoid\\b/i', $this->sectionData['options'] ) &&
                        $result == 'html' &&
-                       !$this->parserTest->runParsoid;
-               $isFiltered = !preg_match( "/" . $this->parserTest->regex . "/i", $this->sectionData['test'] );
+                       !$this->runParsoid;
+               $isFiltered = !preg_match( $this->regex, $this->sectionData['test'] );
                if ( $input == false || $result == false || $isDisabled || $isParsoidOnly || $isFiltered ) {
-                       # disabled test
-                       return false;
+                       // Disabled test
+                       return;
                }
 
-               # We are really going to run the test, run pending hooks and hooks function
-               wfDebug( __METHOD__ . " unleashing delayed test for: {$this->sectionData['test']}" );
-               $hooksResult = $this->delayedParserTest->unleash( $this->parserTest );
-               if ( !$hooksResult ) {
-                       # Some hook reported an issue. Abort.
-                       throw new MWException( "Problem running requested parser hook from the test file" );
-               }
-
-               $this->test = [
+               $test = [
                        'test' => ParserTestRunner::chomp( $this->sectionData['test'] ),
-                       'subtest' => $this->nextSubTest,
                        'input' => ParserTestRunner::chomp( $this->sectionData[$input] ),
                        'result' => ParserTestRunner::chomp( $this->sectionData[$result] ),
                        'options' => ParserTestRunner::chomp( $this->sectionData['options'] ),
                        'config' => ParserTestRunner::chomp( $this->sectionData['config'] ),
                ];
-               if ( $tidy != false ) {
-                       $this->test['options'] .= " tidy";
+               $test['desc'] = $test['test'];
+               $this->tests[] = $test;
+
+               if ( $tidy !== false ) {
+                       $test['options'] .= " tidy";
+                       $test['desc'] .= ' (with tidy)';
+                       $test['result'] = ParserTestRunner::chomp( $this->sectionData[$tidy] );
+                       $this->tests[] = $test;
                }
-               return true;
        }
 
-       function readNextTest() {
-               # Run additional subtests of previous test
-               while ( $this->nextSubTest > 0 ) {
-                       if ( $this->setupCurrentTest() ) {
-                               return true;
-                       }
-               }
-
-               $this->clearSection();
-               # Reset hooks for the delayed test object
-               $this->delayedParserTest->reset();
-
+       private function execute() {
                while ( false !== ( $line = fgets( $this->fh ) ) ) {
                        $this->lineNum++;
                        $matches = [];
@@ -170,7 +130,7 @@ class TestFileReader implements Iterator {
                                        $this->checkSection( 'text' );
                                        $this->checkSection( 'article' );
 
-                                       $this->parserTest->addArticle(
+                                       $this->addArticle(
                                                ParserTestRunner::chomp( $this->sectionData['article'] ),
                                                $this->sectionData['text'], $this->lineNum );
 
@@ -186,7 +146,7 @@ class TestFileReader implements Iterator {
                                                $line = trim( $line );
 
                                                if ( $line ) {
-                                                       $this->delayedParserTest->requireHook( $line );
+                                                       $this->addRequirement( 'hook', $line );
                                                }
                                        }
 
@@ -202,7 +162,7 @@ class TestFileReader implements Iterator {
                                                $line = trim( $line );
 
                                                if ( $line ) {
-                                                       $this->delayedParserTest->requireFunctionHook( $line );
+                                                       $this->addRequirement( 'functionHook', $line );
                                                }
                                        }
 
@@ -218,7 +178,7 @@ class TestFileReader implements Iterator {
                                                $line = trim( $line );
 
                                                if ( $line ) {
-                                                       $this->delayedParserTest->requireTransparentHook( $line );
+                                                       $this->addRequirement( 'transparentHook', $line );
                                                }
                                        }
 
@@ -229,14 +189,8 @@ class TestFileReader implements Iterator {
 
                                if ( $this->section == 'end' ) {
                                        $this->checkSection( 'test' );
-                                       do {
-                                               if ( $this->setupCurrentTest() ) {
-                                                       return true;
-                                               }
-                                       } while ( $this->nextSubTest > 0 );
-                                       # go on to next test (since this was disabled)
+                                       $this->addCurrentTest();
                                        $this->clearSection();
-                                       $this->delayedParserTest->reset();
                                        continue;
                                }
 
@@ -254,8 +208,6 @@ class TestFileReader implements Iterator {
                                $this->sectionData[$this->section] .= $line;
                        }
                }
-
-               return false;
        }
 
        /**
@@ -320,5 +272,18 @@ class TestFileReader implements Iterator {
 
                return array_values( $tokens )[0];
        }
+
+       private function addArticle( $name, $text, $line ) {
+               $this->articles[] = [
+                       'name' => $name,
+                       'text' => $text,
+                       'line' => $line,
+                       'file' => $this->file
+               ];
+       }
+
+       private function addRequirement( $type, $name ) {
+               $this->requirements[$type][$name] = true;
+       }
 }
 
index 2608420..70215b6 100644 (file)
  * @ingroup Testing
  */
 
-class TestRecorder implements ITestRecorder {
-       public $parent;
-       public $term;
+/**
+ * Interface to record parser test results.
+ *
+ * The TestRecorder is an class hierarchy to record the result of
+ * MediaWiki parser tests. One should call start() before running the
+ * full parser tests and end() once all the tests have been finished.
+ * After each test, you should use record() to keep track of your tests
+ * results. Finally, report() is used to generate a summary of your
+ * test run, one could dump it to the console for human consumption or
+ * register the result in a database for tracking purposes.
+ *
+ * @since 1.22
+ */
+abstract class TestRecorder {
 
-       function __construct( $parent ) {
-               $this->parent = $parent;
-               $this->term = $parent->term;
+       /**
+        * Called at beginning of the parser test run
+        */
+       public function start() {
        }
 
-       function start() {
-               $this->total = 0;
-               $this->success = 0;
+       /**
+        * Called before starting a test
+        */
+       public function startTest( $test ) {
        }
 
-       function record( $test, $subtest, $result ) {
-               $this->total++;
-               $this->success += ( $result ? 1 : 0 );
+       /**
+        * Called before starting an input file
+        */
+       public function startSuite( $path ) {
        }
 
-       function end() {
-               // dummy
+       /**
+        * Called after ending an input file
+        */
+       public function endSuite( $path ) {
        }
 
-       function report() {
-               if ( $this->total > 0 ) {
-                       $this->reportPercentage( $this->success, $this->total );
-               } else {
-                       throw new MWException( "No tests found.\n" );
-               }
+       /**
+        * Called after each test
+        * @param array $test
+        * @param ParserTestResult $result
+        */
+       public function record( $test, ParserTestResult $result ) {
        }
 
-       function reportPercentage( $success, $total ) {
-               $ratio = wfPercent( 100 * $success / $total );
-               print $this->term->color( 1 ) . "Passed $success of $total tests ($ratio)... ";
+       /**
+        * Show a warning to the user
+        */
+       public function warning( $message ) {
+       }
 
-               if ( $success == $total ) {
-                       print $this->term->color( 32 ) . "ALL TESTS PASSED!";
-               } else {
-                       $failed = $total - $success;
-                       print $this->term->color( 31 ) . "$failed tests failed!";
-               }
+       /**
+        * Mark a test skipped
+        */
+       public function skipped( $test, $subtest ) {
+       }
 
-               print $this->term->reset() . "\n";
+       /**
+        * Called before finishing the test run
+        */
+       public function report() {
+       }
 
-               return ( $success == $total );
+       /**
+        * Called at the end of the parser test run
+        */
+       public function end() {
        }
+
 }
 
index ddf839e..7437053 100644 (file)
@@ -22,13 +22,16 @@ class ParserFuzzTest extends Maintenance {
        }
 
        function finalSetup() {
-               require_once __DIR__ . '/../common/TestsAutoLoader.php';
+               self::requireTestsAutoloader();
+               TestSetup::applyInitialConfig();
        }
 
        function execute() {
                $files = $this->getOption( 'file', [ __DIR__ . '/parserTests.txt' ] );
                $this->seed = intval( $this->getOption( 'seed', 1 ) ) - 1;
-               $this->parserTest = new ParserTestRunner;
+               $this->parserTest = new ParserTestRunner(
+                       new MultiTestRecorder,
+                       [] );
                $this->fuzzTest( $files );
        }
 
@@ -38,11 +41,23 @@ class ParserFuzzTest extends Maintenance {
         * @param array $filenames
         */
        function fuzzTest( $filenames ) {
-               $GLOBALS['wgContLang'] = Language::factory( 'en' );
                $dict = $this->getFuzzInput( $filenames );
                $dictSize = strlen( $dict );
                $logMaxLength = log( $this->maxFuzzTestLength );
-               $this->parserTest->setupDatabase();
+
+               $teardown = $this->parserTest->staticSetup();
+               $teardown = $this->parserTest->setupDatabase( $teardown );
+               $teardown = $this->parserTest->setupUploads( $teardown );
+
+               $fakeTest = [
+                       'test' => '',
+                       'desc' => '',
+                       'input' => '',
+                       'result' => '',
+                       'options' => '',
+                       'config' => ''
+               ];
+
                ini_set( 'memory_limit', $this->memoryLimit * 1048576 * 2 );
 
                $numTotal = 0;
@@ -64,7 +79,7 @@ class ParserFuzzTest extends Maintenance {
                                $input .= substr( $dict, $offset, $hairLength );
                        }
 
-                       $this->parserTest->setupGlobals();
+                       $perTestTeardown = $this->parserTest->perTestSetup( $fakeTest );
                        $parser = $this->parserTest->getParser();
 
                        // Run the test
@@ -85,8 +100,7 @@ class ParserFuzzTest extends Maintenance {
                        }
 
                        $numTotal++;
-                       $this->parserTest->teardownGlobals();
-                       $parser->__destruct();
+                       ScopedCallback::consume( $perTestTeardown );
 
                        if ( $numTotal % 100 == 0 ) {
                                $usage = intval( memory_get_usage( true ) / $this->memoryLimit / 1048576 * 100 );
index 48c2606..8d5f072 100644 (file)
  * @ingroup Testing
  */
 
+// Some methods which are discouraged for normal code throw exceptions unless
+// we declare this is just a test.
 define( 'MW_PARSER_TEST', true );
 
-$options = [ 'quick', 'color', 'quiet', 'help', 'show-output',
-       'record', 'run-disabled', 'run-parsoid', 'dwdiff', 'mark-ws' ];
-$optionsWithArgs = [ 'regex', 'filter', 'seed', 'setversion', 'file', 'norm' ];
-
-require_once __DIR__ . '/../../maintenance/commandLine.inc';
-require_once __DIR__ . '/../common/TestsAutoLoader.php';
-
-if ( isset( $options['help'] ) ) {
-       echo <<<ENDS
-MediaWiki $wgVersion parser test suite
-Usage: php parserTests.php [options...]
-
-Options:
-  --quick          Suppress diff output of failed tests
-  --quiet          Suppress notification of passed tests (shows only failed tests)
-  --show-output    Show expected and actual output
-  --color[=yes|no] Override terminal detection and force color output on or off
-                   use wgCommandLineDarkBg = true; if your term is dark
-  --regex          Only run tests whose descriptions which match given regex
-  --filter         Alias for --regex
-  --file=<testfile> Run test cases from a custom file instead of parserTests.txt
-  --record         Record tests in database
-  --compare        Compare with recorded results, without updating the database.
-  --setversion     When using --record, set the version string to use (useful
-                   with git-svn so that you can get the exact revision)
-  --keep-uploads   Re-use the same upload directory for each test, don't delete it
-  --run-disabled   run disabled tests
-  --run-parsoid    run parsoid tests (normally disabled)
-  --dwdiff         Use dwdiff to display diff output
-  --mark-ws        Mark whitespace in diffs by replacing it with symbols
-  --norm=<funcs>   Apply a comma-separated list of normalization functions to
-                   both the expected and actual output in order to resolve
-                   irrelevant differences. The accepted normalization functions
-                   are: removeTbody to remove <tbody> tags; and trimWhitespace
-                   to trim whitespace from the start and end of text nodes.
-  --use-tidy-config Use the wiki's Tidy configuration instead of known-good
-                   defaults.
-  --help           Show this help message
-
-ENDS;
-       exit( 0 );
-}
+require __DIR__ . '/../../maintenance/Maintenance.php';
+
+class ParserTestsMaintenance extends Maintenance {
+       function __construct() {
+               parent::__construct();
+               $this->addDescription( 'Run parser tests' );
 
-# Cases of weird db corruption were encountered when running tests on earlyish
-# versions of SQLite
-if ( $wgDBtype == 'sqlite' ) {
-       $db = wfGetDB( DB_MASTER );
-       $version = $db->getServerVersion();
-       if ( version_compare( $version, '3.6' ) < 0 ) {
-               die( "Parser tests require SQLite version 3.6 or later, you have $version\n" );
+               $this->addOption( 'quick', 'Suppress diff output of failed tests' );
+               $this->addOption( 'quiet', 'Suppress notification of passed tests (shows only failed tests)' );
+               $this->addOption( 'show-output', 'Show expected and actual output' );
+               $this->addOption( 'color', '[=yes|no] Override terminal detection and force ' .
+                       'color output on or off. Use wgCommandLineDarkBg = true; if your term is dark',
+                       false, true );
+               $this->addOption( 'regex', 'Only run tests whose descriptions which match given regex',
+                       false, true );
+               $this->addOption( 'filter', 'Alias for --regex', false, true );
+               $this->addOption( 'file', 'Run test cases from a custom file instead of parserTests.txt',
+                       false, true, false, true );
+               $this->addOption( 'record', 'Record tests in database' );
+               $this->addOption( 'compare', 'Compare with recorded results, without updating the database.' );
+               $this->addOption( 'setversion', 'When using --record, set the version string to use (useful' .
+                       'with "git rev-parse HEAD" to get the exact revision)',
+                       false, true );
+               $this->addOption( 'keep-uploads', 'Re-use the same upload directory for each ' .
+                       'test, don\'t delete it' );
+               $this->addOption( 'file-backend', 'Use the file backend with the given name,' .
+                       'and upload files to it, instead of creating a mock file backend.', false, true );
+               $this->addOption( 'upload-dir', 'Specify the upload directory to use. Useful in ' .
+                       'conjunction with --keep-uploads. Causes a real (non-mock) file backend to ' .
+                       'be used.', false, true );
+               $this->addOption( 'run-disabled', 'run disabled tests' );
+               $this->addOption( 'run-parsoid', 'run parsoid tests (normally disabled)' );
+               $this->addOption( 'dwdiff', 'Use dwdiff to display diff output' );
+               $this->addOption( 'mark-ws', 'Mark whitespace in diffs by replacing it with symbols' );
+               $this->addOption( 'norm', 'Apply a comma-separated list of normalization functions to ' .
+                       'both the expected and actual output in order to resolve ' .
+                       'irrelevant differences. The accepted normalization functions ' .
+                       'are: removeTbody to remove <tbody> tags; and trimWhitespace ' .
+                       'to trim whitespace from the start and end of text nodes.',
+                       false, true );
+               $this->addOption( 'use-tidy-config', 'Use the wiki\'s Tidy configuration instead of known-good' .
+                       'defaults.' );
        }
-}
 
-$tester = new ParserTestRunner( $options );
+       public function finalSetup() {
+               parent::finalSetup();
+               self::requireTestsAutoloader();
+               TestSetup::applyInitialConfig();
+       }
 
-if ( isset( $options['file'] ) ) {
-       $files = [ $options['file'] ];
-} else {
-       // Default parser tests and any set from extensions or local config
-       $files = $wgParserTestFiles;
-}
+       public function execute() {
+               global $wgParserTestFiles, $wgDBtype;
+
+               // Cases of weird db corruption were encountered when running tests on earlyish
+               // versions of SQLite
+               if ( $wgDBtype == 'sqlite' ) {
+                       $db = wfGetDB( DB_MASTER );
+                       $version = $db->getServerVersion();
+                       if ( version_compare( $version, '3.6' ) < 0 ) {
+                               die( "Parser tests require SQLite version 3.6 or later, you have $version\n" );
+                       }
+               }
+
+               // Print out software version to assist with locating regressions
+               $version = SpecialVersion::getVersion( 'nodb' );
+               echo "This is MediaWiki version {$version}.\n\n";
+
+               // Only colorize output if stdout is a terminal.
+               $color = !wfIsWindows() && Maintenance::posix_isatty( 1 );
+
+               if ( $this->hasOption( 'color' ) ) {
+                       switch ( $this->getOption( 'color' ) ) {
+                               case 'no':
+                                       $color = false;
+                                       break;
+                               case 'yes':
+                               default:
+                                       $color = true;
+                                       break;
+                       }
+               }
+
+               $record = $this->hasOption( 'record' );
+               $compare = $this->hasOption( 'compare' );
+
+               $regex = $this->getOption( 'filter', $this->getOption( 'regex', false ) );
+               if ( $regex !== false ) {
+                       $regex = "/$regex/i";
 
-# Print out software version to assist with locating regressions
-$version = SpecialVersion::getVersion( 'nodb' );
-echo "This is MediaWiki version {$version}.\n\n";
+                       if ( $record ) {
+                               echo "Warning: --record cannot be used with --regex, disabling --record\n";
+                               $record = false;
+                       }
+               }
+
+               $term = $color
+                       ? new AnsiTermColorer()
+                       : new DummyTermColorer();
+
+               $recorder = new MultiTestRecorder;
+
+               $recorder->addRecorder( new ParserTestPrinter(
+                       $term,
+                       [
+                               'showDiffs' => !$this->hasOption( 'quick' ),
+                               'showProgress' => !$this->hasOption( 'quiet' ),
+                               'showFailure' => !$this->hasOption( 'quiet' )
+                                               || ( !$record && !$compare ), // redundant output
+                               'showOutput' => $this->hasOption( 'show-output' ),
+                               'useDwdiff' => $this->hasOption( 'dwdiff' ),
+                               'markWhitespace' => $this->hasOption( 'mark-ws' ),
+                       ]
+               ) );
+
+               $recorderLB = false;
+               if ( $record || $compare ) {
+                       $recorderLB = wfGetLBFactory()->newMainLB();
+                       // This connection will have the wiki's table prefix, not parsertest_
+                       $recorderDB = $recorderLB->getConnection( DB_MASTER );
+
+                       // Add recorder before previewer because recorder will create the
+                       // DB table if it doesn't exist
+                       if ( $record ) {
+                               $recorder->addRecorder( new DbTestRecorder( $recorderDB ) );
+                       }
+                       $recorder->addRecorder( new DbTestPreviewer(
+                               $recorderDB,
+                               function ( $name ) use ( $regex ) {
+                                       // Filter reports of old tests by the filter regex
+                                       if ( $regex === false ) {
+                                               return true;
+                                       } else {
+                                               return (bool)preg_match( $regex, $name );
+                                       }
+                               } ) );
+               }
+
+               // Default parser tests and any set from extensions or local config
+               $files = $this->getOption( 'file', $wgParserTestFiles );
+
+               $norm = $this->hasOption( 'norm' ) ? explode( ',', $this->getOption( 'norm' ) ) : [];
+
+               $tester = new ParserTestRunner( $recorder, [
+                       'norm' => $norm,
+                       'regex' => $regex,
+                       'keep-uploads' => $this->hasOption( 'keep-uploads' ),
+                       'run-disabled' => $this->hasOption( 'run-disabled' ),
+                       'run-parsoid' => $this->hasOption( 'run-parsoid' ),
+                       'use-tidy-config' => $this->hasOption( 'use-tidy-config' ),
+                       'file-backend' => $this->getOption( 'file-backend' ),
+                       'upload-dir' => $this->getOption( 'upload-dir' ),
+               ] );
+
+               $ok = $tester->runTestsFromFiles( $files );
+               if ( $recorderLB ) {
+                       $recorderLB->closeAll();
+               }
+               return $ok ? 0 : 1;
+       }
+}
 
-$ok = $tester->runTestsFromFiles( $files );
-exit( $ok ? 0 : 1 );
+$maintClass = 'ParserTestsMaintenance';
+require_once RUN_MAINTENANCE_IF_MAIN;
index 3e9fef8..c1c421b 100644 (file)
@@ -14646,7 +14646,7 @@ cat
 !! wikitext
 [[Category:MediaWiki User's Guide]]
 !! html
-<a href="/wiki/Category:MediaWiki_User%27s_Guide" title="Category:MediaWiki User's Guide">MediaWiki User's Guide</a>
+cat=MediaWiki_User's_Guide sort=
 !! end
 
 !! test
@@ -14665,7 +14665,7 @@ cat
 !! wikitext
 [[Category:MediaWiki User's Guide|Foo]]
 !! html
-<a href="/wiki/Category:MediaWiki_User%27s_Guide" title="Category:MediaWiki User's Guide">MediaWiki User's Guide</a>
+cat=MediaWiki_User's_Guide sort=Foo
 !! end
 
 !! test
@@ -14675,7 +14675,7 @@ cat
 !! wikitext
 [[Category:MediaWiki User's Guide|MediaWiki User's Guide]]
 !! html
-<a href="/wiki/Category:MediaWiki_User%27s_Guide" title="Category:MediaWiki User's Guide">MediaWiki User's Guide</a>
+cat=MediaWiki_User's_Guide sort=MediaWiki User's Guide
 !! end
 
 !! test
@@ -19785,7 +19785,7 @@ language=sr cat
 !! wikitext
 [[Category:МедиаWики Усер'с Гуиде]]
 !! html
-<a href="/wiki/%D0%9A%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%98%D0%B0:MediaWiki_User%27s_Guide" title="Категорија:MediaWiki User's Guide">MediaWiki User's Guide</a>
+cat=МедиаWики_Усер'с_Гуиде sort=
 !! end
 
 
@@ -19814,7 +19814,7 @@ parsoid=wt2html
 !! wikitext
 [[A]][[Category:分类]]
 !! html/php
-<a href="/wiki/Category:%E5%88%86%E7%B1%BB" title="Category:分类">分类</a>
+cat=分类 sort=
 !! html/parsoid
 <p><a rel="mw:WikiLink" href="A" title="A">A</a></p>
 <link rel="mw:PageProp/Category" href="Category:分类"/>
index 8503393..d34e183 100644 (file)
@@ -43,26 +43,17 @@ coverage:
 
 parser:
        ${PU} --group Parser
-parserfuzz:
-       @echo "******************************************************************"
-       @echo "* This WILL kill your computer by eating all memory AND all swap *"
-       @echo "*                                                                *"
-       @echo "* If you are on a production machine. ABORT NOW!!                *"
-       @echo "*  Press control+C to stop                                       *"
-       @echo "*                                                                *"
-       @echo "******************************************************************"
-       ${PU} --group Parser,ParserFuzz
 noparser:
-       ${PU} --exclude-group Parser,Broken,ParserFuzz,Stub
+       ${PU} --exclude-group Parser,Broken,Stub
 
 safe:
-       ${PU} --exclude-group Broken,ParserFuzz,Destructive,Stub
+       ${PU} --exclude-group Broken,Destructive,Stub
 
 databaseless:
-       ${PU} --exclude-group Broken,ParserFuzz,Destructive,Database,Stub
+       ${PU} --exclude-group Broken,Destructive,Database,Stub
 
 database:
-       ${PU} --exclude-group Broken,ParserFuzz,Destructive,Stub --group Database
+       ${PU} --exclude-group Broken,Destructive,Stub --group Database
 
 list-groups:
        ${PU} --list-groups
index 18b9dc8..920dbb3 100644 (file)
@@ -122,9 +122,8 @@ abstract class MediaWikiTestCase extends PHPUnit_Framework_TestCase {
        public static function setUpBeforeClass() {
                parent::setUpBeforeClass();
 
-               // NOTE: Usually, PHPUnitMaintClass::finalSetup already called this,
-               // but let's make doubly sure.
-               self::prepareServices( new GlobalVarConfig() );
+               // Get the service locator, and reset services if it's not done already
+               self::$serviceLocator = self::prepareServices( new GlobalVarConfig() );
        }
 
        /**
@@ -180,28 +179,26 @@ abstract class MediaWikiTestCase extends PHPUnit_Framework_TestCase {
         *
         * @param Config $bootstrapConfig The bootstrap config to use with the new
         *        MediaWikiServices. Only used for the first call to this method.
+        * @return MediaWikiServices
         */
        public static function prepareServices( Config $bootstrapConfig ) {
-               static $servicesPrepared = false;
+               static $services = null;
 
-               if ( $servicesPrepared ) {
-                       return;
-               } else {
-                       $servicesPrepared = true;
+               if ( !$services ) {
+                       $services = self::resetGlobalServices( $bootstrapConfig );
                }
-
-               self::resetGlobalServices( $bootstrapConfig );
+               return $services;
        }
 
        /**
         * Reset global services, and install testing environment.
         * This is the testing equivalent of MediaWikiServices::resetGlobalInstance().
         * This should only be used to set up the testing environment, not when
-        * running unit tests. Use overrideMwServices() for that.
+        * running unit tests. Use MediaWikiTestCase::overrideMwServices() for that.
         *
         * @see MediaWikiServices::resetGlobalInstance()
         * @see prepareServices()
-        * @see overrideMwServices()
+        * @see MediaWikiTestCase::overrideMwServices()
         *
         * @param Config|null $bootstrapConfig The bootstrap config to use with the new
         *        MediaWikiServices.
@@ -214,11 +211,12 @@ abstract class MediaWikiTestCase extends PHPUnit_Framework_TestCase {
 
                MediaWikiServices::resetGlobalInstance( $testConfig );
 
-               self::$serviceLocator = MediaWikiServices::getInstance();
+               $serviceLocator = MediaWikiServices::getInstance();
                self::installTestServices(
                        $oldConfigFactory,
-                       self::$serviceLocator
+                       $serviceLocator
                );
+               return $serviceLocator;
        }
 
        /**
@@ -1122,15 +1120,15 @@ abstract class MediaWikiTestCase extends PHPUnit_Framework_TestCase {
         * @throws MWException If the database table prefix is already $prefix
         */
        public static function setupTestDB( DatabaseBase $db, $prefix ) {
+               if ( self::$dbSetup ) {
+                       return;
+               }
+
                if ( $db->tablePrefix() === $prefix ) {
                        throw new MWException(
                                'Cannot run unit tests, the database prefix is already "' . $prefix . '"' );
                }
 
-               if ( self::$dbSetup ) {
-                       return;
-               }
-
                // TODO: the below should be re-written as soon as LBFactory, LoadBalancer,
                // and DatabaseBase no longer use global state.
 
index 0e219ca..698bd0b 100644 (file)
 <?php
 
-use MediaWiki\MediaWikiServices;
-
 /**
- * Although marked as a stub, can work independently.
+ * This is the TestCase subclass for running a single parser test via the
+ * ParserTestRunner integration test system.
+ *
+ * Note: the following groups are not used by PHPUnit.
+ * The list in ParserTestFileSuite::__construct() is used instead.
  *
  * @group Database
  * @group Parser
- * @group Stub
  *
  * @todo covers tags
  */
-class ParserIntegrationTest extends MediaWikiTestCase {
-       static protected $articles = []; // Array of test articles defined by the tests
-       /* The data provider is run on a different instance than the test, so it must be static
-        * When running tests from several files, all tests will see all articles.
-        */
-       static protected $backendToUse;
-
-       public $keepUploads = false;
-       public $runDisabled = false;
-       public $runParsoid = false;
-       public $regex = '';
-       public $showProgress = true;
-       public $savedWeirdGlobals = [];
-       public $savedGlobals = [];
-       public $hooks = [];
-       public $functionHooks = [];
-       public $transparentHooks = [];
-
-       /**
-        * @var DjVuSupport
-        */
-       private $djVuSupport;
-       /**
-        * @var TidySupport
-        */
-       private $tidySupport;
-
-       protected $file = false;
-
-       public static function setUpBeforeClass() {
-               // Inject ParserTestRunner well-known interwikis
-               ParserTestRunner::setupInterwikis();
-       }
-
-       protected function setUp() {
-               global $wgNamespaceAliases, $wgContLang;
-               global $wgHooks, $IP;
-
-               parent::setUp();
-
-               // Setup CLI arguments
-               if ( $this->getCliArg( 'regex' ) ) {
-                       $this->regex = $this->getCliArg( 'regex' );
-               } else {
-                       # Matches anything
-                       $this->regex = '';
-               }
-
-               $this->keepUploads = $this->getCliArg( 'keep-uploads' );
-
-               $tmpGlobals = [];
-
-               $tmpGlobals['wgLanguageCode'] = 'en';
-               $tmpGlobals['wgContLang'] = Language::factory( 'en' );
-               $tmpGlobals['wgSitename'] = 'MediaWiki';
-               $tmpGlobals['wgServer'] = 'http://example.org';
-               $tmpGlobals['wgServerName'] = 'example.org';
-               $tmpGlobals['wgScriptPath'] = '';
-               $tmpGlobals['wgScript'] = '/index.php';
-               $tmpGlobals['wgResourceBasePath'] = '';
-               $tmpGlobals['wgStylePath'] = '/skins';
-               $tmpGlobals['wgExtensionAssetsPath'] = '/extensions';
-               $tmpGlobals['wgArticlePath'] = '/wiki/$1';
-               $tmpGlobals['wgActionPaths'] = [];
-               $tmpGlobals['wgVariantArticlePath'] = false;
-               $tmpGlobals['wgEnableUploads'] = true;
-               $tmpGlobals['wgUploadNavigationUrl'] = false;
-               $tmpGlobals['wgThumbnailScriptPath'] = false;
-               $tmpGlobals['wgLocalFileRepo'] = [
-                       'class' => 'LocalRepo',
-                       'name' => 'local',
-                       'url' => 'http://example.com/images',
-                       'hashLevels' => 2,
-                       'transformVia404' => false,
-                       'backend' => 'local-backend'
-               ];
-               $tmpGlobals['wgForeignFileRepos'] = [];
-               $tmpGlobals['wgDefaultExternalStore'] = [];
-               $tmpGlobals['wgParserCacheType'] = CACHE_NONE;
-               $tmpGlobals['wgCapitalLinks'] = true;
-               $tmpGlobals['wgNoFollowLinks'] = true;
-               $tmpGlobals['wgNoFollowDomainExceptions'] = [ 'no-nofollow.org' ];
-               $tmpGlobals['wgExternalLinkTarget'] = false;
-               $tmpGlobals['wgThumbnailScriptPath'] = false;
-               $tmpGlobals['wgUseImageResize'] = true;
-               $tmpGlobals['wgAllowExternalImages'] = true;
-               $tmpGlobals['wgRawHtml'] = false;
-               $tmpGlobals['wgExperimentalHtmlIds'] = false;
-               $tmpGlobals['wgAdaptiveMessageCache'] = true;
-               $tmpGlobals['wgUseDatabaseMessages'] = true;
-               $tmpGlobals['wgLocaltimezone'] = 'UTC';
-               $tmpGlobals['wgGroupPermissions'] = [
-                       '*' => [
-                               'createaccount' => true,
-                               'read' => true,
-                               'edit' => true,
-                               'createpage' => true,
-                               'createtalk' => true,
-               ] ];
-               $tmpGlobals['wgNamespaceProtection'] = [ NS_MEDIAWIKI => 'editinterface' ];
-
-               $tmpGlobals['wgParser'] = new StubObject(
-                       'wgParser', $GLOBALS['wgParserConf']['class'],
-                       [ $GLOBALS['wgParserConf'] ] );
-
-               $tmpGlobals['wgFileExtensions'][] = 'svg';
-               $tmpGlobals['wgSVGConverter'] = 'rsvg';
-               $tmpGlobals['wgSVGConverters']['rsvg'] =
-                       '$path/rsvg-convert -w $width -h $height -o $output $input';
-
-               if ( $GLOBALS['wgStyleDirectory'] === false ) {
-                       $tmpGlobals['wgStyleDirectory'] = "$IP/skins";
-               }
-
-               $tmpHooks = $wgHooks;
-               $tmpHooks['ParserTestParser'][] = 'ParserTestParserHook::setup';
-               $tmpHooks['ParserGetVariableValueTs'][] = 'ParserTestRunner::getFakeTimestamp';
-               $tmpGlobals['wgHooks'] = $tmpHooks;
-               # add a namespace shadowing a interwiki link, to test
-               # proper precedence when resolving links. (bug 51680)
-               $tmpGlobals['wgExtraNamespaces'] = [
-                       100 => 'MemoryAlpha',
-                       101 => 'MemoryAlpha_talk'
-               ];
-
-               $tmpGlobals['wgLocalInterwikis'] = [ 'local', 'mi' ];
-               # "extra language links"
-               # see https://gerrit.wikimedia.org/r/111390
-               $tmpGlobals['wgExtraInterlanguageLinkPrefixes'] = [ 'mul' ];
-
-               // DjVu support
-               $this->djVuSupport = new DjVuSupport();
-               // Tidy support
-               $this->tidySupport = new TidySupport();
-               $tmpGlobals['wgTidyConfig'] = $this->tidySupport->getConfig();
-               $tmpGlobals['wgUseTidy'] = false;
-
-               $this->setMwGlobals( $tmpGlobals );
-
-               $this->savedWeirdGlobals['image_alias'] = $wgNamespaceAliases['Image'];
-               $this->savedWeirdGlobals['image_talk_alias'] = $wgNamespaceAliases['Image_talk'];
-
-               $wgNamespaceAliases['Image'] = NS_FILE;
-               $wgNamespaceAliases['Image_talk'] = NS_FILE_TALK;
-
-               MWNamespace::getCanonicalNamespaces( true ); # reset namespace cache
-               $wgContLang->resetNamespaces(); # reset namespace cache
-               ParserTestRunner::resetTitleServices();
-               MediaWikiServices::getInstance()->disableService( 'MediaHandlerFactory' );
-               MediaWikiServices::getInstance()->redefineService(
-                       'MediaHandlerFactory',
-                       function() {
-                               return new MockMediaHandlerFactory();
-                       }
-               );
-       }
-
-       protected function tearDown() {
-               global $wgNamespaceAliases, $wgContLang;
-
-               $wgNamespaceAliases['Image'] = $this->savedWeirdGlobals['image_alias'];
-               $wgNamespaceAliases['Image_talk'] = $this->savedWeirdGlobals['image_talk_alias'];
-
-               MWTidy::destroySingleton();
-
-               // Restore backends
-               RepoGroup::destroySingleton();
-               FileBackendGroup::destroySingleton();
-
-               // Remove temporary pages from the link cache
-               LinkCache::singleton()->clear();
-
-               // Restore message cache (temporary pages and $wgUseDatabaseMessages)
-               MessageCache::destroyInstance();
-               MediaWikiServices::getInstance()->resetServiceForTesting( 'MediaHandlerFactory' );
-
-               parent::tearDown();
-
-               MWNamespace::getCanonicalNamespaces( true ); # reset namespace cache
-               $wgContLang->resetNamespaces(); # reset namespace cache
-       }
-
-       public static function tearDownAfterClass() {
-               ParserTestRunner::tearDownInterwikis();
-               parent::tearDownAfterClass();
-       }
-
-       function addDBDataOnce() {
-               # disabled for performance
-               # $this->tablesUsed[] = 'image';
-
-               # Update certain things in site_stats
-               $this->db->insert( 'site_stats',
-                       [ 'ss_row_id' => 1, 'ss_images' => 2, 'ss_good_articles' => 1 ],
-                       __METHOD__,
-                       [ 'IGNORE' ]
-               );
-
-               $user = User::newFromId( 0 );
-               LinkCache::singleton()->clear(); # Avoids the odd failure at creating the nullRevision
-
-               # Upload DB table entries for files.
-               # We will upload the actual files later. Note that if anything causes LocalFile::load()
-               # to be triggered before then, it will break via maybeUpgrade() setting the fileExists
-               # member to false and storing it in cache.
-               # note that the size/width/height/bits/etc of the file
-               # are actually set by inspecting the file itself; the arguments
-               # to recordUpload2 have no effect.  That said, we try to make things
-               # match up so it is less confusing to readers of the code & tests.
-               $image = wfLocalFile( Title::makeTitle( NS_FILE, 'Foobar.jpg' ) );
-               if ( !$this->db->selectField( 'image', '1', [ 'img_name' => $image->getName() ] ) ) {
-                       $image->recordUpload2(
-                               '', // archive name
-                               'Upload of some lame file',
-                               'Some lame file',
-                               [
-                                       'size' => 7881,
-                                       'width' => 1941,
-                                       'height' => 220,
-                                       'bits' => 8,
-                                       'media_type' => MEDIATYPE_BITMAP,
-                                       'mime' => 'image/jpeg',
-                                       'metadata' => serialize( [] ),
-                                       'sha1' => Wikimedia\base_convert( '1', 16, 36, 31 ),
-                                       'fileExists' => true ],
-                               $this->db->timestamp( '20010115123500' ), $user
-                       );
-               }
-
-               $image = wfLocalFile( Title::makeTitle( NS_FILE, 'Thumb.png' ) );
-               if ( !$this->db->selectField( 'image', '1', [ 'img_name' => $image->getName() ] ) ) {
-                       $image->recordUpload2(
-                               '', // archive name
-                               'Upload of some lame thumbnail',
-                               'Some lame thumbnail',
-                               [
-                                       'size' => 22589,
-                                       'width' => 135,
-                                       'height' => 135,
-                                       'bits' => 8,
-                                       'media_type' => MEDIATYPE_BITMAP,
-                                       'mime' => 'image/png',
-                                       'metadata' => serialize( [] ),
-                                       'sha1' => Wikimedia\base_convert( '2', 16, 36, 31 ),
-                                       'fileExists' => true ],
-                               $this->db->timestamp( '20130225203040' ), $user
-                       );
-               }
-
-               # This image will be blacklisted in [[MediaWiki:Bad image list]]
-               $image = wfLocalFile( Title::makeTitle( NS_FILE, 'Bad.jpg' ) );
-               if ( !$this->db->selectField( 'image', '1', [ 'img_name' => $image->getName() ] ) ) {
-                       $image->recordUpload2(
-                               '', // archive name
-                               'zomgnotcensored',
-                               'Borderline image',
-                               [
-                                       'size' => 12345,
-                                       'width' => 320,
-                                       'height' => 240,
-                                       'bits' => 24,
-                                       'media_type' => MEDIATYPE_BITMAP,
-                                       'mime' => 'image/jpeg',
-                                       'metadata' => serialize( [] ),
-                                       'sha1' => Wikimedia\base_convert( '3', 16, 36, 31 ),
-                                       'fileExists' => true ],
-                               $this->db->timestamp( '20010115123500' ), $user
-                       );
-               }
-               $image = wfLocalFile( Title::makeTitle( NS_FILE, 'Foobar.svg' ) );
-               if ( !$this->db->selectField( 'image', '1', [ 'img_name' => $image->getName() ] ) ) {
-                       $image->recordUpload2( '', 'Upload of some lame SVG', 'Some lame SVG', [
-                                       'size'        => 12345,
-                                       'width'       => 240,
-                                       'height'      => 180,
-                                       'bits'        => 0,
-                                       'media_type'  => MEDIATYPE_DRAWING,
-                                       'mime'        => 'image/svg+xml',
-                                       'metadata'    => serialize( [] ),
-                                       'sha1'        => Wikimedia\base_convert( '', 16, 36, 31 ),
-                                       'fileExists'  => true
-                       ], $this->db->timestamp( '20010115123500' ), $user );
-               }
-
-               $image = wfLocalFile( Title::makeTitle( NS_FILE, 'Video.ogv' ) );
-               if ( !$this->db->selectField( 'image', '1', [ 'img_name' => $image->getName() ] ) ) {
-                       $image->recordUpload2( '', 'A pretty movie', 'Will it play', [
-                                       'size'        => 12345,
-                                       'width'       => 320,
-                                       'height'      => 240,
-                                       'bits'        => 0,
-                                       'media_type'  => MEDIATYPE_VIDEO,
-                                       'mime'        => 'application/ogg',
-                                       'metadata'    => serialize( [] ),
-                                       'sha1'        => Wikimedia\base_convert( '', 16, 36, 32 ),
-                                       'fileExists'  => true
-                       ], $this->db->timestamp( '20010115123500' ), $user );
-               }
-
-               $image = wfLocalFile( Title::makeTitle( NS_FILE, 'Audio.oga' ) );
-               if ( !$this->db->selectField( 'image', '1', [ 'img_name' => $image->getName() ] ) ) {
-                       $image->recordUpload2( '', 'An awesome hitsong ', 'Will it play', [
-                                       'size'        => 12345,
-                                       'width'       => 0,
-                                       'height'      => 0,
-                                       'bits'        => 0,
-                                       'media_type'  => MEDIATYPE_AUDIO,
-                                       'mime'        => 'application/ogg',
-                                       'metadata'    => serialize( [] ),
-                                       'sha1'        => Wikimedia\base_convert( '', 16, 36, 32 ),
-                                       'fileExists'  => true
-                       ], $this->db->timestamp( '20010115123500' ), $user );
-               }
-
-               # A DjVu file
-               $image = wfLocalFile( Title::makeTitle( NS_FILE, 'LoremIpsum.djvu' ) );
-               if ( !$this->db->selectField( 'image', '1', [ 'img_name' => $image->getName() ] ) ) {
-                       $image->recordUpload2( '', 'Upload a DjVu', 'A DjVu', [
-                               'size' => 3249,
-                               'width' => 2480,
-                               'height' => 3508,
-                               'bits' => 0,
-                               'media_type' => MEDIATYPE_BITMAP,
-                               'mime' => 'image/vnd.djvu',
-                               'metadata' => '<?xml version="1.0" ?>
-<!DOCTYPE DjVuXML PUBLIC "-//W3C//DTD DjVuXML 1.1//EN" "pubtext/DjVuXML-s.dtd">
-<DjVuXML>
-<HEAD></HEAD>
-<BODY><OBJECT height="3508" width="2480">
-<PARAM name="DPI" value="300" />
-<PARAM name="GAMMA" value="2.2" />
-</OBJECT>
-<OBJECT height="3508" width="2480">
-<PARAM name="DPI" value="300" />
-<PARAM name="GAMMA" value="2.2" />
-</OBJECT>
-<OBJECT height="3508" width="2480">
-<PARAM name="DPI" value="300" />
-<PARAM name="GAMMA" value="2.2" />
-</OBJECT>
-<OBJECT height="3508" width="2480">
-<PARAM name="DPI" value="300" />
-<PARAM name="GAMMA" value="2.2" />
-</OBJECT>
-<OBJECT height="3508" width="2480">
-<PARAM name="DPI" value="300" />
-<PARAM name="GAMMA" value="2.2" />
-</OBJECT>
-</BODY>
-</DjVuXML>',
-                               'sha1' => Wikimedia\base_convert( '', 16, 36, 31 ),
-                               'fileExists' => true
-                       ], $this->db->timestamp( '20140115123600' ), $user );
-               }
-       }
-
-       // ParserTestRunner setup/teardown functions
-
-       /**
-        * Set up the global variables for a consistent environment for each test.
-        * Ideally this should replace the global configuration entirely.
-        * @param array $opts
-        * @param string $config
-        * @return RequestContext
-        */
-       protected function setupGlobals( $opts = [], $config = '' ) {
-               global $wgFileBackends;
-               # Find out values for some special options.
-               $lang =
-                       self::getOptionValue( 'language', $opts, 'en' );
-               $variant =
-                       self::getOptionValue( 'variant', $opts, false );
-               $maxtoclevel =
-                       self::getOptionValue( 'wgMaxTocLevel', $opts, 999 );
-               $linkHolderBatchSize =
-                       self::getOptionValue( 'wgLinkHolderBatchSize', $opts, 1000 );
-
-               $uploadDir = $this->getUploadDir();
-               if ( $this->getCliArg( 'use-filebackend' ) ) {
-                       if ( self::$backendToUse ) {
-                               $backend = self::$backendToUse;
-                       } else {
-                               $name = $this->getCliArg( 'use-filebackend' );
-                               $useConfig = [];
-                               foreach ( $wgFileBackends as $conf ) {
-                                       if ( $conf['name'] == $name ) {
-                                               $useConfig = $conf;
-                                       }
-                               }
-                               $useConfig['name'] = 'local-backend'; // swap name
-                               unset( $useConfig['lockManager'] );
-                               unset( $useConfig['fileJournal'] );
-                               $class = $useConfig['class'];
-                               self::$backendToUse = new $class( $useConfig );
-                               $backend = self::$backendToUse;
-                       }
-               } else {
-                       # Replace with a mock. We do not care about generating real
-                       # files on the filesystem, just need to expose the file
-                       # informations.
-                       $backend = new MockFileBackend( [
-                               'name' => 'local-backend',
-                               'wikiId' => wfWikiID()
-                       ] );
-               }
-
-               $settings = [
-                       'wgLocalFileRepo' => [
-                               'class' => 'LocalRepo',
-                               'name' => 'local',
-                               'url' => 'http://example.com/images',
-                               'hashLevels' => 2,
-                               'transformVia404' => false,
-                               'backend' => $backend
-                       ],
-                       'wgEnableUploads' => self::getOptionValue( 'wgEnableUploads', $opts, true ),
-                       'wgLanguageCode' => $lang,
-                       'wgDBprefix' => $this->db->getType() != 'oracle' ? 'unittest_' : 'ut_',
-                       'wgRawHtml' => self::getOptionValue( 'wgRawHtml', $opts, false ),
-                       'wgNamespacesWithSubpages' => [ NS_MAIN => isset( $opts['subpage'] ) ],
-                       'wgAllowExternalImages' => self::getOptionValue( 'wgAllowExternalImages', $opts, true ),
-                       'wgThumbLimits' => [ self::getOptionValue( 'thumbsize', $opts, 180 ) ],
-                       'wgMaxTocLevel' => $maxtoclevel,
-                       'wgUseTeX' => isset( $opts['math'] ) || isset( $opts['texvc'] ),
-                       'wgMathDirectory' => $uploadDir . '/math',
-                       'wgDefaultLanguageVariant' => $variant,
-                       'wgLinkHolderBatchSize' => $linkHolderBatchSize,
-                       'wgUseTidy' => false,
-                       'wgTidyConfig' => isset( $opts['tidy'] ) ? $this->tidySupport->getConfig() : null
-               ];
-
-               if ( $config ) {
-                       $configLines = explode( "\n", $config );
-
-                       foreach ( $configLines as $line ) {
-                               list( $var, $value ) = explode( '=', $line, 2 );
-
-                               $settings[$var] = eval( "return $value;" ); // ???
-                       }
-               }
-
-               $this->savedGlobals = [];
-
-               /** @since 1.20 */
-               Hooks::run( 'ParserTestGlobals', [ &$settings ] );
-
-               $langObj = Language::factory( $lang );
-               $settings['wgContLang'] = $langObj;
-               $settings['wgLang'] = $langObj;
-
-               $context = new RequestContext();
-               $settings['wgOut'] = $context->getOutput();
-               $settings['wgUser'] = $context->getUser();
-               $settings['wgRequest'] = $context->getRequest();
-
-               // We (re)set $wgThumbLimits to a single-element array above.
-               $context->getUser()->setOption( 'thumbsize', 0 );
-
-               foreach ( $settings as $var => $val ) {
-                       if ( array_key_exists( $var, $GLOBALS ) ) {
-                               $this->savedGlobals[$var] = $GLOBALS[$var];
-                       }
-
-                       $GLOBALS[$var] = $val;
-               }
-
-               MWTidy::destroySingleton();
-               MagicWord::clearCache();
-
-               # The entries saved into RepoGroup cache with previous globals will be wrong.
-               RepoGroup::destroySingleton();
-               FileBackendGroup::destroySingleton();
-
-               # Create dummy files in storage
-               $this->setupUploads();
-
-               # Publish the articles after we have the final language set
-               $this->publishTestArticles();
-
-               MessageCache::destroyInstance();
-
-               return $context;
-       }
-
-       /**
-        * Get an FS upload directory (only applies to FSFileBackend)
-        *
-        * @return string The directory
-        */
-       protected function getUploadDir() {
-               if ( $this->keepUploads ) {
-                       // Don't use getNewTempDirectory() as this is meant to persist
-                       $dir = wfTempDir() . '/mwParser-images';
-
-                       if ( is_dir( $dir ) ) {
-                               return $dir;
-                       }
-               } else {
-                       $dir = $this->getNewTempDirectory();
-               }
-
-               if ( file_exists( $dir ) ) {
-                       wfDebug( "Already exists!\n" );
-
-                       return $dir;
-               }
-
-               return $dir;
-       }
-
-       /**
-        * Create a dummy uploads directory which will contain a couple
-        * of files in order to pass existence tests.
-        *
-        * @return string The directory
-        */
-       protected function setupUploads() {
-               global $IP;
+class ParserIntegrationTest extends PHPUnit_Framework_TestCase {
+       /** @var array */
+       private $ptTest;
 
-               $base = $this->getBaseDir();
-               $backend = RepoGroup::singleton()->getLocalRepo()->getBackend();
-               $backend->prepare( [ 'dir' => "$base/local-public/3/3a" ] );
-               $backend->store( [
-                       'src' => "$IP/tests/phpunit/data/parser/headbg.jpg",
-                       'dst' => "$base/local-public/3/3a/Foobar.jpg"
-               ] );
-               $backend->prepare( [ 'dir' => "$base/local-public/e/ea" ] );
-               $backend->store( [
-                       'src' => "$IP/tests/phpunit/data/parser/wiki.png",
-                       'dst' => "$base/local-public/e/ea/Thumb.png"
-               ] );
-               $backend->prepare( [ 'dir' => "$base/local-public/0/09" ] );
-               $backend->store( [
-                       'src' => "$IP/tests/phpunit/data/parser/headbg.jpg",
-                       'dst' => "$base/local-public/0/09/Bad.jpg"
-               ] );
-               $backend->prepare( [ 'dir' => "$base/local-public/5/5f" ] );
-               $backend->store( [
-                       'src' => "$IP/tests/phpunit/data/parser/LoremIpsum.djvu",
-                       'dst' => "$base/local-public/5/5f/LoremIpsum.djvu"
-               ] );
+       /** @var ParserTestRunner */
+       private $ptRunner;
 
-               // No helpful SVG file to copy, so make one ourselves
-               $data = '<?xml version="1.0" encoding="utf-8"?>' .
-                       '<svg xmlns="http://www.w3.org/2000/svg"' .
-                       ' version="1.1" width="240" height="180"/>';
+       /** @var ScopedCallback */
+       private $ptTeardownScope;
 
-               $backend->prepare( [ 'dir' => "$base/local-public/f/ff" ] );
-               $backend->quickCreate( [
-                       'content' => $data, 'dst' => "$base/local-public/f/ff/Foobar.svg"
-               ] );
+       public function __construct( $runner, $fileName, $test ) {
+               parent::__construct( 'testParse', [ '[details omitted]' ],
+                       basename( $fileName ) . ': ' . $test['desc'] );
+               $this->ptTest = $test;
+               $this->ptRunner = $runner;
        }
 
-       /**
-        * Restore default values and perform any necessary clean-up
-        * after each test runs.
-        */
-       protected function teardownGlobals() {
-               $this->teardownUploads();
-
-               foreach ( $this->savedGlobals as $var => $val ) {
-                       $GLOBALS[$var] = $val;
-               }
-       }
-
-       /**
-        * Remove the dummy uploads directory
-        */
-       private function teardownUploads() {
-               if ( $this->keepUploads ) {
-                       return;
-               }
-
-               $backend = RepoGroup::singleton()->getLocalRepo()->getBackend();
-               if ( $backend instanceof MockFileBackend ) {
-                       # In memory backend, so dont bother cleaning them up.
-                       return;
-               }
-
-               $base = $this->getBaseDir();
-               // delete the files first, then the dirs.
-               self::deleteFiles(
-                       [
-                               "$base/local-public/3/3a/Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/1000px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/100px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/120px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/1280px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/137px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/1500px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/177px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/180px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/200px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/206px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/20px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/220px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/265px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/270px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/274px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/300px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/30px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/330px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/353px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/360px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/400px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/40px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/440px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/442px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/450px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/50px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/600px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/640px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/70px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/75px-Foobar.jpg",
-                               "$base/local-thumb/3/3a/Foobar.jpg/960px-Foobar.jpg",
-
-                               "$base/local-public/e/ea/Thumb.png",
-
-                               "$base/local-public/0/09/Bad.jpg",
-
-                               "$base/local-public/5/5f/LoremIpsum.djvu",
-                               "$base/local-thumb/5/5f/LoremIpsum.djvu/page2-2480px-LoremIpsum.djvu.jpg",
-                               "$base/local-thumb/5/5f/LoremIpsum.djvu/page2-3720px-LoremIpsum.djvu.jpg",
-                               "$base/local-thumb/5/5f/LoremIpsum.djvu/page2-4960px-LoremIpsum.djvu.jpg",
-
-                               "$base/local-public/f/ff/Foobar.svg",
-                               "$base/local-thumb/f/ff/Foobar.svg/180px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/2000px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/270px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/3000px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/360px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/4000px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/langde-180px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/langde-270px-Foobar.svg.png",
-                               "$base/local-thumb/f/ff/Foobar.svg/langde-360px-Foobar.svg.png",
-
-                               "$base/local-public/math/f/a/5/fa50b8b616463173474302ca3e63586b.png",
-                       ]
-               );
-       }
-
-       /**
-        * Delete the specified files, if they exist.
-        * @param array $files Full paths to files to delete.
-        */
-       private static function deleteFiles( $files ) {
-               $backend = RepoGroup::singleton()->getLocalRepo()->getBackend();
-               foreach ( $files as $file ) {
-                       $backend->delete( [ 'src' => $file ], [ 'force' => 1 ] );
-               }
-               foreach ( $files as $file ) {
-                       $tmp = FileBackend::parentStoragePath( $file );
-                       while ( $tmp ) {
-                               if ( !$backend->clean( [ 'dir' => $tmp ] )->isOK() ) {
-                                       break;
-                               }
-                               $tmp = FileBackend::parentStoragePath( $tmp );
-                       }
-               }
-       }
-
-       protected function getBaseDir() {
-               return 'mwstore://local-backend';
-       }
-
-       public function parserTestProvider() {
-               if ( $this->file === false ) {
-                       global $wgParserTestFiles;
-                       $this->file = $wgParserTestFiles[0];
-               }
-
-               return new TestFileDataProvider( $this->file, $this );
+       public function testParse() {
+               $this->ptRunner->getRecorder()->setTestCase( $this );
+               $result = $this->ptRunner->runTest( $this->ptTest );
+               $this->assertEquals( $result->expected, $result->actual );
        }
 
-       /**
-        * Set the file from whose tests will be run by this instance
-        * @param string $filename
-        */
-       public function setParserTestFile( $filename ) {
-               $this->file = $filename;
+       public function setUp() {
+               $this->ptTeardownScope = $this->ptRunner->staticSetup();
        }
 
-       /**
-        * @group medium
-        * @group ParserTests
-        * @dataProvider parserTestProvider
-        * @param string $desc
-        * @param string $input
-        * @param string $result
-        * @param array $opts
-        * @param array $config
-        */
-       public function testParserTest( $desc, $input, $result, $opts, $config ) {
-               if ( $this->regex != '' && !preg_match( '/' . $this->regex . '/', $desc ) ) {
-                       $this->assertTrue( true ); // XXX: don't flood output with "test made no assertions"
-                       // $this->markTestSkipped( 'Filtered out by the user' );
-                       $this->teardownGlobals();
-                       return;
-               }
-
-               if ( !$this->isWikitextNS( NS_MAIN ) ) {
-                       // parser tests frequently assume that the main namespace contains wikitext.
-                       // @todo When setting up pages, force the content model. Only skip if
-                       //        $wgtContentModelUseDB is false.
-                       $this->teardownGlobals();
-                       $this->markTestSkipped( "Main namespace does not support wikitext,"
-                               . "skipping parser test: $desc" );
-               }
-
-               wfDebug( "Running parser test: $desc\n" );
-
-               $opts = $this->parseOptions( $opts );
-               $context = $this->setupGlobals( $opts, $config );
-
-               $user = $context->getUser();
-               $options = ParserOptions::newFromContext( $context );
-
-               if ( isset( $opts['title'] ) ) {
-                       $titleText = $opts['title'];
-               } else {
-                       $titleText = 'Parser test';
-               }
-
-               $local = isset( $opts['local'] );
-               $preprocessor = isset( $opts['preprocessor'] ) ? $opts['preprocessor'] : null;
-               $parser = $this->getParser( $preprocessor );
-
-               $title = Title::newFromText( $titleText );
-
-               # Parser test requiring math. Make sure texvc is executable
-               # or just skip such tests.
-               if ( isset( $opts['math'] ) || isset( $opts['texvc'] ) ) {
-                       global $wgTexvc;
-
-                       if ( !isset( $wgTexvc ) ) {
-                               $this->teardownGlobals();
-                               $this->markTestSkipped( "SKIPPED: \$wgTexvc is not set" );
-                       } elseif ( !is_executable( $wgTexvc ) ) {
-                               $this->teardownGlobals();
-                               $this->markTestSkipped( "SKIPPED: texvc binary does not exist"
-                                       . " or is not executable.\n"
-                                       . "Current configuration is:\n\$wgTexvc = '$wgTexvc'" );
-                       }
-               }
-
-               if ( isset( $opts['djvu'] ) ) {
-                       if ( !$this->djVuSupport->isEnabled() ) {
-                               $this->teardownGlobals();
-                               $this->markTestSkipped( "SKIPPED: djvu binaries do not exist or are not executable.\n" );
-                       }
-               }
-
-               if ( isset( $opts['tidy'] ) ) {
-                       if ( !$this->tidySupport->isEnabled() ) {
-                               $this->teardownGlobals();
-                               $this->markTestSkipped( "SKIPPED: tidy extension is not installed.\n" );
-                       } else {
-                               $options->setTidy( true );
-                       }
-               }
-
-               if ( isset( $opts['pst'] ) ) {
-                       $out = $parser->preSaveTransform( $input, $title, $user, $options );
-               } elseif ( isset( $opts['msg'] ) ) {
-                       $out = $parser->transformMsg( $input, $options, $title );
-               } elseif ( isset( $opts['section'] ) ) {
-                       $section = $opts['section'];
-                       $out = $parser->getSection( $input, $section );
-               } elseif ( isset( $opts['replace'] ) ) {
-                       $section = $opts['replace'][0];
-                       $replace = $opts['replace'][1];
-                       $out = $parser->replaceSection( $input, $section, $replace );
-               } elseif ( isset( $opts['comment'] ) ) {
-                       $out = Linker::formatComment( $input, $title, $local );
-               } elseif ( isset( $opts['preload'] ) ) {
-                       $out = $parser->getPreloadText( $input, $title, $options );
-               } else {
-                       $output = $parser->parse( $input, $title, $options, true, true, 1337 );
-                       $output->setTOCEnabled( !isset( $opts['notoc'] ) );
-                       $out = $output->getText();
-                       if ( isset( $opts['tidy'] ) ) {
-                               $out = preg_replace( '/\s+$/', '', $out );
-                       }
-
-                       if ( isset( $opts['showtitle'] ) ) {
-                               if ( $output->getTitleText() ) {
-                                       $title = $output->getTitleText();
-                               }
-
-                               $out = "$title\n$out";
-                       }
-
-                       if ( isset( $opts['showindicators'] ) ) {
-                               $indicators = '';
-                               foreach ( $output->getIndicators() as $id => $content ) {
-                                       $indicators .= "$id=$content\n";
-                               }
-                               $out = $indicators . $out;
-                       }
-
-                       if ( isset( $opts['ill'] ) ) {
-                               $out = implode( ' ', $output->getLanguageLinks() );
-                       } elseif ( isset( $opts['cat'] ) ) {
-                               $outputPage = $context->getOutput();
-                               $outputPage->addCategoryLinks( $output->getCategories() );
-                               $cats = $outputPage->getCategoryLinks();
-
-                               if ( isset( $cats['normal'] ) ) {
-                                       $out = implode( ' ', $cats['normal'] );
-                               } else {
-                                       $out = '';
-                               }
-                       }
-                       $parser->mPreprocessor = null;
-               }
-
-               $this->teardownGlobals();
-
-               $this->assertEquals( $result, $out, $desc );
-       }
-
-       /**
-        * Get a Parser object
-        * @param Preprocessor $preprocessor
-        * @return Parser
-        */
-       function getParser( $preprocessor = null ) {
-               global $wgParserConf;
-
-               $class = $wgParserConf['class'];
-               $parser = new $class( [ 'preprocessorClass' => $preprocessor ] + $wgParserConf );
-
-               Hooks::run( 'ParserTestParser', [ &$parser ] );
-
-               return $parser;
-       }
-
-       // Various action functions
-
-       public function addArticle( $name, $text, $line ) {
-               self::$articles[$name] = [ $text, $line ];
-       }
-
-       public function publishTestArticles() {
-               if ( empty( self::$articles ) ) {
-                       return;
-               }
-
-               foreach ( self::$articles as $name => $info ) {
-                       list( $text, $line ) = $info;
-                       ParserTestRunner::addArticle( $name, $text, $line, 'ignoreduplicate' );
-               }
-       }
-
-       /**
-        * Steal a callback function from the primary parser, save it for
-        * application to our scary parser. If the hook is not installed,
-        * abort processing of this file.
-        *
-        * @param string $name
-        * @return bool True if tag hook is present
-        */
-       public function requireHook( $name ) {
-               global $wgParser;
-               $wgParser->firstCallInit(); // make sure hooks are loaded.
-               return isset( $wgParser->mTagHooks[$name] );
-       }
-
-       public function requireFunctionHook( $name ) {
-               global $wgParser;
-               $wgParser->firstCallInit(); // make sure hooks are loaded.
-               return isset( $wgParser->mFunctionHooks[$name] );
-       }
-
-       public function requireTransparentHook( $name ) {
-               global $wgParser;
-               $wgParser->firstCallInit(); // make sure hooks are loaded.
-               return isset( $wgParser->mTransparentTagHooks[$name] );
-       }
-
-       // Various "cleanup" functions
-
-       /**
-        * Remove last character if it is a newline
-        * @param string $s
-        * @return string
-        */
-       public function removeEndingNewline( $s ) {
-               if ( substr( $s, -1 ) === "\n" ) {
-                       return substr( $s, 0, -1 );
-               } else {
-                       return $s;
-               }
-       }
-
-       // Test options parser functions
-
-       protected function parseOptions( $instring ) {
-               $opts = [];
-               // foo
-               // foo=bar
-               // foo="bar baz"
-               // foo=[[bar baz]]
-               // foo=bar,"baz quux"
-               $regex = '/\b
-                       ([\w-]+)                                                # Key
-                       \b
-                       (?:\s*
-                               =                                               # First sub-value
-                               \s*
-                               (
-                                       "
-                                               [^"]*                   # Quoted val
-                                       "
-                               |
-                                       \[\[
-                                               [^]]*                   # Link target
-                                       \]\]
-                               |
-                                       [\w-]+                          # Plain word
-                               )
-                               (?:\s*
-                                       ,                                       # Sub-vals 1..N
-                                       \s*
-                                       (
-                                               "[^"]*"                 # Quoted val
-                                       |
-                                               \[\[[^]]*\]\]   # Link target
-                                       |
-                                               [\w-]+                  # Plain word
-                                       )
-                               )*
-                       )?
-                       /x';
-
-               if ( preg_match_all( $regex, $instring, $matches, PREG_SET_ORDER ) ) {
-                       foreach ( $matches as $bits ) {
-                               array_shift( $bits );
-                               $key = strtolower( array_shift( $bits ) );
-                               if ( count( $bits ) == 0 ) {
-                                       $opts[$key] = true;
-                               } elseif ( count( $bits ) == 1 ) {
-                                       $opts[$key] = $this->cleanupOption( array_shift( $bits ) );
-                               } else {
-                                       // Array!
-                                       $opts[$key] = array_map( [ $this, 'cleanupOption' ], $bits );
-                               }
-                       }
-               }
-
-               return $opts;
-       }
-
-       protected function cleanupOption( $opt ) {
-               if ( substr( $opt, 0, 1 ) == '"' ) {
-                       return substr( $opt, 1, -1 );
-               }
-
-               if ( substr( $opt, 0, 2 ) == '[[' ) {
-                       return substr( $opt, 2, -2 );
-               }
-
-               return $opt;
-       }
-
-       /**
-        * Use a regex to find out the value of an option
-        * @param string $key Name of option val to retrieve
-        * @param array $opts Options array to look in
-        * @param mixed $default Default value returned if not found
-        * @return mixed
-        */
-       protected static function getOptionValue( $key, $opts, $default ) {
-               $key = strtolower( $key );
-
-               if ( isset( $opts[$key] ) ) {
-                       return $opts[$key];
-               } else {
-                       return $default;
+       public function tearDown() {
+               if ( $this->ptTeardownScope ) {
+                       ScopedCallback::consume( $this->ptTeardownScope );
                }
        }
 }
index acd8575..d817104 100755 (executable)
@@ -16,12 +16,10 @@ require_once dirname( dirname( __DIR__ ) ) . "/maintenance/Maintenance.php";
 class PHPUnitMaintClass extends Maintenance {
 
        public static $additionalOptions = [
-               'regex' => false,
                'file' => false,
                'use-filebackend' => false,
                'use-bagostuff' => false,
                'use-jobqueue' => false,
-               'keep-uploads' => false,
                'use-normal-tables' => false,
                'reuse-db' => false,
                'wiki' => false,
@@ -42,22 +40,10 @@ class PHPUnitMaintClass extends Maintenance {
                        false, # not required
                        false # no arg needed
                );
-               $this->addOption(
-                       'regex',
-                       'Only run parser tests that match the given regex.',
-                       false,
-                       true
-               );
                $this->addOption( 'file', 'File describing parser tests.', false, true );
                $this->addOption( 'use-filebackend', 'Use filebackend', false, true );
                $this->addOption( 'use-bagostuff', 'Use bagostuff', false, true );
                $this->addOption( 'use-jobqueue', 'Use jobqueue', false, true );
-               $this->addOption(
-                       'keep-uploads',
-                       'Re-use the same upload directory for each test, don\'t delete it.',
-                       false,
-                       false
-               );
                $this->addOption( 'use-normal-tables', 'Use normal DB tables.', false, false );
                $this->addOption(
                        'reuse-db', 'Init DB only if tables are missing and keep after finish.',
@@ -69,104 +55,10 @@ class PHPUnitMaintClass extends Maintenance {
        public function finalSetup() {
                parent::finalSetup();
 
-               global $wgMainCacheType, $wgMessageCacheType, $wgParserCacheType, $wgMainWANCache;
-               global $wgMainStash;
-               global $wgLanguageConverterCacheType, $wgUseDatabaseMessages;
-               global $wgLocaltimezone, $wgLocalisationCacheConf;
-               global $wgDevelopmentWarnings;
-               global $wgSessionProviders, $wgSessionPbkdf2Iterations;
-               global $wgJobTypeConf;
-               global $wgAuthManagerConfig, $wgAuth;
-
                // Inject test autoloader
-               require_once __DIR__ . '/../common/TestsAutoLoader.php';
-
-               // wfWarn should cause tests to fail
-               $wgDevelopmentWarnings = true;
-
-               // Make sure all caches and stashes are either disabled or use
-               // in-process cache only to prevent tests from using any preconfigured
-               // cache meant for the local wiki from outside the test run.
-               // See also MediaWikiTestCase::run() which mocks CACHE_DB and APC.
-
-               // Disabled in DefaultSettings, override local settings
-               $wgMainWANCache =
-               $wgMainCacheType = CACHE_NONE;
-               // Uses CACHE_ANYTHING in DefaultSettings, use hash instead of db
-               $wgMessageCacheType =
-               $wgParserCacheType =
-               $wgSessionCacheType =
-               $wgLanguageConverterCacheType = 'hash';
-               // Uses db-replicated in DefaultSettings
-               $wgMainStash = 'hash';
-               // Use memory job queue
-               $wgJobTypeConf = [
-                       'default' => [ 'class' => 'JobQueueMemory', 'order' => 'fifo' ],
-               ];
-
-               $wgUseDatabaseMessages = false; # Set for future resets
-
-               // Assume UTC for testing purposes
-               $wgLocaltimezone = 'UTC';
-
-               $wgLocalisationCacheConf['storeClass'] = 'LCStoreNull';
-
-               // Generic MediaWiki\Session\SessionManager configuration for tests
-               // We use CookieSessionProvider because things might be expecting
-               // cookies to show up in a FauxRequest somewhere.
-               $wgSessionProviders = [
-                       [
-                               'class' => MediaWiki\Session\CookieSessionProvider::class,
-                               'args' => [ [
-                                       'priority' => 30,
-                                       'callUserSetCookiesHook' => true,
-                               ] ],
-                       ],
-               ];
-
-               // Single-iteration PBKDF2 session secret derivation, for speed.
-               $wgSessionPbkdf2Iterations = 1;
-
-               // Generic AuthManager configuration for testing
-               $wgAuthManagerConfig = [
-                       'preauth' => [],
-                       'primaryauth' => [
-                               [
-                                       'class' => MediaWiki\Auth\TemporaryPasswordPrimaryAuthenticationProvider::class,
-                                       'args' => [ [
-                                               'authoritative' => false,
-                                       ] ],
-                               ],
-                               [
-                                       'class' => MediaWiki\Auth\LocalPasswordPrimaryAuthenticationProvider::class,
-                                       'args' => [ [
-                                               'authoritative' => true,
-                                       ] ],
-                               ],
-                       ],
-                       'secondaryauth' => [],
-               ];
-               $wgAuth = new MediaWiki\Auth\AuthManagerAuthPlugin();
-
-               // Bug 44192 Do not attempt to send a real e-mail
-               Hooks::clear( 'AlternateUserMailer' );
-               Hooks::register(
-                       'AlternateUserMailer',
-                       function () {
-                               return false;
-                       }
-               );
-               // xdebug's default of 100 is too low for MediaWiki
-               ini_set( 'xdebug.max_nesting_level', 1000 );
-
-               // Bug T116683 serialize_precision of 100
-               // may break testing against floating point values
-               // treated with PHP's serialize()
-               ini_set( 'serialize_precision', 17 );
+               self::requireTestsAutoloader();
 
-               // TODO: we should call MediaWikiTestCase::prepareServices( new GlobalVarConfig() ) here.
-               // But PHPUnit may not be loaded yet, so we have to wait until just
-               // before PHPUnit_TextUI_Command::main() is executed.
+               TestSetup::applyInitialConfig();
        }
 
        public function execute() {
index 6443ec4..16299aa 100644 (file)
        <testsuites>
                <testsuite name="includes">
                        <directory>includes</directory>
+                       <!-- Parser tests must be invoked via their suite -->
+                       <exclude>includes/parser/ParserIntegrationTest.php</exclude>
                </testsuite>
                <testsuite name="languages">
                        <directory>languages</directory>
                </testsuite>
                <testsuite name="parsertests">
-                       <file>suites/ParserTestTopLevelSuite.php</file>
+                       <file>suites/CoreParserTestSuite.php</file>
                        <file>suites/ExtensionsParserTestSuite.php</file>
                </testsuite>
                <testsuite name="skins">
@@ -55,7 +57,6 @@
                <exclude>
                        <group>Utility</group>
                        <group>Broken</group>
-                       <group>ParserFuzz</group>
                        <group>Stub</group>
                </exclude>
        </groups>
diff --git a/tests/phpunit/suites/CoreParserTestSuite.php b/tests/phpunit/suites/CoreParserTestSuite.php
new file mode 100644 (file)
index 0000000..e48a116
--- /dev/null
@@ -0,0 +1,10 @@
+<?php
+
+class CoreParserTestSuite extends PHPUnit_Framework_TestSuite {
+
+       public static function suite() {
+               return ParserTestTopLevelSuite::suite( ParserTestTopLevelSuite::CORE_ONLY );
+       }
+
+}
+
diff --git a/tests/phpunit/suites/ParserTestFileSuite.php b/tests/phpunit/suites/ParserTestFileSuite.php
new file mode 100644 (file)
index 0000000..d3129b1
--- /dev/null
@@ -0,0 +1,28 @@
+<?php
+
+/**
+ * This is the suite class for running tests within a single .txt source file.
+ * It is not invoked directly. Use --filter to select files, or
+ * use parserTests.php.
+ */
+class ParserTestFileSuite extends PHPUnit_Framework_TestSuite {
+       private $ptRunner;
+       private $ptFileName;
+       private $ptFileInfo;
+
+       function __construct( $runner, $name, $fileName ) {
+               parent::__construct( $name );
+               $this->ptRunner = $runner;
+               $this->ptFileName = $fileName;
+               $this->ptFileInfo = TestFileReader::read( $this->ptFileName );
+
+               foreach ( $this->ptFileInfo['tests'] as $test ) {
+                       $this->addTest( new ParserIntegrationTest( $runner, $fileName, $test ),
+                               [ 'Database', 'Parser' ] );
+               }
+       }
+
+       function setUp() {
+               $this->ptRunner->addArticles( $this->ptFileInfo[ 'articles'] );
+       }
+}
index 36ecf73..4284a77 100644 (file)
@@ -1,5 +1,4 @@
 <?php
-require_once __DIR__ . '/../includes/parser/ParserIntegrationTest.php';
 
 /**
  * The UnitTest must be either a class that inherits from MediaWikiTestCase
@@ -10,7 +9,12 @@ require_once __DIR__ . '/../includes/parser/ParserIntegrationTest.php';
  * @group ParserTests
  * @group Database
  */
-class ParserTestTopLevelSuite {
+class ParserTestTopLevelSuite extends PHPUnit_Framework_TestSuite {
+       /** @var ParserTestRunner */
+       private $ptRunner;
+
+       /** @var ScopedCallback */
+       private $ptTeardownScope;
 
        /**
         * @defgroup filtering_constants Filtering constants
@@ -52,6 +56,15 @@ class ParserTestTopLevelSuite {
         * @return PHPUnit_Framework_TestSuite
         */
        public static function suite( $flags = self::CORE_ONLY ) {
+               return new self( $flags );
+       }
+
+       function __construct( $flags ) {
+               parent::__construct();
+
+               $this->ptRecorder = new PhpunitTestRecorder;
+               $this->ptRunner = new ParserTestRunner( $this->ptRecorder );
+
                if ( is_string( $flags ) ) {
                        $flags = self::CORE_ONLY;
                }
@@ -83,7 +96,6 @@ class ParserTestTopLevelSuite {
                self::debug( 'parser tests files: '
                        . implode( ' ', $filesToTest ) );
 
-               $suite = new PHPUnit_Framework_TestSuite;
                $testList = [];
                $counter = 0;
                foreach ( $filesToTest as $fileName ) {
@@ -93,7 +105,6 @@ class ParserTestTopLevelSuite {
                        // things, which is good enough for our purposes.
                        $extensionName = basename( dirname( $fileName ) );
                        $testsName = $extensionName . '__' . basename( $fileName, '.txt' );
-                       $escapedFileName = strtr( $fileName, [ "'" => "\\'", '\\' => '\\\\' ] );
                        $parserTestClassName = ucfirst( $testsName );
 
                        // Official spec for class names: http://php.net/manual/en/language.oop5.basic.php
@@ -102,30 +113,38 @@ class ParserTestTopLevelSuite {
                                preg_replace( '/[^a-zA-Z0-9_\x7f-\xff]/', '_', $parserTestClassName );
 
                        if ( isset( $testList[$parserTestClassName] ) ) {
-                               // If a conflict happens, gives a very unclear fatal.
-                               // So as a last ditch effort to prevent that eventuality, if there
-                               // is a conflict, append a number.
+                               // If there is a conflict, append a number.
                                $counter++;
                                $parserTestClassName .= $counter;
                        }
                        $testList[$parserTestClassName] = true;
-                       $parserTestClassDefinition = <<<EOT
-/**
- * @group Database
- * @group Parser
- * @group ParserTests
- * @group ParserTests_$parserTestClassName
- */
-class $parserTestClassName extends ParserIntegrationTest {
-       protected \$file = '$escapedFileName';
-}
-EOT;
 
-                       eval( $parserTestClassDefinition );
+                       // Previously we actually created a class here, with eval(). We now
+                       // just override the name.
+
                        self::debug( "Adding test class $parserTestClassName" );
-                       $suite->addTestSuite( $parserTestClassName );
+                       $this->addTest( new ParserTestFileSuite(
+                               $this->ptRunner, $parserTestClassName, $fileName ) );
+               }
+       }
+
+       public function setUp() {
+               wfDebug( __METHOD__ );
+               $db = wfGetDB( DB_MASTER );
+               $type = $db->getType();
+               $prefix = $type === 'oracle' ?
+                       MediaWikiTestCase::ORA_DB_PREFIX : MediaWikiTestCase::DB_PREFIX;
+               MediaWikiTestCase::setupTestDB( $db, $prefix );
+               $teardown = $this->ptRunner->setDatabase( $db );
+               $teardown = $this->ptRunner->setupUploads( $teardown );
+               $this->ptTeardownScope = $teardown;
+       }
+
+       public function tearDown() {
+               wfDebug( __METHOD__ );
+               if ( $this->ptTeardownScope ) {
+                       ScopedCallback::consume( $this->ptTeardownScope );
                }
-               return $suite;
        }
 
        /**