Fix support for TestSwarm on SpecialJavaScriptTest/qunit
authorKrinkle <ttijhof@wikimedia.org>
Sun, 25 Mar 2012 01:09:58 +0000 (03:09 +0200)
committerKrinkle <ttijhof@wikimedia.org>
Sun, 25 Mar 2012 01:15:19 +0000 (03:15 +0200)
So far we've still been using ./tests/qunit/index.html in TestSwarm, today I've tested locally
to submit a url to SpecialJavaScriptTest instead and made a bunch of browsers join my swarm,
quite a few problems popped up. This commit fixes those issues so that we can actually use
SpecialJavaScriptTest in TestSwarm.

* Add QUnit configuration variable for TestSwarm's inject.js

  In order to use TestSwarm, the urls that TestSwarm loads in clients that has
  the QUnit test suite running on it need to include a little javascript.
  This inject.js registers hooks with QUnit to listen for when the test suite finishes
  and contacts the parent window (TestSwarm loads the qunit test suite url in an iframe)
  to submit the results. Previously I included a copy of TestSwarm's inject.js in
  ./tests/qunit/data and in our testrunner.js a relative link to that.
  However this is currently breaking because it is an outdated version. Updating brings
  no good since someone else might use their own TestSwarm would could still run on an old
  version etc. The TestSwarm submitted too always expects that it's own inject.js is used,
  not some snapshot copy. I've removed the copy of it in MediaWiki and instead added a
  configuration option to point to wherever the you want is located.

  Also, since the old static index.html version of the unit test can't retrieve PHP based
  content, this means TestSwarm submissions through the old static index.html are no longer
  supported. Only through the new Special:JavaScriptTest from now on. I'll probably remove
  the whole index.html soon-ish as it's getting quite annoying to maintain all that by hand,
  and it's been superseded in everywhere imaginable now anyway. Even not used anymore by
  intergration.mediawiki.org because that's been quiet since the Git-switchover.., and when
  we update it, we can update it to point to the new SpecialPage instead.

* OutputPage::allowClickjacking() on SpecialJavaScriptTest/qunit.

  When initially testing the TestSwarm setup to submit SpecialJavaScriptTest/qunit urls
  (instead of the old ./tests/qunit/index.html) it was failing due to an iframe DENY.
  This was a bit odd since `$wgBreakFrames = false;` by default, and although
  `$wgEditPageFrameOptions = 'DENY';` by default, it wasn't obvious at all that that value
  ("DENY") is used for all OutputPages by default (as supposed to just action=edit and the
  like). This is because OutputPage has  mPreventClickjacking=true by default and when it's
  true-ish it uses $wgEditPageFrameOptions for the X-Frame-Options.

* 'position' => true; for the mediawiki.tests.qunit.suites module.

  QUnit has a hook for "done". Which is called when QUnit.start() is called and all queued tests
  have been executed. QUnit.start() is automatically called on window.onload by QUnit.

  TestSwarm uses QUnit's hook system to hook into the QUnit "done" event, and at that point
  takes the stats, submits them to TestSwarm and go on with the next job.

  When testing locally, I got semi-random failures reporting that only 0/0 tests were
  successfully ran in IE6. This is because when QUnit.start (and consequently QUnit.done)
  are first called, apparently no test suites had finished downloading and/or execution yet
  (the bottom queue is asynchronous, and doesn't postpone domready nor window.onload).

  When normally viewing Special:JavaScriptTest/qunit this doesn't break anything, because if
  QUnit start/done is in the past and another module(), test(), or equal() etc. is called it just
  picks up again and adds more results to the page and calls QUnit.done() again.
  However in the case of the TestSwarm embed, it submits the results after the first done() and
  cleans up the iframe. So I'm making mediawiki.tests.qunit.suites a blocking module instead, so
  that there will only be one QUnit.start/done and that's the one that TestSwarm gets and after
  which TestSwarm can safely garbage the iframe.

  This means that basically all test suite modules and the original modules they are testing will
  be loaded from the head. Shouldn't have any side effects, but might cause minor breakage in
  future in modules that badly assume they're being put on the bottom.
  I'm not considering that a bug in the test, it'll just help catch that bad code sooner :),
  it's a test suite after all.

(Yay, my first Git commit to MediaWiki core)

Change-Id: I83f83377f2183b6deb4e901af602ac9a5628558b

includes/DefaultSettings.php
includes/specials/SpecialJavaScriptTest.php
tests/qunit/QUnitTestResources.php
tests/qunit/data/testrunner.js
tests/qunit/data/testwarm.inject.js [deleted file]
tests/qunit/index.html

index e214b7b..e3dbfa6 100644 (file)
@@ -4240,7 +4240,17 @@ $wgEnableJavaScriptTest = false;
  */
 $wgJavaScriptTestConfig = array(
        'qunit' => array(
+               // Page where documentation can be found relevant to the QUnit test suite being ran.
+               // Used in the intro paragraph on [[Special:JavaScriptTest/qunit]] for the
+               // documentation link in the "javascripttest-qunit-intro" message.
                'documentation' => '//www.mediawiki.org/wiki/Manual:JavaScript_unit_testing',
+               // If you are submitting the QUnit test suite to a TestSwarm instance,
+               // point this to the "inject.js" script of that instance. This is was registers
+               // the QUnit hooks to extract the test results and push them back up into the
+               // TestSwarm database.
+               // @example 'http://localhost/testswarm/js/inject.js'
+               // @example '//integration.mediawiki.org/testswarm/js/inject.js'
+               'testswarm-injectjs' => false,
        ),
 );
 
index 7162393..f7e81ee 100644 (file)
@@ -138,6 +138,14 @@ class SpecialJavaScriptTest extends SpecialPage {
 HTML;
                $out->addHtml( $this->wrapSummaryHtml( $summary, 'frameworkfound' ) . $baseHtml );
 
+               // This special page is disabled by default ($wgEnableJavaScriptTest), and contains
+               // no sensitive data. In order to allow TestSwarm to embed it into a test client window,
+               // we need to allow iframing of this page.
+               $out->allowClickjacking();
+
+               // Used in ./tests/qunit/data/testrunner.js, see also documentation of
+               // $wgJavaScriptTestConfig in DefaultSettings.php
+               $out->addJsConfigVars( 'QUnitTestSwarmInjectJSPath', $wgJavaScriptTestConfig['qunit']['testswarm-injectjs'] );
        }
 
        public function isListed(){
index 9f2cf8e..687ad44 100644 (file)
@@ -48,5 +48,6 @@ return array(
                        'mediawiki.special.recentchanges',
                        'mediawiki.jqueryMsg',
                ),
+               'position' => 'top',
        )
 );
index c1cce83..30ae5bc 100644 (file)
@@ -29,9 +29,14 @@ QUnit.config.urlConfig.push( 'debug' );
 /**
  *  Load TestSwarm agent
  */
-if ( QUnit.urlParams.swarmURL  ) {
-       document.write( "<scr" + "ipt src='" + QUnit.fixurl( mw.config.get( 'wgScriptPath' )
-               + '/tests/qunit/data/testwarm.inject.js' ) + "'></scr" + "ipt>" );
+// Only if the current url indicates that there is a TestSwarm instance watching us
+// (TestSwarm appends swarmURL to the test suites url it loads in iframes).
+// Otherwise this is just a simple view of Special:JavaScriptTest/qunit directly,
+// no point in loading inject.js in that case. Also, make sure that this instance
+// of MediaWiki has actually been configured with the required url to that inject.js
+// script. By default it is false.
+if ( QUnit.urlParams.swarmURL && mw.config.get( 'QUnitTestSwarmInjectJSPath' ) ) {
+       document.write( "<scr" + "ipt src='" + QUnit.fixurl( mw.config.get( 'QUnitTestSwarmInjectJSPath' ) ) + "'></scr" + "ipt>" );
 }
 
 /**
diff --git a/tests/qunit/data/testwarm.inject.js b/tests/qunit/data/testwarm.inject.js
deleted file mode 100644 (file)
index 14ee8f9..0000000
+++ /dev/null
@@ -1,349 +0,0 @@
-/*
-       Copyright (c) 2009 John Resig
-       
-       Permission is hereby granted, free of charge, to any person
-       obtaining a copy of this software and associated documentation
-       files (the "Software"), to deal in the Software without
-       restriction, including without limitation the rights to use,
-       copy, modify, merge, publish, distribute, sublicense, and/or sell
-       copies of the Software, and to permit persons to whom the
-       Software is furnished to do so, subject to the following
-       conditions:
-       
-       The above copyright notice and this permission notice shall be
-       included in all copies or substantial portions of the Software.
-       
-       THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-       EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
-       OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-       NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
-       HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
-       WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-       FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
-       OTHER DEALINGS IN THE SOFTWARE.
-
-*/
-(function(){
-
-       var DEBUG = false;
-
-       var doPost = false;
-
-       try {
-               doPost = !!window.top.postMessage;
-       } catch(e){}
-
-       var search = window.location.search,
-               url, index;
-       if( ( index = search.indexOf( "swarmURL=" ) ) != -1 )
-               url = decodeURIComponent( search.slice( index + 9 ) );
-
-       if ( !DEBUG && (!url || url.indexOf("http") !== 0) ) {
-               return;
-       }
-
-       var submitTimeout = 5;
-
-       var curHeartbeat;
-       var beatRate = 20;
-
-       // Expose the TestSwarm API
-       window.TestSwarm = {
-               submit: submit,
-               heartbeat: function(){
-                       if ( curHeartbeat ) {
-                               clearTimeout( curHeartbeat );
-                       }
-
-                       curHeartbeat = setTimeout(function(){
-                               submit({ fail: -1, total: -1 });
-                       }, beatRate * 1000);
-               },
-               serialize: function(){
-                       return trimSerialize();
-               }
-       };
-
-       // Prevent careless things from executing
-       window.print = window.confirm = window.alert = window.open = function(){};
-
-       window.onerror = function(e){
-               document.body.appendChild( document.createTextNode( "ERROR: " + e ));
-               submit({ fail: 0, error: 1, total: 1 });
-               return false;
-       };
-
-       // QUnit (jQuery)
-       // http://docs.jquery.com/QUnit
-       if ( typeof QUnit !== "undefined" ) {
-               QUnit.done = function(results){
-                       submit({
-                               fail: results.failed,
-                               error: 0,
-                               total: results.total
-                       });
-               };
-
-               QUnit.log = window.TestSwarm.heartbeat;
-               window.TestSwarm.heartbeat();
-
-               window.TestSwarm.serialize = function(){
-                       // Clean up the HTML (remove any un-needed test markup)
-                       remove("nothiddendiv");
-                       remove("loadediframe");
-                       remove("dl");
-                       remove("main");
-
-                       // Show any collapsed results
-                       var ol = document.getElementsByTagName("ol");
-                       for ( var i = 0; i < ol.length; i++ ) {
-                               ol[i].style.display = "block";
-                       }
-
-                       return trimSerialize();
-               };
-
-       // UnitTestJS (Prototype, Scriptaculous)
-       // http://github.com/tobie/unittest_js/tree/master
-       } else if ( typeof Test !== "undefined" && Test && Test.Unit && Test.Unit.runners ) {
-               var total_runners = Test.Unit.runners.length, cur_runners = 0;
-               var total = 0, fail = 0, error = 0;
-
-               for (var i = 0; i < Test.Unit.runners.length; i++) (function(i){
-                       var finish = Test.Unit.runners[i].finish;
-                       Test.Unit.runners[i].finish = function(){
-                               finish.call( this );
-
-                               var results = this.getResult();
-                               total += results.assertions;
-                               fail += results.failures;
-                               error += results.errors;
-
-                               if ( ++cur_runners === total_runners ) {
-                                       submit({
-                                               fail: fail,
-                                               error: error,
-                                               total: total
-                                       });
-                               }
-                       };
-               })(i);
-
-       // JSSpec (MooTools)
-       // http://jania.pe.kr/aw/moin.cgi/JSSpec
-       } else if ( typeof JSSpec !== "undefined" && JSSpec && JSSpec.Logger ) {
-               var onRunnerEnd = JSSpec.Logger.prototype.onRunnerEnd;
-               JSSpec.Logger.prototype.onRunnerEnd = function(){
-                       onRunnerEnd.call(this);
-
-                       // Show any collapsed results
-                       var ul = document.getElementsByTagName("ul");
-                       for ( var i = 0; i < ul.length; i++ ) {
-                               ul[i].style.display = "block";
-                       }
-
-                       submit({
-                               fail: JSSpec.runner.getTotalFailures(),
-                               error: JSSpec.runner.getTotalErrors(),
-                               total: JSSpec.runner.totalExamples
-                       });
-               };
-
-               window.TestSwarm.serialize = function(){
-                       // Show any collapsed results
-                       var ul = document.getElementsByTagName("ul");
-                       for ( var i = 0; i < ul.length; i++ ) {
-                               ul[i].style.display = "block";
-                       }
-
-                       return trimSerialize();
-               };
-
-       // JSUnit
-       // http://www.jsunit.net/
-       // Note: Injection file must be included before the frames
-       //       are document.write()d into the page.
-       } else if ( typeof JsUnitTestManager !== "undefined" ) {
-               var _done = JsUnitTestManager.prototype._done;
-               JsUnitTestManager.prototype._done = function(){
-                       _done.call(this);
-
-                       submit({
-                               fail: this.failureCount,
-                               error: this.errorCount,
-                               total: this.totalCount
-                       });
-               };
-
-               window.TestSwarm.serialize = function(){
-                       return "<pre>" + this.log.join("\n") + "</pre>";
-               };
-
-       // Selenium Core
-       // http://seleniumhq.org/projects/core/
-       } else if ( typeof SeleniumTestResult !== "undefined" && typeof LOG !== "undefined" ) {
-               // Completely overwrite the postback
-               SeleniumTestResult.prototype.post = function(){
-                       submit({
-                               fail: this.metrics.numCommandFailures,
-                               error: this.metrics.numCommandErrors,
-                               total: this.metrics.numCommandPasses + this.metrics.numCommandFailures + this.metrics.numCommandErrors
-                       });
-               };
-
-               window.TestSwarm.serialize = function(){
-                       var results = [];
-                       while ( LOG.pendingMessages.length ) {
-                               var msg = LOG.pendingMessages.shift();
-                               results.push( msg.type + ": " + msg.msg );
-                       }
-
-                       return "<pre>" + results.join("\n") + "</pre>";
-               };
-
-       // Dojo Objective Harness
-       // http://docs.dojocampus.org/quickstart/doh
-       } else if ( typeof doh !== "undefined" && doh._report ) {
-               var _report = doh._report;
-               doh._report = function(){
-                       _report.apply(this, arguments);
-
-                       submit({
-                               fail: doh._failureCount,
-                               error: doh._errorCount,
-                               total: doh._testCount
-                       });
-               };
-
-               window.TestSwarm.serialize = function(){
-                       return "<pre>" + document.getElementById("logBody").innerHTML + "</pre>";
-               };
-  // Screw.Unit
-  // git://github.com/nathansobo/screw-unit.git
-       } else if ( typeof Screw !== "undefined" && typeof jQuery !== 'undefined' && Screw && Screw.Unit ) {
-    $(Screw).bind("after", function() {
-     var passed = $('.passed').length;
-     var failed = $('.failed').length;
-     submit({
-        fail: failed,
-        error: 0,
-        total: failed + passed
-      });
-    });
-
-    $(Screw).bind("loaded", function() {
-      $('.it')
-        .bind("passed", window.TestSwarm.heartbeat)
-        .bind("failed", window.TestSwarm.heartbeat);
-      window.TestSwarm.heartbeat();
-    });
-
-    window.TestSwarm.serialize = function(){
-       return trimSerialize();
-    };
-  }
-
-       function trimSerialize(doc) {
-               doc = doc || document;
-
-               var scripts = doc.getElementsByTagName("script");
-               while ( scripts.length ) {
-                       remove( scripts[0] );
-               }
-
-               var root = window.location.href.replace(/(https?:\/\/.*?)\/.*/, "$1");
-               var cur = window.location.href.replace(/[^\/]*$/, "");
-
-               var links = doc.getElementsByTagName("link");
-               for ( var i = 0; i < links.length; i++ ) {
-                       var href = links[i].href;
-                       if ( href.indexOf("/") === 0 ) {
-                               href = root + href;
-                       } else if ( !/^https?:\/\//.test( href ) ) {
-                               href = cur + href;
-                       }
-                       links[i].href = href;
-               }
-
-               return ("<html>" + doc.documentElement.innerHTML + "</html>")
-                       .replace(/\s+/g, " ");
-       }
-
-       function remove(elem){
-               if ( typeof elem === "string" ) {
-                       elem = document.getElementById( elem );
-               }
-
-               if ( elem ) {
-                       elem.parentNode.removeChild( elem );
-               }
-       }
-
-       function submit(params){
-               if ( curHeartbeat ) {
-                       clearTimeout( curHeartbeat );
-               }
-
-               var paramItems = (url.split("?")[1] || "").split("&");
-
-               for ( var i = 0; i < paramItems.length; i++ ) {
-                       if ( paramItems[i] ) {
-                               var parts = paramItems[i].split("=");
-                               if ( !params[ parts[0] ] ) {
-                                       params[ parts[0] ] = parts[1];
-                               }
-                       }
-               }
-
-               if ( !params.state ) {
-                       params.state = "saverun";
-               }
-
-               if ( !params.results ) {
-                       params.results = window.TestSwarm.serialize();
-               }
-
-               if ( doPost ) {
-                       // Build Query String
-                       var query = "";
-
-                       for ( var i in params ) {
-                               query += ( query ? "&" : "" ) + i + "=" +
-                                       encodeURIComponent(params[i]);
-                       }
-
-                       if ( DEBUG ) {
-                               alert( query );
-                       } else {
-                               window.top.postMessage( query, "*" );
-                       }
-
-               } else {
-                       var form = document.createElement("form");
-                       form.action = url;
-                       form.method = "POST";
-
-                       for ( var i in params ) {
-                               var input = document.createElement("input");
-                               input.type = "hidden";
-                               input.name = i;
-                               input.value = params[i];
-                               form.appendChild( input );
-                       }
-
-                       if ( DEBUG ) {
-                               alert( form.innerHTML );
-                       } else {
-
-                               // Watch for the result submission timing out
-                               setTimeout(function(){
-                                       submit( params );
-                               }, submitTimeout * 1000);
-
-                               document.body.appendChild( form );
-                               form.submit();
-                       }
-               }
-       }
-
-})();
index 913c1b2..c0557ec 100644 (file)
                
                /* WikiPage specific */
                mw.config.set({"wgCanonicalNamespace": "", "wgCanonicalSpecialPageName": false, "wgNamespaceNumber": 0, "wgPageName": "Sandbox", "wgTitle": "Sandbox", "wgCurRevisionId": 486, "wgArticleId": 84, "wgIsArticle": true, "wgAction": "view", "wgUserName": null, "wgUserGroups": ["*"], "wgCategories": [], "wgBreakFrames": false, "wgPageContentLanguage": "en", "wgSeparatorTransformTable": ["", ""], "wgDigitTransformTable": ["", ""], "wgRestrictionEdit": [], "wgRestrictionMove": [], "wgRedirectedFrom": "Sandbox"});
-               
+
+               /* Special:JavaScriptTest/qunit specific */
+               mw.config.set({"QUnitTestSwarmInjectJSPath": false});
+
                /**
                 * Fix wgScriptPath and the like to the real thing,
                 * instead of fake ones (for access to /tests/qunit/data/)