From 5e590be3d689a1bcc7490102f7ab8463d231e8e5 Mon Sep 17 00:00:00 2001 From: Krinkle Date: Sun, 25 Mar 2012 03:09:58 +0200 Subject: [PATCH] Fix support for TestSwarm on SpecialJavaScriptTest/qunit So far we've still been using ./tests/qunit/index.html in TestSwarm, today I've tested locally to submit a url to SpecialJavaScriptTest instead and made a bunch of browsers join my swarm, quite a few problems popped up. This commit fixes those issues so that we can actually use SpecialJavaScriptTest in TestSwarm. * Add QUnit configuration variable for TestSwarm's inject.js In order to use TestSwarm, the urls that TestSwarm loads in clients that has the QUnit test suite running on it need to include a little javascript. This inject.js registers hooks with QUnit to listen for when the test suite finishes and contacts the parent window (TestSwarm loads the qunit test suite url in an iframe) to submit the results. Previously I included a copy of TestSwarm's inject.js in ./tests/qunit/data and in our testrunner.js a relative link to that. However this is currently breaking because it is an outdated version. Updating brings no good since someone else might use their own TestSwarm would could still run on an old version etc. The TestSwarm submitted too always expects that it's own inject.js is used, not some snapshot copy. I've removed the copy of it in MediaWiki and instead added a configuration option to point to wherever the you want is located. Also, since the old static index.html version of the unit test can't retrieve PHP based content, this means TestSwarm submissions through the old static index.html are no longer supported. Only through the new Special:JavaScriptTest from now on. I'll probably remove the whole index.html soon-ish as it's getting quite annoying to maintain all that by hand, and it's been superseded in everywhere imaginable now anyway. Even not used anymore by intergration.mediawiki.org because that's been quiet since the Git-switchover.., and when we update it, we can update it to point to the new SpecialPage instead. * OutputPage::allowClickjacking() on SpecialJavaScriptTest/qunit. When initially testing the TestSwarm setup to submit SpecialJavaScriptTest/qunit urls (instead of the old ./tests/qunit/index.html) it was failing due to an iframe DENY. This was a bit odd since `$wgBreakFrames = false;` by default, and although `$wgEditPageFrameOptions = 'DENY';` by default, it wasn't obvious at all that that value ("DENY") is used for all OutputPages by default (as supposed to just action=edit and the like). This is because OutputPage has mPreventClickjacking=true by default and when it's true-ish it uses $wgEditPageFrameOptions for the X-Frame-Options. * 'position' => true; for the mediawiki.tests.qunit.suites module. QUnit has a hook for "done". Which is called when QUnit.start() is called and all queued tests have been executed. QUnit.start() is automatically called on window.onload by QUnit. TestSwarm uses QUnit's hook system to hook into the QUnit "done" event, and at that point takes the stats, submits them to TestSwarm and go on with the next job. When testing locally, I got semi-random failures reporting that only 0/0 tests were successfully ran in IE6. This is because when QUnit.start (and consequently QUnit.done) are first called, apparently no test suites had finished downloading and/or execution yet (the bottom queue is asynchronous, and doesn't postpone domready nor window.onload). When normally viewing Special:JavaScriptTest/qunit this doesn't break anything, because if QUnit start/done is in the past and another module(), test(), or equal() etc. is called it just picks up again and adds more results to the page and calls QUnit.done() again. However in the case of the TestSwarm embed, it submits the results after the first done() and cleans up the iframe. So I'm making mediawiki.tests.qunit.suites a blocking module instead, so that there will only be one QUnit.start/done and that's the one that TestSwarm gets and after which TestSwarm can safely garbage the iframe. This means that basically all test suite modules and the original modules they are testing will be loaded from the head. Shouldn't have any side effects, but might cause minor breakage in future in modules that badly assume they're being put on the bottom. I'm not considering that a bug in the test, it'll just help catch that bad code sooner :), it's a test suite after all. (Yay, my first Git commit to MediaWiki core) Change-Id: I83f83377f2183b6deb4e901af602ac9a5628558b --- includes/DefaultSettings.php | 10 + includes/specials/SpecialJavaScriptTest.php | 8 + tests/qunit/QUnitTestResources.php | 1 + tests/qunit/data/testrunner.js | 11 +- tests/qunit/data/testwarm.inject.js | 349 -------------------- tests/qunit/index.html | 5 +- 6 files changed, 31 insertions(+), 353 deletions(-) delete mode 100644 tests/qunit/data/testwarm.inject.js diff --git a/includes/DefaultSettings.php b/includes/DefaultSettings.php index e214b7b302..e3dbfa6e54 100644 --- a/includes/DefaultSettings.php +++ b/includes/DefaultSettings.php @@ -4240,7 +4240,17 @@ $wgEnableJavaScriptTest = false; */ $wgJavaScriptTestConfig = array( 'qunit' => array( + // Page where documentation can be found relevant to the QUnit test suite being ran. + // Used in the intro paragraph on [[Special:JavaScriptTest/qunit]] for the + // documentation link in the "javascripttest-qunit-intro" message. 'documentation' => '//www.mediawiki.org/wiki/Manual:JavaScript_unit_testing', + // If you are submitting the QUnit test suite to a TestSwarm instance, + // point this to the "inject.js" script of that instance. This is was registers + // the QUnit hooks to extract the test results and push them back up into the + // TestSwarm database. + // @example 'http://localhost/testswarm/js/inject.js' + // @example '//integration.mediawiki.org/testswarm/js/inject.js' + 'testswarm-injectjs' => false, ), ); diff --git a/includes/specials/SpecialJavaScriptTest.php b/includes/specials/SpecialJavaScriptTest.php index 71623930eb..f7e81eeada 100644 --- a/includes/specials/SpecialJavaScriptTest.php +++ b/includes/specials/SpecialJavaScriptTest.php @@ -138,6 +138,14 @@ class SpecialJavaScriptTest extends SpecialPage { HTML; $out->addHtml( $this->wrapSummaryHtml( $summary, 'frameworkfound' ) . $baseHtml ); + // This special page is disabled by default ($wgEnableJavaScriptTest), and contains + // no sensitive data. In order to allow TestSwarm to embed it into a test client window, + // we need to allow iframing of this page. + $out->allowClickjacking(); + + // Used in ./tests/qunit/data/testrunner.js, see also documentation of + // $wgJavaScriptTestConfig in DefaultSettings.php + $out->addJsConfigVars( 'QUnitTestSwarmInjectJSPath', $wgJavaScriptTestConfig['qunit']['testswarm-injectjs'] ); } public function isListed(){ diff --git a/tests/qunit/QUnitTestResources.php b/tests/qunit/QUnitTestResources.php index 9f2cf8e15c..687ad44375 100644 --- a/tests/qunit/QUnitTestResources.php +++ b/tests/qunit/QUnitTestResources.php @@ -48,5 +48,6 @@ return array( 'mediawiki.special.recentchanges', 'mediawiki.jqueryMsg', ), + 'position' => 'top', ) ); diff --git a/tests/qunit/data/testrunner.js b/tests/qunit/data/testrunner.js index c1cce83116..30ae5bc79b 100644 --- a/tests/qunit/data/testrunner.js +++ b/tests/qunit/data/testrunner.js @@ -29,9 +29,14 @@ QUnit.config.urlConfig.push( 'debug' ); /** * Load TestSwarm agent */ -if ( QUnit.urlParams.swarmURL ) { - document.write( "" ); +// Only if the current url indicates that there is a TestSwarm instance watching us +// (TestSwarm appends swarmURL to the test suites url it loads in iframes). +// Otherwise this is just a simple view of Special:JavaScriptTest/qunit directly, +// no point in loading inject.js in that case. Also, make sure that this instance +// of MediaWiki has actually been configured with the required url to that inject.js +// script. By default it is false. +if ( QUnit.urlParams.swarmURL && mw.config.get( 'QUnitTestSwarmInjectJSPath' ) ) { + document.write( "" ); } /** diff --git a/tests/qunit/data/testwarm.inject.js b/tests/qunit/data/testwarm.inject.js deleted file mode 100644 index 14ee8f93ab..0000000000 --- a/tests/qunit/data/testwarm.inject.js +++ /dev/null @@ -1,349 +0,0 @@ -/* - Copyright (c) 2009 John Resig - - Permission is hereby granted, free of charge, to any person - obtaining a copy of this software and associated documentation - files (the "Software"), to deal in the Software without - restriction, including without limitation the rights to use, - copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the - Software is furnished to do so, subject to the following - conditions: - - The above copyright notice and this permission notice shall be - included in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES - OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND - NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT - HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, - WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING - FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - -*/ -(function(){ - - var DEBUG = false; - - var doPost = false; - - try { - doPost = !!window.top.postMessage; - } catch(e){} - - var search = window.location.search, - url, index; - if( ( index = search.indexOf( "swarmURL=" ) ) != -1 ) - url = decodeURIComponent( search.slice( index + 9 ) ); - - if ( !DEBUG && (!url || url.indexOf("http") !== 0) ) { - return; - } - - var submitTimeout = 5; - - var curHeartbeat; - var beatRate = 20; - - // Expose the TestSwarm API - window.TestSwarm = { - submit: submit, - heartbeat: function(){ - if ( curHeartbeat ) { - clearTimeout( curHeartbeat ); - } - - curHeartbeat = setTimeout(function(){ - submit({ fail: -1, total: -1 }); - }, beatRate * 1000); - }, - serialize: function(){ - return trimSerialize(); - } - }; - - // Prevent careless things from executing - window.print = window.confirm = window.alert = window.open = function(){}; - - window.onerror = function(e){ - document.body.appendChild( document.createTextNode( "ERROR: " + e )); - submit({ fail: 0, error: 1, total: 1 }); - return false; - }; - - // QUnit (jQuery) - // http://docs.jquery.com/QUnit - if ( typeof QUnit !== "undefined" ) { - QUnit.done = function(results){ - submit({ - fail: results.failed, - error: 0, - total: results.total - }); - }; - - QUnit.log = window.TestSwarm.heartbeat; - window.TestSwarm.heartbeat(); - - window.TestSwarm.serialize = function(){ - // Clean up the HTML (remove any un-needed test markup) - remove("nothiddendiv"); - remove("loadediframe"); - remove("dl"); - remove("main"); - - // Show any collapsed results - var ol = document.getElementsByTagName("ol"); - for ( var i = 0; i < ol.length; i++ ) { - ol[i].style.display = "block"; - } - - return trimSerialize(); - }; - - // UnitTestJS (Prototype, Scriptaculous) - // http://github.com/tobie/unittest_js/tree/master - } else if ( typeof Test !== "undefined" && Test && Test.Unit && Test.Unit.runners ) { - var total_runners = Test.Unit.runners.length, cur_runners = 0; - var total = 0, fail = 0, error = 0; - - for (var i = 0; i < Test.Unit.runners.length; i++) (function(i){ - var finish = Test.Unit.runners[i].finish; - Test.Unit.runners[i].finish = function(){ - finish.call( this ); - - var results = this.getResult(); - total += results.assertions; - fail += results.failures; - error += results.errors; - - if ( ++cur_runners === total_runners ) { - submit({ - fail: fail, - error: error, - total: total - }); - } - }; - })(i); - - // JSSpec (MooTools) - // http://jania.pe.kr/aw/moin.cgi/JSSpec - } else if ( typeof JSSpec !== "undefined" && JSSpec && JSSpec.Logger ) { - var onRunnerEnd = JSSpec.Logger.prototype.onRunnerEnd; - JSSpec.Logger.prototype.onRunnerEnd = function(){ - onRunnerEnd.call(this); - - // Show any collapsed results - var ul = document.getElementsByTagName("ul"); - for ( var i = 0; i < ul.length; i++ ) { - ul[i].style.display = "block"; - } - - submit({ - fail: JSSpec.runner.getTotalFailures(), - error: JSSpec.runner.getTotalErrors(), - total: JSSpec.runner.totalExamples - }); - }; - - window.TestSwarm.serialize = function(){ - // Show any collapsed results - var ul = document.getElementsByTagName("ul"); - for ( var i = 0; i < ul.length; i++ ) { - ul[i].style.display = "block"; - } - - return trimSerialize(); - }; - - // JSUnit - // http://www.jsunit.net/ - // Note: Injection file must be included before the frames - // are document.write()d into the page. - } else if ( typeof JsUnitTestManager !== "undefined" ) { - var _done = JsUnitTestManager.prototype._done; - JsUnitTestManager.prototype._done = function(){ - _done.call(this); - - submit({ - fail: this.failureCount, - error: this.errorCount, - total: this.totalCount - }); - }; - - window.TestSwarm.serialize = function(){ - return "
" + this.log.join("\n") + "
"; - }; - - // Selenium Core - // http://seleniumhq.org/projects/core/ - } else if ( typeof SeleniumTestResult !== "undefined" && typeof LOG !== "undefined" ) { - // Completely overwrite the postback - SeleniumTestResult.prototype.post = function(){ - submit({ - fail: this.metrics.numCommandFailures, - error: this.metrics.numCommandErrors, - total: this.metrics.numCommandPasses + this.metrics.numCommandFailures + this.metrics.numCommandErrors - }); - }; - - window.TestSwarm.serialize = function(){ - var results = []; - while ( LOG.pendingMessages.length ) { - var msg = LOG.pendingMessages.shift(); - results.push( msg.type + ": " + msg.msg ); - } - - return "
" + results.join("\n") + "
"; - }; - - // Dojo Objective Harness - // http://docs.dojocampus.org/quickstart/doh - } else if ( typeof doh !== "undefined" && doh._report ) { - var _report = doh._report; - doh._report = function(){ - _report.apply(this, arguments); - - submit({ - fail: doh._failureCount, - error: doh._errorCount, - total: doh._testCount - }); - }; - - window.TestSwarm.serialize = function(){ - return "
" + document.getElementById("logBody").innerHTML + "
"; - }; - // Screw.Unit - // git://github.com/nathansobo/screw-unit.git - } else if ( typeof Screw !== "undefined" && typeof jQuery !== 'undefined' && Screw && Screw.Unit ) { - $(Screw).bind("after", function() { - var passed = $('.passed').length; - var failed = $('.failed').length; - submit({ - fail: failed, - error: 0, - total: failed + passed - }); - }); - - $(Screw).bind("loaded", function() { - $('.it') - .bind("passed", window.TestSwarm.heartbeat) - .bind("failed", window.TestSwarm.heartbeat); - window.TestSwarm.heartbeat(); - }); - - window.TestSwarm.serialize = function(){ - return trimSerialize(); - }; - } - - function trimSerialize(doc) { - doc = doc || document; - - var scripts = doc.getElementsByTagName("script"); - while ( scripts.length ) { - remove( scripts[0] ); - } - - var root = window.location.href.replace(/(https?:\/\/.*?)\/.*/, "$1"); - var cur = window.location.href.replace(/[^\/]*$/, ""); - - var links = doc.getElementsByTagName("link"); - for ( var i = 0; i < links.length; i++ ) { - var href = links[i].href; - if ( href.indexOf("/") === 0 ) { - href = root + href; - } else if ( !/^https?:\/\//.test( href ) ) { - href = cur + href; - } - links[i].href = href; - } - - return ("" + doc.documentElement.innerHTML + "") - .replace(/\s+/g, " "); - } - - function remove(elem){ - if ( typeof elem === "string" ) { - elem = document.getElementById( elem ); - } - - if ( elem ) { - elem.parentNode.removeChild( elem ); - } - } - - function submit(params){ - if ( curHeartbeat ) { - clearTimeout( curHeartbeat ); - } - - var paramItems = (url.split("?")[1] || "").split("&"); - - for ( var i = 0; i < paramItems.length; i++ ) { - if ( paramItems[i] ) { - var parts = paramItems[i].split("="); - if ( !params[ parts[0] ] ) { - params[ parts[0] ] = parts[1]; - } - } - } - - if ( !params.state ) { - params.state = "saverun"; - } - - if ( !params.results ) { - params.results = window.TestSwarm.serialize(); - } - - if ( doPost ) { - // Build Query String - var query = ""; - - for ( var i in params ) { - query += ( query ? "&" : "" ) + i + "=" + - encodeURIComponent(params[i]); - } - - if ( DEBUG ) { - alert( query ); - } else { - window.top.postMessage( query, "*" ); - } - - } else { - var form = document.createElement("form"); - form.action = url; - form.method = "POST"; - - for ( var i in params ) { - var input = document.createElement("input"); - input.type = "hidden"; - input.name = i; - input.value = params[i]; - form.appendChild( input ); - } - - if ( DEBUG ) { - alert( form.innerHTML ); - } else { - - // Watch for the result submission timing out - setTimeout(function(){ - submit( params ); - }, submitTimeout * 1000); - - document.body.appendChild( form ); - form.submit(); - } - } - } - -})(); diff --git a/tests/qunit/index.html b/tests/qunit/index.html index 913c1b2de3..c0557ecae7 100644 --- a/tests/qunit/index.html +++ b/tests/qunit/index.html @@ -18,7 +18,10 @@ /* WikiPage specific */ mw.config.set({"wgCanonicalNamespace": "", "wgCanonicalSpecialPageName": false, "wgNamespaceNumber": 0, "wgPageName": "Sandbox", "wgTitle": "Sandbox", "wgCurRevisionId": 486, "wgArticleId": 84, "wgIsArticle": true, "wgAction": "view", "wgUserName": null, "wgUserGroups": ["*"], "wgCategories": [], "wgBreakFrames": false, "wgPageContentLanguage": "en", "wgSeparatorTransformTable": ["", ""], "wgDigitTransformTable": ["", ""], "wgRestrictionEdit": [], "wgRestrictionMove": [], "wgRedirectedFrom": "Sandbox"}); - + + /* Special:JavaScriptTest/qunit specific */ + mw.config.set({"QUnitTestSwarmInjectJSPath": false}); + /** * Fix wgScriptPath and the like to the real thing, * instead of fake ones (for access to /tests/qunit/data/) -- 2.20.1