Replace XVO with support for the Key HTTP header

MediaWiki currently has support for a header called X-Vary-Options
(XVO), used to communicate to upstream caches more granular cache
variance options than the Vary header can.

The header was envisioned by Tim Starling back in 2008 and implemented
into MediaWiki and Squid 2.0, with those patches submitted to the
squid-dev mailing list at the time:
http://www.squid-cache.org/mail-archive/squid-dev/200802/0085.html
The patches never actually made it into an upstream Squid release,
however, and Squid has since evolved in potentially significant ways.

Wikimedia has since switched to Varnish but XVO was not ported over as
it was deemed too complex at the time; custom VCL was used instead. To
our knowledge, noone else is using XVO in production and certainly not
with recent, up-to-date MediaWiki releases.

There is currently work at IETF's httpbis working group for a "Key"
header that is in concept and implementation very similar to Tim's XVO
header: https://datatracker.ietf.org/doc/draft-fielding-http-key/

Rather than rip XVO out of MediaWiki, replace it with support for the
Key header, as preliminary defined by the draft spec. This is an almost
straight search-and-replace.

No other software (caching proxy or user-agent) currently implements Key
to my knowledge, so this is essentially untested.

Change-Id: I949fc289dd5d48bd34f3b37f7739e2b9cd5db277

diff --git a/RELEASE-NOTES-1.27 b/RELEASE-NOTES-1.27
index c96eed2..cf22127 100644 (file)
--- a/RELEASE-NOTES-1.27
+++ b/RELEASE-NOTES-1.27
@@ -17,6 +17,11 @@ production.
 * $wgDebugDumpSqlLength was removed (deprecated in 1.24).
 * $wgDebugDBTransactions was removed (deprecated in 1.20).
 * $wgRemoteUploadTarget (added in 1.26) removed, replaced by $wgForeignUploadTargets
+* $wgUseXVO has been removed, as it provides functionality only used by
+  custom Wikimedia patches against Squid 2.x that probably noone uses in
+  production anymore. There is now $wgUseKeyHeader that provides similar
+  functionality but instead of the MediaWiki-specific X-Vary-Options header,
+  uses the draft Key header standard.
 
 === New features in 1.27 ===
 * $wgDataCenterId and $wgDataCenterRoles where added, which will serve as

diff --git a/includes/DefaultSettings.php b/includes/DefaultSettings.php
index 7733c93..0ce80bc 100644 (file)
--- a/includes/DefaultSettings.php
+++ b/includes/DefaultSettings.php
@@ -2544,13 +2544,15 @@ $wgUseSquid = false;
 $wgUseESI = false;
 
 /**
- * Send X-Vary-Options header for better caching (requires patched Squid)
+ * Send the Key HTTP header for better caching.
+ * See https://datatracker.ietf.org/doc/draft-fielding-http-key/ for details.
+ * @since 1.27
  */
-$wgUseXVO = false;
+$wgUseKeyHeader = false;
 
 /**
- * Add X-Forwarded-Proto to the Vary and X-Vary-Options headers for API
- * requests and RSS/Atom feeds. Use this if you have an SSL termination setup
+ * Add X-Forwarded-Proto to the Vary and Key headers for API requests and
+ * RSS/Atom feeds. Use this if you have an SSL termination setup
  * and need to split the cache between HTTP and HTTPS for API requests,
  * feed requests and HTTP redirect responses in order to prevent cache
  * pollution. This does not affect 'normal' requests to index.php other than

diff --git a/includes/OutputHandler.php b/includes/OutputHandler.php
index c6209ee..39716ca 100644 (file)
--- a/includes/OutputHandler.php
+++ b/includes/OutputHandler.php
@@ -137,9 +137,9 @@ function wfGzipHandler( $s ) {
        }
        if ( !$foundVary ) {
                header( 'Vary: Accept-Encoding' );
-               global $wgUseXVO;
-               if ( $wgUseXVO ) {
-                       header( 'X-Vary-Options: Accept-Encoding;list-contains=gzip' );
+               global $wgUseKeyHeader;
+               if ( $wgUseKeyHeader ) {
+                       header( 'Key: Accept-Encoding;match=gzip' );
                }
        }
        return $s;

diff --git a/includes/OutputPage.php b/includes/OutputPage.php
index 03ae8c9..755b165 100644 (file)
--- a/includes/OutputPage.php
+++ b/includes/OutputPage.php
@@ -273,7 +273,7 @@ class OutputPage extends ContextSource {
        private $mIndexPolicy = 'index';
        private $mFollowPolicy = 'follow';
        private $mVaryHeader = array(
-               'Accept-Encoding' => array( 'list-contains=gzip' ),
+               'Accept-Encoding' => array( 'match=gzip' ),
        );
 
        /**
@@ -2002,14 +2002,9 @@ class OutputPage extends ContextSource {
         * @return bool
         */
        function haveCacheVaryCookies() {
-               $cookieHeader = $this->getRequest()->getHeader( 'cookie' );
-               if ( $cookieHeader === false ) {
-                       return false;
-               }
-               $cvCookies = $this->getCacheVaryCookies();
-               foreach ( $cvCookies as $cookieName ) {
-                       # Check for a simple string match, like the way squid does it
-                       if ( strpos( $cookieHeader, $cookieName ) !== false ) {
+               $request = $this->getRequest();
+               foreach ( $this->getCacheVaryCookies() as $cookieName ) {
+                       if ( $request->getCookie( $cookieName, '' ) ) {
                                wfDebug( __METHOD__ . ": found $cookieName\n" );
                                return true;
                        }
@@ -2022,11 +2017,9 @@ class OutputPage extends ContextSource {
         * Add an HTTP header that will influence on the cache
         *
         * @param string $header Header name
-        * @param string[]|null $option Options for X-Vary-Options. Possible options are:
-        *  - "string-contains=$XXX" varies on whether the header value as a string
-        *    contains $XXX as a substring.
-        *  - "list-contains=$XXX" varies on whether the header value as a
-        *    comma-separated list contains $XXX as one of the list items.
+        * @param string[]|null $option Options for the Key header. See
+        * https://datatracker.ietf.org/doc/draft-fielding-http-key/
+        * for the list of valid options.
         */
        public function addVaryHeader( $header, array $option = null ) {
                if ( !array_key_exists( $header, $this->mVaryHeader ) ) {
@@ -2049,16 +2042,16 @@ class OutputPage extends ContextSource {
        }
 
        /**
-        * Get a complete X-Vary-Options header
+        * Get a complete Key header
         *
         * @return string
         */
-       public function getXVO() {
+       public function getKeyHeader() {
                $cvCookies = $this->getCacheVaryCookies();
 
                $cookiesOption = array();
                foreach ( $cvCookies as $cookieName ) {
-                       $cookiesOption[] = 'string-contains=' . $cookieName;
+                       $cookiesOption[] = 'param=' . $cookieName;
                }
                $this->addVaryHeader( 'Cookie', $cookiesOption );
 
@@ -2070,13 +2063,13 @@ class OutputPage extends ContextSource {
                        }
                        $headers[] = $newheader;
                }
-               $xvo = 'X-Vary-Options: ' . implode( ',', $headers );
+               $key = 'Key: ' . implode( ',', $headers );
 
-               return $xvo;
+               return $key;
        }
 
        /**
-        * bug 21672: Add Accept-Language to Vary and XVO headers
+        * T23672: Add Accept-Language to Vary and Key headers
         * if there's no 'variant' parameter existed in GET.
         *
         * For example:
@@ -2097,14 +2090,14 @@ class OutputPage extends ContextSource {
                                if ( $variant === $lang->getCode() ) {
                                        continue;
                                } else {
-                                       $aloption[] = 'string-contains=' . $variant;
+                                       $aloption[] = 'substr=' . $variant;
 
                                        // IE and some other browsers use BCP 47 standards in
                                        // their Accept-Language header, like "zh-CN" or "zh-Hant".
                                        // We should handle these too.
                                        $variantBCP47 = wfBCP47( $variant );
                                        if ( $variantBCP47 !== $variant ) {
-                                               $aloption[] = 'string-contains=' . $variantBCP47;
+                                               $aloption[] = 'substr=' . $variantBCP47;
                                        }
                                }
                        }
@@ -2179,9 +2172,8 @@ class OutputPage extends ContextSource {
                # maintain different caches for logged-in users and non-logged in ones
                $response->header( $this->getVaryHeader() );
 
-               if ( $config->get( 'UseXVO' ) ) {
-                       # Add an X-Vary-Options header for Squid with Wikimedia patches
-                       $response->header( $this->getXVO() );
+               if ( $config->get( 'UseKeyHeader' ) ) {
+                       $response->header( $this->getKeyHeader() );
                }
 
                if ( $this->mEnableClientCache ) {

diff --git a/includes/api/ApiMain.php b/includes/api/ApiMain.php
index 8e0ba8b..5d2db47 100644 (file)
--- a/includes/api/ApiMain.php
+++ b/includes/api/ApiMain.php
@@ -760,12 +760,12 @@ class ApiMain extends ApiBase {
                        return;
                }
 
-               $useXVO = $config->get( 'UseXVO' );
+               $useKeyHeader = $config->get( 'UseKeyHeader' );
                if ( $this->mCacheMode == 'anon-public-user-private' ) {
                        $out->addVaryHeader( 'Cookie' );
                        $response->header( $out->getVaryHeader() );
-                       if ( $useXVO ) {
-                               $response->header( $out->getXVO() );
+                       if ( $useKeyHeader ) {
+                               $response->header( $out->getKeyHeader() );
                                if ( $out->haveCacheVaryCookies() ) {
                                        // Logged in, mark this request private
                                        $response->header( "Cache-Control: $privateCache" );
@@ -778,13 +778,13 @@ class ApiMain extends ApiBase {
                                $response->header( "Cache-Control: $privateCache" );
 
                                return;
-                       } // else no XVO and anonymous, send public headers below
+                       } // else no Key and anonymous, send public headers below
                }
 
                // Send public headers
                $response->header( $out->getVaryHeader() );
-               if ( $useXVO ) {
-                       $response->header( $out->getXVO() );
+               if ( $useKeyHeader ) {
+                       $response->header( $out->getKeyHeader() );
                }
 
                // If nobody called setCacheMaxAge(), use the (s)maxage parameters

diff --git a/tests/phpunit/includes/OutputPageTest.php b/tests/phpunit/includes/OutputPageTest.php
index 51a19c6..aa6655d 100644 (file)
--- a/tests/phpunit/includes/OutputPageTest.php
+++ b/tests/phpunit/includes/OutputPageTest.php
@@ -266,9 +266,9 @@ class OutputPageTest extends MediaWikiTestCase {
         * @dataProvider provideVaryHeaders
         * @covers OutputPage::addVaryHeader
         * @covers OutputPage::getVaryHeader
-        * @covers OutputPage::getXVO
+        * @covers OutputPage::getKeyHeader
         */
-       public function testVaryHeaders( $calls, $vary, $xvo ) {
+       public function testVaryHeaders( $calls, $vary, $key ) {
                // get rid of default Vary fields
                $outputPage = $this->getMockBuilder( 'OutputPage' )
                        ->setConstructorArgs( array( new RequestContext() ) )
@@ -283,18 +283,18 @@ class OutputPageTest extends MediaWikiTestCase {
                        call_user_func_array( array( $outputPage, 'addVaryHeader' ), $call );
                }
                $this->assertEquals( $vary, $outputPage->getVaryHeader(), 'Vary:' );
-               $this->assertEquals( $xvo, $outputPage->getXVO(), 'X-Vary-Options:' );
+               $this->assertEquals( $key, $outputPage->getKeyHeader(), 'Key:' );
        }
 
        public function provideVaryHeaders() {
-               // note: getXVO() automatically adds Vary: Cookie
+               // note: getKeyHeader() automatically adds Vary: Cookie
                return array(
                        array( // single header
                                array(
                                        array( 'Cookie' ),
                                ),
                                'Vary: Cookie',
-                               'X-Vary-Options: Cookie',
+                               'Key: Cookie',
                        ),
                        array( // non-unique headers
                                array(
@@ -303,39 +303,39 @@ class OutputPageTest extends MediaWikiTestCase {
                                        array( 'Cookie' ),
                                ),
                                'Vary: Cookie, Accept-Language',
-                               'X-Vary-Options: Cookie,Accept-Language',
+                               'Key: Cookie,Accept-Language',
                        ),
                        array( // two headers with single options
                                array(
-                                       array( 'Cookie', array( 'string-contains=phpsessid' ) ),
-                                       array( 'Accept-Language', array( 'string-contains=en' ) ),
+                                       array( 'Cookie', array( 'param=phpsessid' ) ),
+                                       array( 'Accept-Language', array( 'substr=en' ) ),
                                ),
                                'Vary: Cookie, Accept-Language',
-                               'X-Vary-Options: Cookie;string-contains=phpsessid,Accept-Language;string-contains=en',
+                               'Key: Cookie;param=phpsessid,Accept-Language;substr=en',
                        ),
                        array( // one header with multiple options
                                array(
-                                       array( 'Cookie', array( 'string-contains=phpsessid', 'string-contains=userId' ) ),
+                                       array( 'Cookie', array( 'param=phpsessid', 'param=userId' ) ),
                                ),
                                'Vary: Cookie',
-                               'X-Vary-Options: Cookie;string-contains=phpsessid;string-contains=userId',
+                               'Key: Cookie;param=phpsessid;param=userId',
                        ),
                        array( // Duplicate option
                                array(
-                                       array( 'Cookie', array( 'string-contains=phpsessid' ) ),
-                                       array( 'Cookie', array( 'string-contains=phpsessid' ) ),
-                                       array( 'Accept-Language', array( 'string-contains=en', 'string-contains=en' ) ),
+                                       array( 'Cookie', array( 'param=phpsessid' ) ),
+                                       array( 'Cookie', array( 'param=phpsessid' ) ),
+                                       array( 'Accept-Language', array( 'substr=en', 'substr=en' ) ),
                                ),
                                'Vary: Cookie, Accept-Language',
-                               'X-Vary-Options: Cookie;string-contains=phpsessid,Accept-Language;string-contains=en',
+                               'Key: Cookie;param=phpsessid,Accept-Language;substr=en',
                        ),
                        array( // Same header, different options
                                array(
-                                       array( 'Cookie', array( 'string-contains=phpsessid' ) ),
-                                       array( 'Cookie', array( 'string-contains=userId' ) ),
+                                       array( 'Cookie', array( 'param=phpsessid' ) ),
+                                       array( 'Cookie', array( 'param=userId' ) ),
                                ),
                                'Vary: Cookie',
-                               'X-Vary-Options: Cookie;string-contains=phpsessid;string-contains=userId',
+                               'Key: Cookie;param=phpsessid;param=userId',
                        ),
                );
        }