$wgHTCPMulticastRouting rename + multi hosts support

This patch does two things:

A) rename $wgHTCPMulticastRouting to $wgHTCPRouting since you can have
   MediaWiki send purge packets over unicast.
B) add support for multi hosts purge in the few cases one want to split
   the multicast groups per cache role but still want to purge more than
   one cache group.

A) rename

The rename adds deprecation notices in the comments and adds a back
compatibility layer in the case someone is already using this feature.
Given Wikimedia Foundation is not even using it, it is very unlikely,
but yet: better safe than sorry.

My logic is flawed sometime, so that needs a bit of review :) There is
two levels of deprecations to watch for:
- pre mw1.20 which useds $wgHTCPMulticastAddress and
  $wgHTCPMulticastPort
- pre mw1.22 (aka before this patch) that used $wgHTCPMulticastRouting

The resulting configuration should be properly loaded in $wgHTCPRouting.

B) multi hosts

The HTCP routing let you split purges to different hosts according to
the URL.  In some case, we want to be able to purge an URL from more
than one multicast group.  A Wikimedia example would be to send text
related purges to the text and mobiles caches while upload purges are
sent to the upload caches.

An abstracted example would be:

$wgHTCPRouting = array(

	// upload URLs to upload caches
	'/(upload|thumbs)/' => array(
		'host' => '<ip for upload caches>'
	),

	// Everything else to text & mobile
	'' => array(
		array( 'host' => '<ip for text caches>' ),
		array( 'host' => '<ip for mobile caches>' ),
	),

);

Change-Id: Ie87f6b81007f47f9cb439371743f3ad9a4b0c774

diff --git a/RELEASE-NOTES-1.22 b/RELEASE-NOTES-1.22
index e67bb00..ee98ed8 100644 (file)
--- a/RELEASE-NOTES-1.22
+++ b/RELEASE-NOTES-1.22
@@ -167,6 +167,10 @@ production.
 * (bug 30713) New mw.hook "wikipage.content".
 * (bug 40430) jquery.placeholder gets a new parameter to set the attribute value
   to be used.
+* $wgHTCPMulticastRouting renamed $wgHTCPRouting since it accepts unicast.
+* $wgHTCPRouting rules can now be passed an array of hosts/ports to send purge
+  too. Can be used whenever several multicast group could be interested by a
+  specific purge.
 
 === Bug fixes in 1.22 ===
 * Disable Special:PasswordReset when $wgEnableEmail is false. Previously one

diff --git a/includes/DefaultSettings.php b/includes/DefaultSettings.php
index f8e67f5..5eb1bdd 100644 (file)
--- a/includes/DefaultSettings.php
+++ b/includes/DefaultSettings.php
@@ -2166,12 +2166,12 @@ $wgSquidPurgeUseHostHeader = true;
  * Each key in this array is a regular expression to match against the purged
  * URL, or an empty string to match all URLs. The purged URL is matched against
  * the regexes in the order specified, and the first rule whose regex matches
- * is used.
+ * is used, all remaining rules will thus be ignored.
  *
- * Example configuration to send purges for upload.wikimedia.org to one
+ * @par Example configuration to send purges for upload.wikimedia.org to one
  * multicast group and all other purges to another:
  * @code
- * $wgHTCPMulticastRouting = array(
+ * $wgHTCPRouting = array(
  *         '|^https?://upload\.wikimedia\.org|' => array(
  *                 'host' => '239.128.0.113',
  *                 'port' => 4827,
@@ -2183,11 +2183,44 @@ $wgSquidPurgeUseHostHeader = true;
  * );
  * @endcode
  *
- * @since 1.20
+ * You can also pass an array of hosts to send purges too. This is useful when
+ * you have several multicast groups or unicast address that should receive a
+ * given purge.  Multiple hosts support was introduced in MediaWiki 1.22.
+ *
+ * @par Example of sending purges to multiple hosts:
+ * @code
+ * $wgHTCPRouting = array(
+ *     '' => array(
+ *         // Purges to text caches using multicast
+ *         array( 'host' => '239.128.0.114', 'port' => '4827' ),
+ *         // Purges to a hardcoded list of caches
+ *         array( 'host' => '10.88.66.1', 'port' => '4827' ),
+ *         array( 'host' => '10.88.66.2', 'port' => '4827' ),
+ *         array( 'host' => '10.88.66.3', 'port' => '4827' ),
+ *     ),
+ * );
+ * @endcode
+ *
+ * @since 1.22
+ *
+ * $wgHTCPRouting replaces $wgHTCPMulticastRouting that was introduced in 1.20.
+ * For back compatibility purposes, whenever its array is empty
+ * $wgHTCPMutlicastRouting will be used as a fallback if it not null.
  *
  * @see $wgHTCPMulticastTTL
  */
-$wgHTCPMulticastRouting = array();
+$wgHTCPRouting = array();
+
+/**
+ * @deprecated since 1.22, please use $wgHTCPRouting instead.
+ *
+ * Whenever this is set and $wgHTCPRouting evaluates to false, $wgHTCPRouting 
+ * will be set to this value.
+ * This is merely for back compatibility.
+ *
+ * @since 1.20
+ */
+$wgHTCPMulticastRouting = null;
 
 /**
  * HTCP multicast address. Set this to a multicast IP address to enable HTCP.
@@ -2195,25 +2228,28 @@ $wgHTCPMulticastRouting = array();
  * Note that MediaWiki uses the old non-RFC compliant HTCP format, which was
  * present in the earliest Squid implementations of the protocol.
  *
- * This setting is DEPRECATED in favor of $wgHTCPMulticastRouting , and kept
- * for backwards compatibility only. If $wgHTCPMulticastRouting is set, this
- * setting is ignored. If $wgHTCPMulticastRouting is not set and this setting
- * is, it is used to populate $wgHTCPMulticastRouting.
+ * This setting is DEPRECATED in favor of $wgHTCPRouting , and kept for
+ * backwards compatibility only. If $wgHTCPRouting is set, this setting is
+ * ignored. If $wgHTCPRouting is not set and this setting is, it is used to
+ * populate $wgHTCPRouting.
  *
- * @deprecated since 1.20 in favor of $wgHTCPMulticastRouting
+ * @deprecated since 1.20 in favor of $wgHTCPMulticastRouting and since 1.22 in
+ * favor of $wgHTCPRouting.
  */
 $wgHTCPMulticastAddress = false;
 
 /**
  * HTCP multicast port.
- * @deprecated since 1.20 in favor of $wgHTCPMulticastRouting
+ * @deprecated since 1.20 in favor of $wgHTCPMulticastRouting and since 1.22 in
+ * favor of $wgHTCPRouting.
+ *
  * @see $wgHTCPMulticastAddress
  */
 $wgHTCPPort = 4827;
 
 /**
  * HTCP multicast TTL.
- * @see $wgHTCPMulticastRouting
+ * @see $wgHTCPRouting
  */
 $wgHTCPMulticastTTL = 1;
 

diff --git a/includes/Setup.php b/includes/Setup.php
index 028758f..4f5ac92 100644 (file)
--- a/includes/Setup.php
+++ b/includes/Setup.php
@@ -434,9 +434,16 @@ if ( $wgCanonicalServer === false ) {
        $wgCanonicalServer = wfExpandUrl( $wgServer, PROTO_HTTP );
 }
 
-// Initialize $wgHTCPMulticastRouting from backwards-compatible settings
-if ( !$wgHTCPMulticastRouting && $wgHTCPMulticastAddress ) {
-       $wgHTCPMulticastRouting = array(
+// $wgHTCPMulticastRouting got renamed to $wgHTCPRouting in MediaWiki 1.22
+// ensure back compatibility.
+if ( !$wgHTCPRouting && $wgHTCPMulticastRouting ) {
+       $wgHTCPRouting = $wgHTCPMulticastRouting;
+}
+
+// Initialize $wgHTCPRouting from backwards-compatible settings that
+// comes from pre 1.20 version.
+if ( !$wgHTCPRouting && $wgHTCPMulticastAddress ) {
+       $wgHTCPRouting = array(
                '' => array(
                        'host' => $wgHTCPMulticastAddress,
                        'port' => $wgHTCPPort,

diff --git a/includes/cache/SquidUpdate.php b/includes/cache/SquidUpdate.php
index 6cc59b7..ebac788 100644 (file)
--- a/includes/cache/SquidUpdate.php
+++ b/includes/cache/SquidUpdate.php
@@ -123,7 +123,7 @@ class SquidUpdate {
         * @return void
         */
        static function purge( $urlArr ) {
-               global $wgSquidServers, $wgHTCPMulticastRouting;
+               global $wgSquidServers, $wgHTCPRouting;
 
                if ( !$urlArr ) {
                        return;
@@ -131,7 +131,7 @@ class SquidUpdate {
 
                wfDebugLog( 'squid', __METHOD__ . ': ' . implode( ' ', $urlArr ) . "\n" );
 
-               if ( $wgHTCPMulticastRouting ) {
+               if ( $wgHTCPRouting ) {
                        SquidUpdate::HTCPPurge( $urlArr );
                }
 
@@ -166,7 +166,7 @@ class SquidUpdate {
         * @param $urlArr array
         */
        static function HTCPPurge( $urlArr ) {
-               global $wgHTCPMulticastRouting, $wgHTCPMulticastTTL;
+               global $wgHTCPRouting, $wgHTCPMulticastTTL;
                wfProfileIn( __METHOD__ );
 
                $htcpOpCLR = 4; // HTCP CLR
@@ -202,15 +202,22 @@ class SquidUpdate {
                                throw new MWException( 'Bad purge URL' );
                        }
                        $url = SquidUpdate::expand( $url );
-                       $conf = self::getRuleForURL( $url, $wgHTCPMulticastRouting );
+                       $conf = self::getRuleForURL( $url, $wgHTCPRouting );
                        if ( !$conf ) {
                                wfDebugLog( 'squid', __METHOD__ .
                                        "No HTCP rule configured for URL $url , skipping\n" );
                                continue;
                        }
-                       if ( !isset( $conf['host'] ) || !isset( $conf['port'] ) ) {
-                               wfProfileOut( __METHOD__ );
-                               throw new MWException( "Invalid HTCP rule for URL $url\n" );
+
+                       if( isset( $conf['host'] ) && isset( $conf['port'] ) ) {
+                               // Normalize single entries
+                               $conf = array( $conf );
+                       }
+                       foreach( $conf as $subconf ) {
+                               if ( !isset( $subconf['host'] ) || !isset( $subconf['port'] ) ) {
+                                       wfProfileOut( __METHOD__ );
+                                       throw new MWException( "Invalid HTCP rule for URL $url\n" );
+                               }
                        }
 
                        // Construct a minimal HTCP request diagram
@@ -232,11 +239,12 @@ class SquidUpdate {
                                $htcpLen, $htcpDataLen, $htcpOpCLR,
                                $htcpTransID, $htcpSpecifier, 2 );
 
-                       // Send out
                        wfDebugLog( 'squid', __METHOD__ .
                                "Purging URL $url via HTCP\n" );
-                       socket_sendto( $conn, $htcpPacket, $htcpLen, 0,
-                               $conf['host'], $conf['port'] );
+                       foreach( $conf as $subconf ) {
+                               socket_sendto( $conn, $htcpPacket, $htcpLen, 0,
+                                       $subconf['host'], $subconf['port'] );
+                       }
                }
                wfProfileOut( __METHOD__ );
        }
@@ -263,7 +271,7 @@ class SquidUpdate {
        /**
         * Find the HTCP routing rule to use for a given URL.
         * @param string $url URL to match
-        * @param array $rules Array of rules, see $wgHTCPMulticastRouting for format and behavior
+        * @param array $rules Array of rules, see $wgHTCPRouting for format and behavior
         * @return mixed Element of $rules that matched, or false if nothing matched
         */
        static function getRuleForURL( $url, $rules ) {