3b2479f1ebe7a38120db403d855f7df1229e143a
3 * Database load balancing interface
5 * This program is free software; you can redistribute it and/or modify
6 * it under the terms of the GNU General Public License as published by
7 * the Free Software Foundation; either version 2 of the License, or
8 * (at your option) any later version.
10 * This program is distributed in the hope that it will be useful,
11 * but WITHOUT ANY WARRANTY; without even the implied warranty of
12 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13 * GNU General Public License for more details.
15 * You should have received a copy of the GNU General Public License along
16 * with this program; if not, write to the Free Software Foundation, Inc.,
17 * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
18 * http://www.gnu.org/copyleft/gpl.html
22 * @author Aaron Schulz
26 * Database cluster connection, tracking, load balancing, and transaction manager interface
28 * A "cluster" is considered to be one master database and zero or more replica databases.
29 * Typically, the replica DBs replicate from the master asynchronously. The first node in the
30 * "servers" configuration array is always considered the "master". However, this class can still
31 * be used when all or some of the "replica" DBs are multi-master peers of the master or even
32 * when all the DBs are non-replicating clones of each other holding read-only data. Thus, the
33 * role of "master" is in some cases merely nominal.
35 * By default, each DB server uses DBO_DEFAULT for its 'flags' setting, unless explicitly set
36 * otherwise in configuration. DBO_DEFAULT behavior depends on whether 'cliMode' is set:
37 * - In CLI mode, the flag has no effect with regards to LoadBalancer.
38 * - In non-CLI mode, the flag causes implicit transactions to be used; the first query on
39 * a database starts a transaction on that database. The transactions are meant to remain
40 * pending until either commitMasterChanges() or rollbackMasterChanges() is called. The
41 * application must have some point where it calls commitMasterChanges() near the end of
43 * Every iteration of beginMasterChanges()/commitMasterChanges() is called a "transaction round".
44 * Rounds are useful on the master DB connections because they make single-DB (and by and large
45 * multi-DB) updates in web requests all-or-nothing. Also, transactions on replica DBs are useful
46 * when REPEATABLE-READ or SERIALIZABLE isolation is used because all foriegn keys and constraints
47 * hold across separate queries in the DB transaction since the data appears within a consistent
48 * point-in-time snapshot.
50 * The typical caller will use LoadBalancer::getConnection( DB_* ) to yield a live database
51 * connection handle. The choice of which DB server to use is based on pre-defined loads for
52 * weighted random selection, adjustments thereof by LoadMonitor, and the amount of replication
53 * lag on each DB server. Lag checks might cause problems in certain setups, so they should be
54 * tuned in the server configuration maps as follows:
55 * - Master + N Replica(s): set 'max lag' to an appropriate threshold for avoiding any database
56 * lagged by this much or more. If all DBs are this lagged, then the load balancer considers
57 * the cluster to be read-only.
58 * - Galera Cluster: Seconds_Behind_Master will be 0, so there probably is nothing to tune.
59 * Note that lag is still possible depending on how wsrep-sync-wait is set server-side.
60 * - Read-only archive clones: set 'is static' in the server configuration maps. This will
61 * treat all such DBs as having 0 lag.
62 * - SQL load balancing proxy: any proxy should handle lag checks on its own, so the 'max lag'
63 * parameter should probably be set to INF in the server configuration maps. This will make
64 * the load balancer ignore whatever it detects as the lag of the logical replica is (which
65 * would probably just randomly bounce around).
67 * If using a SQL proxy service, it would probably be best to have two proxy hosts for the
68 * load balancer to talk to. One would be the 'host' of the master server entry and another for
69 * the (logical) replica server entry. The proxy could map the load balancer's "replica" DB to
70 * any number of physical replica DBs.
75 interface ILoadBalancer
{
77 * Construct a manager of IDatabase connection objects
79 * @param array $params Parameter map with keys:
80 * - servers : Required. Array of server info structures.
81 * - localDomain: A DatabaseDomain or domain ID string.
82 * - loadMonitor : Name of a class used to fetch server lag and load.
83 * - readOnlyReason : Reason the master DB is read-only if so [optional]
84 * - waitTimeout : Maximum time to wait for replicas for consistency [optional]
85 * - srvCache : BagOStuff object for server cache [optional]
86 * - memCache : BagOStuff object for cluster memory cache [optional]
87 * - wanCache : WANObjectCache object [optional]
88 * - hostname : The name of the current server [optional]
89 * - cliMode: Whether the execution context is a CLI script. [optional]
90 * - profiler : Class name or instance with profileIn()/profileOut() methods. [optional]
91 * - trxProfiler: TransactionProfiler instance. [optional]
92 * - replLogger: PSR-3 logger instance. [optional]
93 * - connLogger: PSR-3 logger instance. [optional]
94 * - queryLogger: PSR-3 logger instance. [optional]
95 * - perfLogger: PSR-3 logger instance. [optional]
96 * - errorLogger : Callback that takes an Exception and logs it. [optional]
97 * @throws InvalidArgumentException
99 public function __construct( array $params );
102 * Get the index of the reader connection, which may be a replica DB
103 * This takes into account load ratios and lag times. It should
104 * always return a consistent index during a given invocation
106 * Side effect: opens connections to databases
107 * @param string|bool $group Query group, or false for the generic reader
108 * @param string|bool $domain Domain ID, or false for the current domain
110 * @return bool|int|string
112 public function getReaderIndex( $group = false, $domain = false );
115 * Set the master wait position
116 * If a DB_REPLICA connection has been opened already, waits
117 * Otherwise sets a variable telling it to wait if such a connection is opened
118 * @param DBMasterPos $pos
120 public function waitFor( $pos );
123 * Set the master wait position and wait for a "generic" replica DB to catch up to it
125 * This can be used a faster proxy for waitForAll()
127 * @param DBMasterPos $pos
128 * @param int $timeout Max seconds to wait; default is mWaitTimeout
129 * @return bool Success (able to connect and no timeouts reached)
131 public function waitForOne( $pos, $timeout = null );
134 * Set the master wait position and wait for ALL replica DBs to catch up to it
135 * @param DBMasterPos $pos
136 * @param int $timeout Max seconds to wait; default is mWaitTimeout
137 * @return bool Success (able to connect and no timeouts reached)
139 public function waitForAll( $pos, $timeout = null );
142 * Get any open connection to a given server index, local or foreign
143 * Returns false if there is no connection open
145 * @param int $i Server index
146 * @return IDatabase|bool False on failure
148 public function getAnyOpenConnection( $i );
151 * Get a connection by index
152 * This is the main entry point for this class.
154 * @param int $i Server index
155 * @param array|string|bool $groups Query group(s), or false for the generic reader
156 * @param string|bool $domain Domain ID, or false for the current domain
161 public function getConnection( $i, $groups = [], $domain = false );
164 * Mark a foreign connection as being available for reuse under a different
165 * DB name or prefix. This mechanism is reference-counted, and must be called
166 * the same number of times as getConnection() to work.
168 * @param IDatabase $conn
169 * @throws InvalidArgumentException
171 public function reuseConnection( $conn );
174 * Get a database connection handle reference
176 * The handle's methods wrap simply wrap those of a IDatabase handle
178 * @see LoadBalancer::getConnection() for parameter information
181 * @param array|string|bool $groups Query group(s), or false for the generic reader
182 * @param string|bool $domain Domain ID, or false for the current domain
185 public function getConnectionRef( $db, $groups = [], $domain = false );
188 * Get a database connection handle reference without connecting yet
190 * The handle's methods wrap simply wrap those of a IDatabase handle
192 * @see LoadBalancer::getConnection() for parameter information
195 * @param array|string|bool $groups Query group(s), or false for the generic reader
196 * @param string|bool $domain Domain ID, or false for the current domain
199 public function getLazyConnectionRef( $db, $groups = [], $domain = false );
202 * Open a connection to the server given by the specified index
203 * Index must be an actual index into the array.
204 * If the server is already open, returns it.
206 * On error, returns false, and the connection which caused the
207 * error will be available via $this->mErrorConnection.
209 * @note If disable() was called on this LoadBalancer, this method will throw a DBAccessError.
211 * @param int $i Server index
212 * @param string|bool $domain Domain ID, or false for the current domain
213 * @return IDatabase|bool Returns false on errors
215 public function openConnection( $i, $domain = false );
220 public function getWriterIndex();
223 * Returns true if the specified index is a valid server index
228 public function haveIndex( $i );
231 * Returns true if the specified index is valid and has non-zero load
236 public function isNonZeroLoad( $i );
239 * Get the number of defined servers (not the number of open connections)
243 public function getServerCount();
246 * Get the host name or IP address of the server with the specified index
247 * Prefer a readable name if available.
251 public function getServerName( $i );
254 * Return the server info structure for a given index, or false if the index is invalid.
258 public function getServerInfo( $i );
261 * Sets the server info structure for the given index. Entry at index $i
262 * is created if it doesn't exist
264 * @param array $serverInfo
266 public function setServerInfo( $i, array $serverInfo );
269 * Get the current master position for chronology control purposes
270 * @return DBMasterPos|bool Returns false if not applicable
272 public function getMasterPos();
275 * Disable this load balancer. All connections are closed, and any attempt to
276 * open a new connection will result in a DBAccessError.
278 public function disable();
281 * Close all open connections
283 public function closeAll();
288 * Using this function makes sure the LoadBalancer knows the connection is closed.
289 * If you use $conn->close() directly, the load balancer won't update its state.
291 * @param IDatabase $conn
293 public function closeConnection( IDatabase
$conn );
296 * Commit transactions on all open connections
297 * @param string $fname Caller name
298 * @throws DBExpectedError
300 public function commitAll( $fname = __METHOD__
);
303 * Perform all pre-commit callbacks that remain part of the atomic transactions
304 * and disable any post-commit callbacks until runMasterPostTrxCallbacks()
306 * Use this only for mutli-database commits
308 public function finalizeMasterChanges();
311 * Perform all pre-commit checks for things like replication safety
313 * Use this only for mutli-database commits
315 * @param array $options Includes:
316 * - maxWriteDuration : max write query duration time in seconds
317 * @throws DBTransactionError
319 public function approveMasterChanges( array $options );
322 * Flush any master transaction snapshots and set DBO_TRX (if DBO_DEFAULT is set)
324 * The DBO_TRX setting will be reverted to the default in each of these methods:
325 * - commitMasterChanges()
326 * - rollbackMasterChanges()
328 * This allows for custom transaction rounds from any outer transaction scope.
330 * @param string $fname
331 * @throws DBExpectedError
333 public function beginMasterChanges( $fname = __METHOD__
);
336 * Issue COMMIT on all master connections where writes where done
337 * @param string $fname Caller name
338 * @throws DBExpectedError
340 public function commitMasterChanges( $fname = __METHOD__
);
343 * Issue all pending post-COMMIT/ROLLBACK callbacks
345 * Use this only for mutli-database commits
347 * @param integer $type IDatabase::TRIGGER_* constant
348 * @return Exception|null The first exception or null if there were none
350 public function runMasterPostTrxCallbacks( $type );
353 * Issue ROLLBACK only on master, only if queries were done on connection
354 * @param string $fname Caller name
355 * @throws DBExpectedError
357 public function rollbackMasterChanges( $fname = __METHOD__
);
360 * Suppress all pending post-COMMIT/ROLLBACK callbacks
362 * Use this only for mutli-database commits
364 * @return Exception|null The first exception or null if there were none
366 public function suppressTransactionEndCallbacks();
369 * Commit all replica DB transactions so as to flush any REPEATABLE-READ or SSI snapshot
371 * @param string $fname Caller name
373 public function flushReplicaSnapshots( $fname = __METHOD__
);
376 * @return bool Whether a master connection is already open
378 public function hasMasterConnection();
381 * Determine if there are pending changes in a transaction by this thread
384 public function hasMasterChanges();
387 * Get the timestamp of the latest write query done by this thread
388 * @return float|bool UNIX timestamp or false
390 public function lastMasterChangeTimestamp();
393 * Check if this load balancer object had any recent or still
394 * pending writes issued against it by this PHP thread
396 * @param float $age How many seconds ago is "recent" [defaults to mWaitTimeout]
399 public function hasOrMadeRecentMasterChanges( $age = null );
402 * Get the list of callers that have pending master changes
404 * @return string[] List of method names
406 public function pendingMasterChangeCallers();
409 * @note This method will trigger a DB connection if not yet done
410 * @param string|bool $domain Domain ID, or false for the current domain
411 * @return bool Whether the generic connection for reads is highly "lagged"
413 public function getLaggedReplicaMode( $domain = false );
416 * @note This method will never cause a new DB connection
417 * @return bool Whether any generic connection used for reads was highly "lagged"
419 public function laggedReplicaUsed();
422 * @note This method may trigger a DB connection if not yet done
423 * @param string|bool $domain Domain ID, or false for the current domain
424 * @param IDatabase|null DB master connection; used to avoid loops [optional]
425 * @return string|bool Reason the master is read-only or false if it is not
427 public function getReadOnlyReason( $domain = false, IDatabase
$conn = null );
430 * Disables/enables lag checks
431 * @param null|bool $mode
434 public function allowLagged( $mode = null );
439 public function pingAll();
442 * Call a function with each open connection object
443 * @param callable $callback
444 * @param array $params
446 public function forEachOpenConnection( $callback, array $params = [] );
449 * Call a function with each open connection object to a master
450 * @param callable $callback
451 * @param array $params
453 public function forEachOpenMasterConnection( $callback, array $params = [] );
456 * Call a function with each open replica DB connection object
457 * @param callable $callback
458 * @param array $params
460 public function forEachOpenReplicaConnection( $callback, array $params = [] );
463 * Get the hostname and lag time of the most-lagged replica DB
465 * This is useful for maintenance scripts that need to throttle their updates.
466 * May attempt to open connections to replica DBs on the default DB. If there is
467 * no lag, the maximum lag will be reported as -1.
469 * @param bool|string $domain Domain ID, or false for the default database
470 * @return array ( host, max lag, index of max lagged host )
472 public function getMaxLag( $domain = false );
475 * Get an estimate of replication lag (in seconds) for each server
477 * Results are cached for a short time in memcached/process cache
479 * Values may be "false" if replication is too broken to estimate
481 * @param string|bool $domain
482 * @return int[] Map of (server index => float|int|bool)
484 public function getLagTimes( $domain = false );
487 * Get the lag in seconds for a given connection, or zero if this load
488 * balancer does not have replication enabled.
490 * This should be used in preference to Database::getLag() in cases where
491 * replication may not be in use, since there is no way to determine if
492 * replication is in use at the connection level without running
493 * potentially restricted queries such as SHOW SLAVE STATUS. Using this
494 * function instead of Database::getLag() avoids a fatal error in this
495 * case on many installations.
497 * @param IDatabase $conn
498 * @return int|bool Returns false on error
500 public function safeGetLag( IDatabase
$conn );
503 * Wait for a replica DB to reach a specified master position
505 * This will connect to the master to get an accurate position if $pos is not given
507 * @param IDatabase $conn Replica DB
508 * @param DBMasterPos|bool $pos Master position; default: current position
509 * @param integer|null $timeout Timeout in seconds [optional]
510 * @return bool Success
512 public function safeWaitForMasterPos( IDatabase
$conn, $pos = false, $timeout = null );
515 * Clear the cache for slag lag delay times
517 * This is only used for testing
519 public function clearLagTimeCache();
522 * Set a callback via IDatabase::setTransactionListener() on
523 * all current and future master connections of this load balancer
525 * @param string $name Callback name
526 * @param callable|null $callback
528 public function setTransactionListener( $name, callable
$callback = null );
531 * Set a new table prefix for the existing local domain ID for testing
533 * @param string $prefix
535 public function setDomainPrefix( $prefix );
538 * Make certain table names use their own database, schema, and table prefix
539 * when passed into SQL queries pre-escaped and without a qualified database name
541 * For example, "user" can be converted to "myschema.mydbname.user" for convenience.
542 * Appearances like `user`, somedb.user, somedb.someschema.user will used literally.
544 * Calling this twice will completely clear any old table aliases. Also, note that
545 * callers are responsible for making sure the schemas and databases actually exist.
547 * @param array[] $aliases Map of (table => (dbname, schema, prefix) map)
549 public function setTableAliases( array $aliases );