Use LB server configuration to force DB domains in ExternalStorageDB
[lhc/web/wiklou.git] / includes / libs / rdbms / loadbalancer / ILoadBalancer.php
1 <?php
2 /**
3 * Database load balancing interface
4 *
5 * This program is free software; you can redistribute it and/or modify
6 * it under the terms of the GNU General Public License as published by
7 * the Free Software Foundation; either version 2 of the License, or
8 * (at your option) any later version.
9 *
10 * This program is distributed in the hope that it will be useful,
11 * but WITHOUT ANY WARRANTY; without even the implied warranty of
12 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13 * GNU General Public License for more details.
14 *
15 * You should have received a copy of the GNU General Public License along
16 * with this program; if not, write to the Free Software Foundation, Inc.,
17 * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
18 * http://www.gnu.org/copyleft/gpl.html
19 *
20 * @file
21 * @ingroup Database
22 */
23 namespace Wikimedia\Rdbms;
24
25 use Exception;
26 use InvalidArgumentException;
27
28 /**
29 * Database cluster connection, tracking, load balancing, and transaction manager interface
30 *
31 * A "cluster" is considered to be one master database and zero or more replica databases.
32 * Typically, the replica DBs replicate from the master asynchronously. The first node in the
33 * "servers" configuration array is always considered the "master". However, this class can still
34 * be used when all or some of the "replica" DBs are multi-master peers of the master or even
35 * when all the DBs are non-replicating clones of each other holding read-only data. Thus, the
36 * role of "master" is in some cases merely nominal.
37 *
38 * By default, each DB server uses DBO_DEFAULT for its 'flags' setting, unless explicitly set
39 * otherwise in configuration. DBO_DEFAULT behavior depends on whether 'cliMode' is set:
40 * - In CLI mode, the flag has no effect with regards to LoadBalancer.
41 * - In non-CLI mode, the flag causes implicit transactions to be used; the first query on
42 * a database starts a transaction on that database. The transactions are meant to remain
43 * pending until either commitMasterChanges() or rollbackMasterChanges() is called. The
44 * application must have some point where it calls commitMasterChanges() near the end of
45 * the PHP request.
46 * Every iteration of beginMasterChanges()/commitMasterChanges() is called a "transaction round".
47 * Rounds are useful on the master DB connections because they make single-DB (and by and large
48 * multi-DB) updates in web requests all-or-nothing. Also, transactions on replica DBs are useful
49 * when REPEATABLE-READ or SERIALIZABLE isolation is used because all foriegn keys and constraints
50 * hold across separate queries in the DB transaction since the data appears within a consistent
51 * point-in-time snapshot.
52 *
53 * The typical caller will use LoadBalancer::getConnection( DB_* ) to yield a live database
54 * connection handle. The choice of which DB server to use is based on pre-defined loads for
55 * weighted random selection, adjustments thereof by LoadMonitor, and the amount of replication
56 * lag on each DB server. Lag checks might cause problems in certain setups, so they should be
57 * tuned in the server configuration maps as follows:
58 * - Master + N Replica(s): set 'max lag' to an appropriate threshold for avoiding any database
59 * lagged by this much or more. If all DBs are this lagged, then the load balancer considers
60 * the cluster to be read-only.
61 * - Galera Cluster: Seconds_Behind_Master will be 0, so there probably is nothing to tune.
62 * Note that lag is still possible depending on how wsrep-sync-wait is set server-side.
63 * - Read-only archive clones: set 'is static' in the server configuration maps. This will
64 * treat all such DBs as having 0 lag.
65 * - SQL load balancing proxy: any proxy should handle lag checks on its own, so the 'max lag'
66 * parameter should probably be set to INF in the server configuration maps. This will make
67 * the load balancer ignore whatever it detects as the lag of the logical replica is (which
68 * would probably just randomly bounce around).
69 *
70 * If using a SQL proxy service, it would probably be best to have two proxy hosts for the
71 * load balancer to talk to. One would be the 'host' of the master server entry and another for
72 * the (logical) replica server entry. The proxy could map the load balancer's "replica" DB to
73 * any number of physical replica DBs.
74 *
75 * @since 1.28
76 * @ingroup Database
77 */
78 interface ILoadBalancer {
79 /** @var int Request a replica DB connection */
80 const DB_REPLICA = -1;
81 /** @var int Request a master DB connection */
82 const DB_MASTER = -2;
83
84 /** @var string Domain specifier when no specific database needs to be selected */
85 const DOMAIN_ANY = '';
86
87 /** @var int DB handle should have DBO_TRX disabled and the caller will leave it as such */
88 const CONN_TRX_AUTOCOMMIT = 1;
89 /** @var int Alias for CONN_TRX_AUTOCOMMIT for b/c; deprecated since 1.31 */
90 const CONN_TRX_AUTO = 1;
91
92 /**
93 * Construct a manager of IDatabase connection objects
94 *
95 * @param array $params Parameter map with keys:
96 * - servers : Required. Array of server info structures.
97 * - localDomain: A DatabaseDomain or domain ID string.
98 * - loadMonitor : Name of a class used to fetch server lag and load.
99 * - readOnlyReason : Reason the master DB is read-only if so [optional]
100 * - waitTimeout : Maximum time to wait for replicas for consistency [optional]
101 * - maxLag: Avoid replica DB servers with more lag than this [optional]
102 * - srvCache : BagOStuff object for server cache [optional]
103 * - wanCache : WANObjectCache object [optional]
104 * - chronologyCallback: Callback to run before the first connection attempt [optional]
105 * - hostname : The name of the current server [optional]
106 * - cliMode: Whether the execution context is a CLI script. [optional]
107 * - profiler : Class name or instance with profileIn()/profileOut() methods. [optional]
108 * - trxProfiler: TransactionProfiler instance. [optional]
109 * - replLogger: PSR-3 logger instance. [optional]
110 * - connLogger: PSR-3 logger instance. [optional]
111 * - queryLogger: PSR-3 logger instance. [optional]
112 * - perfLogger: PSR-3 logger instance. [optional]
113 * - errorLogger : Callback that takes an Exception and logs it. [optional]
114 * - deprecationLogger: Callback to log a deprecation warning. [optional]
115 * @throws InvalidArgumentException
116 */
117 public function __construct( array $params );
118
119 /**
120 * Get the index of the reader connection, which may be a replica DB
121 *
122 * This takes into account load ratios and lag times. It should
123 * always return a consistent index during a given invocation.
124 *
125 * Side effect: opens connections to databases
126 * @param string|bool $group Query group, or false for the generic reader
127 * @param string|bool $domain Domain ID, or false for the current domain
128 * @throws DBError
129 * @return bool|int|string
130 */
131 public function getReaderIndex( $group = false, $domain = false );
132
133 /**
134 * Set the master wait position
135 *
136 * If a DB_REPLICA connection has been opened already, then wait immediately.
137 * Otherwise sets a variable telling it to wait if such a connection is opened.
138 *
139 * This only applies to connections to the generic replica DB for this request.
140 * If a timeout happens when waiting, then getLaggedReplicaMode()/laggedReplicaUsed()
141 * will return true.
142 *
143 * @param DBMasterPos|bool $pos Master position or false
144 */
145 public function waitFor( $pos );
146
147 /**
148 * Set the master wait position and wait for a "generic" replica DB to catch up to it
149 *
150 * This can be used a faster proxy for waitForAll()
151 *
152 * @param DBMasterPos|bool $pos Master position or false
153 * @param int $timeout Max seconds to wait; default is mWaitTimeout
154 * @return bool Success (able to connect and no timeouts reached)
155 */
156 public function waitForOne( $pos, $timeout = null );
157
158 /**
159 * Set the master wait position and wait for ALL replica DBs to catch up to it
160 *
161 * @param DBMasterPos|bool $pos Master position or false
162 * @param int $timeout Max seconds to wait; default is mWaitTimeout
163 * @return bool Success (able to connect and no timeouts reached)
164 */
165 public function waitForAll( $pos, $timeout = null );
166
167 /**
168 * Get any open connection to a given server index, local or foreign
169 *
170 * Use CONN_TRX_AUTOCOMMIT to only look for connections opened with that flag
171 *
172 * @param int $i Server index or DB_MASTER/DB_REPLICA
173 * @param int $flags Bitfield of CONN_* class constants
174 * @return Database|bool False if no such connection is open
175 */
176 public function getAnyOpenConnection( $i, $flags = 0 );
177
178 /**
179 * Get a connection handle by server index
180 *
181 * The CONN_TRX_AUTOCOMMIT flag is ignored for databases with ATTR_DB_LEVEL_LOCKING
182 * (e.g. sqlite) in order to avoid deadlocks. ILoadBalancer::getServerAttributes()
183 * can be used to check such flags beforehand.
184 *
185 * If the caller uses $domain or sets CONN_TRX_AUTOCOMMIT in $flags, then it must also
186 * call ILoadBalancer::reuseConnection() on the handle when finished using it.
187 * In all other cases, this is not necessary, though not harmful either.
188 *
189 * @param int $i Server index (overrides $groups) or DB_MASTER/DB_REPLICA
190 * @param array|string|bool $groups Query group(s), or false for the generic reader
191 * @param string|bool $domain Domain ID, or false for the current domain
192 * @param int $flags Bitfield of CONN_* class constants
193 *
194 * @note This method throws DBAccessError if ILoadBalancer::disable() was called
195 *
196 * @throws DBError
197 * @return Database
198 */
199 public function getConnection( $i, $groups = [], $domain = false, $flags = 0 );
200
201 /**
202 * Mark a foreign connection as being available for reuse under a different DB domain
203 *
204 * This mechanism is reference-counted, and must be called the same number of times
205 * as getConnection() to work.
206 *
207 * @param IDatabase $conn
208 * @throws InvalidArgumentException
209 */
210 public function reuseConnection( IDatabase $conn );
211
212 /**
213 * Get a database connection handle reference
214 *
215 * The handle's methods simply wrap those of a Database handle
216 *
217 * The CONN_TRX_AUTOCOMMIT flag is ignored for databases with ATTR_DB_LEVEL_LOCKING
218 * (e.g. sqlite) in order to avoid deadlocks. ILoadBalancer::getServerAttributes()
219 * can be used to check such flags beforehand.
220 *
221 * @see ILoadBalancer::getConnection() for parameter information
222 *
223 * @param int $i Server index or DB_MASTER/DB_REPLICA
224 * @param array|string|bool $groups Query group(s), or false for the generic reader
225 * @param string|bool $domain Domain ID, or false for the current domain
226 * @param int $flags Bitfield of CONN_* class constants (e.g. CONN_TRX_AUTOCOMMIT)
227 * @return DBConnRef
228 */
229 public function getConnectionRef( $i, $groups = [], $domain = false, $flags = 0 );
230
231 /**
232 * Get a database connection handle reference without connecting yet
233 *
234 * The handle's methods simply wrap those of a Database handle
235 *
236 * The CONN_TRX_AUTOCOMMIT flag is ignored for databases with ATTR_DB_LEVEL_LOCKING
237 * (e.g. sqlite) in order to avoid deadlocks. ILoadBalancer::getServerAttributes()
238 * can be used to check such flags beforehand.
239 *
240 * @see ILoadBalancer::getConnection() for parameter information
241 *
242 * @param int $i Server index or DB_MASTER/DB_REPLICA
243 * @param array|string|bool $groups Query group(s), or false for the generic reader
244 * @param string|bool $domain Domain ID, or false for the current domain
245 * @param int $flags Bitfield of CONN_* class constants (e.g. CONN_TRX_AUTOCOMMIT)
246 * @return DBConnRef
247 */
248 public function getLazyConnectionRef( $i, $groups = [], $domain = false, $flags = 0 );
249
250 /**
251 * Get a maintenance database connection handle reference for migrations and schema changes
252 *
253 * The handle's methods simply wrap those of a Database handle
254 *
255 * The CONN_TRX_AUTOCOMMIT flag is ignored for databases with ATTR_DB_LEVEL_LOCKING
256 * (e.g. sqlite) in order to avoid deadlocks. ILoadBalancer::getServerAttributes()
257 * can be used to check such flags beforehand.
258 *
259 * @see ILoadBalancer::getConnection() for parameter information
260 *
261 * @param int $db Server index or DB_MASTER/DB_REPLICA
262 * @param array|string|bool $groups Query group(s), or false for the generic reader
263 * @param string|bool $domain Domain ID, or false for the current domain
264 * @param int $flags Bitfield of CONN_* class constants (e.g. CONN_TRX_AUTOCOMMIT)
265 * @return MaintainableDBConnRef
266 */
267 public function getMaintenanceConnectionRef( $db, $groups = [], $domain = false, $flags = 0 );
268
269 /**
270 * Open a connection to the server given by the specified index
271 *
272 * The index must be an actual index into the array. If a connection to the server is
273 * already open and not considered an "in use" foreign connection, this simply returns it.
274 *
275 * Avoid using CONN_TRX_AUTOCOMMIT for databases with ATTR_DB_LEVEL_LOCKING (e.g. sqlite) in
276 * order to avoid deadlocks. ILoadBalancer::getServerAttributes() can be used to check
277 * such flags beforehand.
278 *
279 * If the caller uses $domain or sets CONN_TRX_AUTOCOMMIT in $flags, then it must also
280 * call ILoadBalancer::reuseConnection() on the handle when finished using it.
281 * In all other cases, this is not necessary, though not harmful either.
282 *
283 * @note This method throws DBAccessError if ILoadBalancer::disable() was called
284 *
285 * @param int $i Server index (does not support DB_MASTER/DB_REPLICA)
286 * @param string|bool $domain Domain ID, or false for the current domain
287 * @param int $flags Bitfield of CONN_* class constants (e.g. CONN_TRX_AUTOCOMMIT)
288 * @return Database|bool Returns false on errors
289 * @throws DBAccessError
290 */
291 public function openConnection( $i, $domain = false, $flags = 0 );
292
293 /**
294 * @return int
295 */
296 public function getWriterIndex();
297
298 /**
299 * Returns true if the specified index is a valid server index
300 *
301 * @param int $i
302 * @return bool
303 */
304 public function haveIndex( $i );
305
306 /**
307 * Returns true if the specified index is valid and has non-zero load
308 *
309 * @param int $i
310 * @return bool
311 */
312 public function isNonZeroLoad( $i );
313
314 /**
315 * Get the number of defined servers (not the number of open connections)
316 *
317 * @return int
318 */
319 public function getServerCount();
320
321 /**
322 * Get the host name or IP address of the server with the specified index
323 *
324 * @param int $i
325 * @return string Readable name if available or IP/host otherwise
326 */
327 public function getServerName( $i );
328
329 /**
330 * Get DB type of the server with the specified index
331 *
332 * @param int $i
333 * @return string One of (mysql,postgres,sqlite,...) or "unknown" for bad indexes
334 * @since 1.30
335 */
336 public function getServerType( $i );
337
338 /**
339 * Return the server info structure for a given index, or false if the index is invalid.
340 * @param int $i
341 * @return array|bool
342 * @since 1.31
343 */
344 public function getServerInfo( $i );
345
346 /**
347 * @param int $i Server index
348 * @return array (Database::ATTRIBUTE_* constant => value) for all such constants
349 * @since 1.31
350 */
351 public function getServerAttributes( $i );
352
353 /**
354 * Get the current master position for chronology control purposes
355 * @return DBMasterPos|bool Returns false if not applicable
356 */
357 public function getMasterPos();
358
359 /**
360 * Disable this load balancer. All connections are closed, and any attempt to
361 * open a new connection will result in a DBAccessError.
362 */
363 public function disable();
364
365 /**
366 * Close all open connections
367 */
368 public function closeAll();
369
370 /**
371 * Close a connection
372 *
373 * Using this function makes sure the LoadBalancer knows the connection is closed.
374 * If you use $conn->close() directly, the load balancer won't update its state.
375 *
376 * @param IDatabase $conn
377 */
378 public function closeConnection( IDatabase $conn );
379
380 /**
381 * Commit transactions on all open connections
382 * @param string $fname Caller name
383 * @throws DBExpectedError
384 */
385 public function commitAll( $fname = __METHOD__ );
386
387 /**
388 * Perform all pre-commit callbacks that remain part of the atomic transactions
389 * and disable any post-commit callbacks until runMasterPostTrxCallbacks()
390 *
391 * Use this only for mutli-database commits
392 */
393 public function finalizeMasterChanges();
394
395 /**
396 * Perform all pre-commit checks for things like replication safety
397 *
398 * Use this only for mutli-database commits
399 *
400 * @param array $options Includes:
401 * - maxWriteDuration : max write query duration time in seconds
402 * @throws DBTransactionError
403 */
404 public function approveMasterChanges( array $options );
405
406 /**
407 * Flush any master transaction snapshots and set DBO_TRX (if DBO_DEFAULT is set)
408 *
409 * The DBO_TRX setting will be reverted to the default in each of these methods:
410 * - commitMasterChanges()
411 * - rollbackMasterChanges()
412 * - commitAll()
413 * This allows for custom transaction rounds from any outer transaction scope.
414 *
415 * @param string $fname
416 * @throws DBExpectedError
417 */
418 public function beginMasterChanges( $fname = __METHOD__ );
419
420 /**
421 * Issue COMMIT on all master connections where writes where done
422 * @param string $fname Caller name
423 * @throws DBExpectedError
424 */
425 public function commitMasterChanges( $fname = __METHOD__ );
426
427 /**
428 * Issue all pending post-COMMIT/ROLLBACK callbacks
429 *
430 * Use this only for mutli-database commits
431 *
432 * @param int $type IDatabase::TRIGGER_* constant
433 * @return Exception|null The first exception or null if there were none
434 */
435 public function runMasterPostTrxCallbacks( $type );
436
437 /**
438 * Issue ROLLBACK only on master, only if queries were done on connection
439 * @param string $fname Caller name
440 * @throws DBExpectedError
441 */
442 public function rollbackMasterChanges( $fname = __METHOD__ );
443
444 /**
445 * Suppress all pending post-COMMIT/ROLLBACK callbacks
446 *
447 * Use this only for mutli-database commits
448 *
449 * @return Exception|null The first exception or null if there were none
450 */
451 public function suppressTransactionEndCallbacks();
452
453 /**
454 * Commit all replica DB transactions so as to flush any REPEATABLE-READ or SSI snapshot
455 *
456 * @param string $fname Caller name
457 */
458 public function flushReplicaSnapshots( $fname = __METHOD__ );
459
460 /**
461 * @return bool Whether a master connection is already open
462 */
463 public function hasMasterConnection();
464
465 /**
466 * Determine if there are pending changes in a transaction by this thread
467 * @return bool
468 */
469 public function hasMasterChanges();
470
471 /**
472 * Get the timestamp of the latest write query done by this thread
473 * @return float|bool UNIX timestamp or false
474 */
475 public function lastMasterChangeTimestamp();
476
477 /**
478 * Check if this load balancer object had any recent or still
479 * pending writes issued against it by this PHP thread
480 *
481 * @param float $age How many seconds ago is "recent" [defaults to mWaitTimeout]
482 * @return bool
483 */
484 public function hasOrMadeRecentMasterChanges( $age = null );
485
486 /**
487 * Get the list of callers that have pending master changes
488 *
489 * @return string[] List of method names
490 */
491 public function pendingMasterChangeCallers();
492
493 /**
494 * @note This method will trigger a DB connection if not yet done
495 * @param string|bool $domain Domain ID, or false for the current domain
496 * @return bool Whether the database for generic connections this request is highly "lagged"
497 */
498 public function getLaggedReplicaMode( $domain = false );
499
500 /**
501 * Checks whether the database for generic connections this request was both:
502 * - a) Already choosen due to a prior connection attempt
503 * - b) Considered highly "lagged"
504 *
505 * @note This method will never cause a new DB connection
506 * @return bool
507 */
508 public function laggedReplicaUsed();
509
510 /**
511 * @note This method may trigger a DB connection if not yet done
512 * @param string|bool $domain Domain ID, or false for the current domain
513 * @param IDatabase|null $conn DB master connection; used to avoid loops [optional]
514 * @return string|bool Reason the master is read-only or false if it is not
515 */
516 public function getReadOnlyReason( $domain = false, IDatabase $conn = null );
517
518 /**
519 * Disables/enables lag checks
520 * @param null|bool $mode
521 * @return bool
522 */
523 public function allowLagged( $mode = null );
524
525 /**
526 * @return bool
527 */
528 public function pingAll();
529
530 /**
531 * Call a function with each open connection object
532 * @param callable $callback
533 * @param array $params
534 */
535 public function forEachOpenConnection( $callback, array $params = [] );
536
537 /**
538 * Call a function with each open connection object to a master
539 * @param callable $callback
540 * @param array $params
541 */
542 public function forEachOpenMasterConnection( $callback, array $params = [] );
543
544 /**
545 * Call a function with each open replica DB connection object
546 * @param callable $callback
547 * @param array $params
548 */
549 public function forEachOpenReplicaConnection( $callback, array $params = [] );
550
551 /**
552 * Get the hostname and lag time of the most-lagged replica DB
553 *
554 * This is useful for maintenance scripts that need to throttle their updates.
555 * May attempt to open connections to replica DBs on the default DB. If there is
556 * no lag, the maximum lag will be reported as -1.
557 *
558 * @param bool|string $domain Domain ID, or false for the default database
559 * @return array ( host, max lag, index of max lagged host )
560 */
561 public function getMaxLag( $domain = false );
562
563 /**
564 * Get an estimate of replication lag (in seconds) for each server
565 *
566 * Results are cached for a short time in memcached/process cache
567 *
568 * Values may be "false" if replication is too broken to estimate
569 *
570 * @param string|bool $domain
571 * @return int[] Map of (server index => float|int|bool)
572 */
573 public function getLagTimes( $domain = false );
574
575 /**
576 * Get the lag in seconds for a given connection, or zero if this load
577 * balancer does not have replication enabled.
578 *
579 * This should be used in preference to Database::getLag() in cases where
580 * replication may not be in use, since there is no way to determine if
581 * replication is in use at the connection level without running
582 * potentially restricted queries such as SHOW SLAVE STATUS. Using this
583 * function instead of Database::getLag() avoids a fatal error in this
584 * case on many installations.
585 *
586 * @param IDatabase $conn
587 * @return int|bool Returns false on error
588 */
589 public function safeGetLag( IDatabase $conn );
590
591 /**
592 * Wait for a replica DB to reach a specified master position
593 *
594 * This will connect to the master to get an accurate position if $pos is not given
595 *
596 * @param IDatabase $conn Replica DB
597 * @param DBMasterPos|bool $pos Master position; default: current position
598 * @param int $timeout Timeout in seconds [optional]
599 * @return bool Success
600 */
601 public function safeWaitForMasterPos( IDatabase $conn, $pos = false, $timeout = 10 );
602
603 /**
604 * Set a callback via IDatabase::setTransactionListener() on
605 * all current and future master connections of this load balancer
606 *
607 * @param string $name Callback name
608 * @param callable|null $callback
609 */
610 public function setTransactionListener( $name, callable $callback = null );
611
612 /**
613 * Set a new table prefix for the existing local domain ID for testing
614 *
615 * @param string $prefix
616 */
617 public function setDomainPrefix( $prefix );
618
619 /**
620 * Make certain table names use their own database, schema, and table prefix
621 * when passed into SQL queries pre-escaped and without a qualified database name
622 *
623 * For example, "user" can be converted to "myschema.mydbname.user" for convenience.
624 * Appearances like `user`, somedb.user, somedb.someschema.user will used literally.
625 *
626 * Calling this twice will completely clear any old table aliases. Also, note that
627 * callers are responsible for making sure the schemas and databases actually exist.
628 *
629 * @param array[] $aliases Map of (table => (dbname, schema, prefix) map)
630 */
631 public function setTableAliases( array $aliases );
632
633 /**
634 * Convert certain index names to alternative names before querying the DB
635 *
636 * Note that this applies to indexes regardless of the table they belong to.
637 *
638 * This can be employed when an index was renamed X => Y in code, but the new Y-named
639 * indexes were not yet built on all DBs. After all the Y-named ones are added by the DBA,
640 * the aliases can be removed, and then the old X-named indexes dropped.
641 *
642 * @param string[] $aliases
643 * @return mixed
644 * @since 1.31
645 */
646 public function setIndexAliases( array $aliases );
647 }