lp:maria

Author: Sergei Golubchik
Author Date: 2023-11-13 22:38:47 UTC

MDEV-22243 type_test.type_test_double fails with 'NUMERIC_SCALE NULL'

1. fix useless correction of dec_arg *after it was used*
2. use DECIMAL_NOT_SPECIFIED instead of NOT_FIXED_DEC, because the latter
   is defined conditionally and has a wrong value in plugins

Author: Marko Mäkelä
Author Date: 2023-11-16 15:45:37 UTC

MDEV-26055: Correct the formula for adaptive flushing

This is a 10.5 backport of 10.6
commit d4265fbde587bd79b6fe3793225d3f4798ee955e.

page_cleaner_flush_pages_recommendation(): If dirty_pct is
between innodb_max_dirty_pages_pct_lwm
and innodb_max_dirty_pages_pct,
scale the effort relative to how close we are to
innodb_max_dirty_pages_pct.

The previous formula was missing a multiplication by 100.

Author: Marko Mäkelä
Author Date: 2023-11-16 15:01:28 UTC

MDEV-32820 Race condition between trx_purge_free_segment() and trx_undo_create()

trx_purge_free_segment(): If fseg_free_step_not_header() needs to be
called multiple times, acquire an exclusive latch on the
rollback segment header page after restarting the mini-transaction
so that the rest of this function cannot execute concurrently
with trx_undo_create() on the same rollback segment.

This fixes a regression that was introduced in
commit c14a39431b211017e6809bb79c4079b38ffc3dff (MDEV-30753).

Note: The buffer-fixes that we are holding across the mini-transaction
restart will prevent the pages from being evicted from the buffer pool.
They may be accessed by other threads or written back to data files
while we are not holding exclusive latches.

Author: Marko Mäkelä
Author Date: 2023-11-16 14:57:42 UTC

MDEV-31861 Empty INSERT crashes with innodb_force_recovery=6 or innodb_read_only=ON

ha_innobase::extra(): Do not invoke log_buffer_flush_to_disk()
if high_level_read_only holds.

log_buffer_flush_to_disk(): Remove an assertion that duplicates one
at the start of log_write_up_to().

Author: Marko Mäkelä
Author Date: 2023-11-16 14:57:42 UTC

MDEV-31861 Empty INSERT crashes with innodb_force_recovery=6 or innodb_read_only=ON

ha_innobase::extra(): Do not invoke log_buffer_flush_to_disk()
if high_level_read_only holds.

log_buffer_flush_to_disk(): Remove an assertion that duplicates one
at the start of log_write_up_to().

Author: Marko Mäkelä
Author Date: 2023-11-16 14:27:57 UTC

MDEV-32820 Race condition between trx_purge_free_segment() and trx_undo_create()

trx_purge_free_segment(): If fseg_free_step_not_header() needs to be
called multiple times, acquire an exclusive latch on the
rollback segment header page after restarting the mini-transaction
so that the rest of this function cannot execute concurrently
with trx_undo_create() on the same rollback segment.

This fixes a regression that was introduced in
commit c14a39431b211017e6809bb79c4079b38ffc3dff (MDEV-30753).

Note: The buffer-fixes that we are holding across the mini-transaction
restart will prevent the pages from being evicted from the buffer pool.
They may be accessed by other threads or written back to data files
while we are not holding exclusive latches.

Author: Alexander Barkov
Author Date: 2023-11-02 10:16:09 UTC

MDEV-25829 Change default collation to utf8mb4_1400_ai_ci

Step#3 The main patch

Author: Nikita Malyavin
Author Date: 2023-11-16 09:50:38 UTC

fix uninitialized field warnings

Author: Kristian Nielsen
Author Date: 2023-11-16 11:46:30 UTC

MDEV-32819: main.show_explain failed in buildbot

The testcase had a race in two places where a KILL QUERY is made towards a
running query in another connection. The query can complete early so the kill
is lost, and the test fails due to expecting ER_QUERY_INTERRUPTED.

Fix by removing the KILL QUERY. It is not needed, as the query completes by
itself after SHOW EXPLAIN FOR.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Oleg Smirnov
Author Date: 2023-11-16 09:54:16 UTC

MDEV-24486 Add `table_privileges` view to the `sys` schema

The existing INFORMATION_SCHEMA.TABLE_PRIVILEGES displays only those
privileges that were specifically granted on the table level,
whereas it may be useful to see privileges granted at the database
and global level.

This commit adds a new view `table_privileges` to the `sys` schema
for that purpose. The view shows privileges to existing tables
and views, combining all possible levels:
 - user_privileges
 - schema_privileges
 - table_privileges

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-16 11:09:14 UTC

Merge branch '11.2' into 11.3

Author: Nikita Malyavin
Author Date: 2023-11-16 09:50:38 UTC

fix uninitialized field warnings

Author: Anel Husakovic
Author Date: 2023-09-18 15:18:01 UTC

MDEV-32168: slave_error_param condition is never checked from the wait_for_slave_param.inc

- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
            <andrei.elkin@mariadb.com>

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-16 09:11:25 UTC

MDEV-32811 Potentially broken crash recovery if a mini-transaction frees a page, not modifying previously clean pages

- The 11.2 test innodb.sys_truncate_debug fails while executing insert statement.
Reason for the failure is that same mini-transaction does freeing, allocating
and freeing the same page. Page initialization clears the FIL_PAGE_LSN
on the page, fails to set the FIL_PAGE_LSN after freeing the same page.
This issue is caused by commit f46efb4476d4cb7401ea73e4c452d9460bdd0f33

mtr_t::commit(): Should set the FIL_PAGE_LSN even though page is freed

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-16 09:11:25 UTC

MDEV-32811 Potentially broken crash recovery if a mini-transaction frees a page, not modifying previously clean pages

- The 11.2 test innodb.sys_truncate_debug fails while executing insert statement.
Reason for the failure is that same mini-transaction does freeing, allocating
and freeing the same page. Page initialization clears the FIL_PAGE_LSN
on the page, fails to set the FIL_PAGE_LSN after freeing the same page.
This issue is caused by commit f46efb4476d4cb7401ea73e4c452d9460bdd0f33

mtr_t::commit(): Should set the FIL_PAGE_LSN even though page is freed

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2023-11-14 07:24:46 UTC

MDEV-31273: Precompute binlog checksums

After b8f9f796 Format_description_log_event constructor
requires binlog checksum algorithm or BINLOG_CHECKSUM_ALG_OFF
for Galera also.

Thanks to Kristian Nielsen <knielsen@knielsen-hq.org>
for pointing a fix.

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2023-11-14 07:24:46 UTC

MDEV-31273: Precompute binlog checksums

After b8f9f796 Format_description_log_event constructor
requires binlog checksum algorithm or BINLOG_CHECKSUM_ALG_OFF
for Galera also.

Thanks to Kristian Nielsen <knielsen@knielsen-hq.org>
for pointing a fix.

Author: Rex Johnston
Author Date: 2023-11-13 21:12:25 UTC

MDEV-31793 LooseScan join on derived table crashes

A loose scan join on a derived table may cause a zero length key to be
calculated, causing a call to allocate zero bytes of memory.
This can cause issues when later freeing up this memory.

Author: Yuchen Pei
Author Date: 2023-11-15 01:38:43 UTC

MDEV-32804 Remove references to spider_rewrite_plugin tables

They are not used. The code paths are never reached.

Author: Yuchen Pei
Author Date: 2023-11-15 01:38:43 UTC

MDEV-32804 Remove references to spider_rewrite_plugin tables

They are not used. The code paths are never reached.

Author: Yuchen Pei
Author Date: 2023-11-15 01:38:43 UTC

MDEV-32804 Remove references to spider_rewrite_plugin tables

They are not used. The code paths are never reached.

Author: Brandon Nesterenko
Author Date: 2023-07-10 15:53:19 UTC

MDEV-7850 make Thread-id available in Gtid

WIP, preliminary merge from bb-10.5-andrei to 11.4

This patch augements Gtid_log_event with the user thread-id.
In particular that compensates for the loss of this info in
Rows_log_events.

Gtid_log_event::thread_id gets visible in mysqlbinlog output like

  #231025 16:21:45 server id 1 end_log_pos 537 CRC32 0x1cf1d963 GTID 0-1-2 ddl thread_id=10

as 64 bit unsigned integer.

While the size of Gtid event has grown by 8-9 bytes
replication from OLD <-> NEW is not affected by it.

This work was started by the late Sujatha Sivakumar.
Brandon Nesterenko took it over, reviewed initial patches and extended
the work.

Reviewed-by: <andrei.elkin@mariadb.com>

Author: Igor Babaev
Author Date: 2023-11-15 20:56:54 UTC

MDEV-30073 Wrong result on 2nd execution of PS for query with NOT EXISTS

This is a preliminary patch to check possible failures in buildbot.
Variant 2.

Author: Rex Johnston
Author Date: 2023-09-25 01:56:30 UTC

Merge 10.4 into 10.5

Author: Rex Johnston
Author Date: 2023-09-25 01:56:30 UTC

Merge 10.4 into 10.5

Author: midenok
Author Date: 2023-11-15 16:28:07 UTC

MDEV-22164 without validation for exchange partition/convert in

1. WITHOUT/WITH VALIDATION may be added to EXCHANGE PARTITION or CONVERT TABLE:

  alter table tp exchange partition p1 with table t with validation;
  alter table tp exchange partition p1 with table t; -- same as with validation
  alter table tp exchange partition p1 with table t without validation;

2. Optional THAN keyword for RANGE partitioning. Normally you type:

  create table tp (a int primary key) partition by range (a) (
    partition p0 values less than (100),
    partition p1 values less than maxvalue);

  Now you may type (PARTITION keyword is also optional):

  create table tp (a int primary key) partition by range (a) (
    p0 values less (100),
    p1 values less maxvalue);

Author: Andrei
Author Date: 2023-11-15 18:13:56 UTC

Completed the cleanup of refactoring of xa binlogging:

- binlog_rollback() 's empty branch is made compatible to binlog_commit's;
- a read-only xa prepare MDEV-32813 analyzed (not to fix right away
  'cos of few alternatives exist how to)

Author: midenok
Author Date: 2023-11-15 16:28:07 UTC

MDEV-22164 without validation for exchange partition/convert in

1. WITHOUT/WITH VALIDATION may be added to EXCHANGE PARTITION or CONVERT TABLE:

  alter table tp exchange partition p1 with table t with validation;
  alter table tp exchange partition p1 with table t; -- same as with validation
  alter table tp exchange partition p1 with table t without validation;

2. Optional THAN keyword for RANGE partitioning. Normally you type:

  create table tp (a int primary key) partition by range (a) (
    partition p0 values less than (100),
    partition p1 values less than maxvalue);

  Now you may type (PARTITION keyword is also optional):

  create table tp (a int primary key) partition by range (a) (
    p0 values less (100),
    p1 values less maxvalue);

Author: Brandon Nesterenko
Author Date: 2023-11-15 16:16:54 UTC

MDEV-32628: Cryptic ERROR message & inconsistent behavior on incorrect SHOW BINLOG EVENTS FROM ...

Calling SHOW BINLOG EVENTS FROM <offset> with an invalid offset
writes error messages into the server log about invalid reads. The
read errors that occur from this command should only be relayed back
to the user though, and not written into the server log. This is
because they are read-only and have no impact on server operation,
and the client only need be informed to correct the parameter.

This patch fixes this by omitting binary log read errors from the
server when the invocation happens from SHOW BINLOG EVENTS.
Additionally, redundant error messages are omitted when calling the
string based read_log_event from the IO_Cache based read_log_event,
as the later already will report the error of the former.

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Lena Startseva
Author Date: 2023-11-14 12:03:16 UTC

MDEV-32177: Exclude double sorting of results for ps2-protocol

Author: Lena Startseva
Author Date: 2023-11-15 15:22:00 UTC

Added DEBUG_ASSERT

Author: Kristian Nielsen
Author Date: 2023-11-15 15:13:46 UTC

MDEV-30064: binlog.binlog_mysqlbinlog_raw_flush sometimes fails with Errcode: 2 "No such file or directory"

Increase a 1-second timeout, which is a bit too small for slow buildbot
builders.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Dmitry Shulga
Author Date: 2023-11-15 14:04:28 UTC

MDEV-32466: Potential memory leak on execuing of create view statement

This patch is actually follow-up for the task
  MDEV-23902: MariaDB crash on calling function
to use correct query arena for a statement. In case a function
invocation is in progress use its call arena, else use current
query arena that can be either a statement or a regular query arena.

Author: Sergei Golubchik
Author Date: 2023-11-15 13:17:26 UTC

MDEV-32810 events.events_1 fails in 11.1 and above

fix test cases like

... prepare the test, create a table or a user
create event ... on schedule every 1 second ... do something ...
... verify that something was done
... cleanup drop event, table or user

here the event scheduler can manage to start the worker for the
second execution of the event, then cleanup happens
and the worker realizes that it cannot change the security context
(as the user is dropped) or cannot insert (as the table is dropped).

followup for 279b0db8c60 (that removed suppression of /Event Schedule/)

Author: Sergei Golubchik
Author Date: 2023-11-04 09:11:51 UTC

don't disable lto in DEB builds

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2023-11-06 05:37:34 UTC

MDEV-32634 : wsrep_provider_options can be truncated on deep and long directory paths

This is caused by fact that VARIABLE_VALUE is defined in
variables_fields_info (sql_show.cc) as 2048.
wsrep_provider_options contain few path variables and
this could cause string truncation on deep and long
directory paths.

Author: Denis Protivensky
Author Date: 2023-10-27 09:26:08 UTC

MDEV-22232: Fix CTAS replay & retry in case it gets BF-aborted

- Add selected tables as shared keys for CTAS certification
- Set proper security context on the replayer thread
- Disallow CTAS command retry

Author: Kristian Nielsen
Author Date: 2023-09-08 11:12:49 UTC

MDEV-4991: GTID binlog indexing

Improve the performance of slave connect using B+-Tree indexes on each binlog
file. The index allows fast lookup of a GTID position to the corresponding
offset in the binlog file, as well as lookup of a position to find the
corresponding GTID position.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Yuchen Pei
Author Date: 2023-11-15 07:19:53 UTC

MDEV-27254 Delete spider_crd_type and spider_crd_weight

Author: Yuchen Pei
Author Date: 2023-11-15 03:27:56 UTC

MDEV-29020 Reduce default spider bg sts/crd thread counts

Author: Yuchen Pei
Author Date: 2023-11-14 06:32:56 UTC

MDEV-32753 Make spider init queries compatible with oracle sql mode

Remove ORACLE from the (session) sql_mode in connections made with sql
service to run init queries

The connection is new and the global variable value takes effect
rather than the session value from the caller of spider_db_init.

Author: Rex Johnston
Author Date: 2023-11-06 03:04:30 UTC

MDEV-32212 DELETE with ORDER BY and semijoin optimization causing crash

Classes for multitable update aren't currently able to take into
account any filesort structure in read_record->sort_info and assume
that one hasn't been built.

Semijoin optimization on an innocent looking query, such as

DELETE FROM t1 WHERE c1 IN (select c2 from t2) ORDER BY c1;

can cause a filesort structure to be built while forcing use of the
multitable delete classes, previously used only in the context of
multiple-table delete syntax which does not allow an order by clause.

This filesort structure causes the read_record classes in
init_read_record to be one of the rr_unpack_from_buffer templated
routines, none of which track the table->file cursor assumed to be
positioned correctly in multi_delete::send_data.
This triggers an ASSERT in InnoDB and may cause problems with other
storage engines.

We could either disable this semijoin optimization or completely
ignore the order by clause where there is no subsequent LIMIT
clause. Fortunately, this optimization is already disabled where
there is a limit clause by
Sql_cmd_delete::processing_as_multitable_delete_prohibited()

So, this variant of the fix, rather than disabling semijoin
optimization, waits until the order list elements have been
resolved (there is a test for this) in JOIN::prepare(),
then simply removes the order by element from the join.

Author: Michael Widenius
Author Date: 2023-11-08 23:57:58 UTC

MDEV-32551: "Read semi-sync reply magic number error" warnings on master

The test and the related commit message comes from Brandon Nesterenko

A test to show how to induce the ‘Read semi-sync reply magic number
error’ message on a primary. In short, if semi-sync is turned on
during the hand-shake process between a primary and replica, but
later a user negates the rpl_semi_sync_slave_enabled variable while
the replica’s IO thread is running; if the io thread exits, the
replica can skip a necessary call to kill_connection() in
repl_semisync_slave.slave_stop() due to its reliance on a global
variable. Then, the replica will send a COM_QUIT packet to the
primary on an active semi-sync connection, causing the magic number
error.

The test in this patch exits the IO thread by forcing an error;
though note a call to STOP SLAVE could also do this, but it ends up
needing more synchronization. That is, the STOP SLAVE command also
tries to kill the vio of the replica, which makes a race with the IO
thread to try and send the COM_QUIT before this happens (which would
need more debug_sync to get around). See THD::awake_no_mutex for
details as to the killing of the replica’s vio.

Changes done:
- Some timings changed slight with startup of slave which caused
  rpl_binlog_dump_slave_gtid_state_info.text to fail as it checked the
  error log file before the slave had started properly. Fixed by adding
  wait_for_pattern_in_file.inc that allows waiting for the pattern to
  appear in the log file.
- The rpl_semi_sync_slave_enabled variable is now a default value for
  when semisync is started. The variable does not anymore affect
  semisync if it is already running. This fixes the original reported bug.
  Internally we now use repl_semisync_slave.get_slave_enabled() instead of
  rpl_semi_sync_slave_enabled. To check if semisync is active on should
  check the @@rpl_semi_sync_slave_status variable (as before).
- Changed the default value for rpl-semi-sync-master-wait-no-slave to 0 as
  the old one did not make much sense as default. The main difference is
  that now we do not wait for semisync ack if there are no slave threads.
  In the old code, we waited once, which did not bring any notable benefits
  except slower startup of master configured for using semisync.
- Removed some duplicate semisync notes from the error log.
- We do not call semisync.switch_off() if there are no active slaves.
  Instead we check in Repl_semi_sync_master::commit_trx() if there are
  no active threads. This simplices a lot of code.
- Changed assert() to DBUG_ASSERT() to ensure that the DBUG log is flushed
  in case of asserts.
- Removed Repl_semi_sync_master::check_and_switch() as it is not needed
  anymore.
- Remove rpl_semi_sync_slave_status as it is not needed anymore.
- Removed rpl_semi_sync_slave_enabled as it is not needed anymore.
  Repl_semi_sync_slave::get_slave_enabled() contains the active status.
- Added checking that we do not add a slave twice with
  Ack_receiver::add_slave(). This could happen with old code.
- Ensure that when we call Ack_receiver::remove_slave() that the slave
  is removed from the listener before function returns.
- Call listener.listen_on_sockets() outside of mutex for better performance
  and less contested mutex.
- Ensure that listening is ignoring newly added slaves when checking for
  responses.
- kill_zombie_dump_threads() now does killing of dump threads properly.
- Added an error if kill_zombie_dump_threads() fails.

Co-author: Brandon Nesterenko <brandon.nesterenko@mariadb.com>

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 17:29:53 UTC

Merge branch 'nn-11.2-merge' into bb-11.3-release

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 17:22:29 UTC

Merge branch '11.2' into bb-11.2-release

Author: midenok
Author Date: 2023-11-14 10:39:48 UTC

MDEV-29872 MSAN/Valgrind uninitialised value errors in TABLE::vers_switch_partition

Delayed_insert has its own THD (initialized at mysql_insert()) and
hence its own LEX. Delayed_insert initalizes a very few parameters for
LEX and 'duplicates' is not in this list. Now we copy this missing
parameter from parser LEX (as well as sql_command).

Author: Nikita Malyavin
Author Date: 2023-11-14 10:20:49 UTC

main.alter_table_online_debug: fix race in XA tests

Author: Kristian Nielsen
Author Date: 2023-11-14 14:00:41 UTC

MDEV-32673 Handle InnoDB binlog position recovery for RESET MASTER or binlog name change

Another attempt, to do most of what Marko's solution does without
requiring a new handlerton call: Compare each binlog position against
the previous one, and reset all old entries if the position moves
backwards.

Does not work though, it asserts in multiple places about pages not
marked as modified (IIUC) during the reset of commit processing. Also,
even if the mark-as-modified could be added, I'n not sure what the
consequences will be of trying to modify all rseg headers in the final
mtr during commit-to-memory. How would the locking work, if multiple
threads would try to do this simulaneosly? (though that might not be
possible due to LOCK_commit_ordered). Or could there be other locking
problems?

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-14 13:52:56 UTC

MDEV-32050 Fixup

- Fixing mariabackup.full_backup test case

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-14 13:52:56 UTC

MDEV-32050 Fixup

- Fixing mariabackup.full_backup test case

Author: Marko Mäkelä
Author Date: 2023-11-14 13:51:23 UTC

MDEV-32050 test fixup

innodb.instant_alter_crash: Try to fix an occasional result difference.

Author: Anel Husakovic
Author Date: 2023-10-05 14:06:27 UTC

MDEV-32168: 10.10

Author: Anel Husakovic
Author Date: 2023-10-05 13:57:03 UTC

MDEV-32168: 10.9 reviewer

Author: Anel Husakovic
Author Date: 2023-11-13 10:00:22 UTC

MDEV-32168: 10.6 reviewer 1

Author: Anel Husakovic
Author Date: 2023-10-05 13:26:01 UTC

MDEV-32168: 10.5 reviewr

Author: Marko Mäkelä
Author Date: 2023-11-14 12:03:13 UTC

Merge 10.6

Author: Dmitry Shulga
Author Date: 2023-11-10 10:46:19 UTC

MDEV-32733: Two JSON related tests running in PS mode fail on server built with -DWITH_PROTECT_STATEMENT_MEMROOT=YES

The tests main.func_json and json.json_no_table fail on server built with
the option -DWITH_PROTECT_STATEMENT_MEMROOT=YES by the reason that a memory
is allocated on the statement's memory root on the second execution of
a query that uses the function json_contains_path().

The reason that a memory is allocated on second execution of a prepared
statement that ivokes the function json_contains_path() is that a memory
allocated on every call of the method Item_json_str_multipath::fix_fields

To fix the issue, memory allocation should be done only once on first
call of the method Item_json_str_multipath::fix_fields. Simmilar issue
take place inside the method Item_func_json_contains_path::fix_fields.
Both methods are modified to make memory allocation only once on its
first execution and later re-use the allocated memory.

Before this patch the memory referenced by the pointers stored in the array
tmp_paths were released by the method Item_func_json_contains_path::cleanup
that is called on finishing execution of a prepared statement. Now that
memory allocation performed inside the method Item_json_str_multipath::fix_fields
is done only once, the item clean up has degenerate form and can be
delegated to the cleanup() method of the base class and memory deallocation
can be performed in the destructor.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 09:14:07 UTC

Merge branch '11.1' into mariadb-11.1.3

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 09:14:07 UTC

Merge branch '11.1' into mariadb-11.1.3

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-14 08:54:58 UTC

MDEV-32242 innodb.doublewrite test case always gets skipped

- Rewrote the doublewrite test case and removes the usage of
innodb_buf_flush_list variable, added search pattern to make
sure that InnoDB uses doublewrite buffer. Replaced all
kill_mysqld.inc with shutdown_mysqld.inc and zero shutdown timeout

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 08:21:35 UTC

Merge branch '11.0' into mariadb-11.0.4

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 08:18:56 UTC

Merge branch '10.11' into mariadb-10.11.6

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 07:56:16 UTC

Merge branch '10.6' into mariadb-10.6.16

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 07:41:23 UTC

Merge branch '10.5' into mariadb-10.5.23

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-14 07:35:10 UTC

MDEV-32242 innodb.doublewrite test case always gets skipped

- Rewrote the doublewrite test case and removes the usage of
innodb_buf_flush_list variable, added search pattern to make
sure that InnoDB uses doublewrite buffer. Replaced all
kill_mysqld.inc with shutdown_mysqld.inc and zero shutdown timeout

Author: Oleksandr "Sanja" Byelkin
Author Date: 2023-11-14 07:14:44 UTC

Merge branch '10.4' into mariadb-10.4.32

Author: Christian Hesse
Author Date: 2023-08-09 00:18:36 UTC

MDEV-19210: do not run pre and post scripts as root

Now that we do not pollute systemd's environment but write private
environment files running these as root is not longer required. So
let's drop `PermissionsStartOnly=true`.

Debian adds extra `ExecStartPre=` and `ExecStartPost=`, though.
Use special executable prefix for full privileges there. (See
systemd.service(5) for details.)

Author: Alexander Barkov
Author Date: 2023-11-14 02:34:28 UTC

An attempt to fix type_test_double failure on some platforms

Author: Yuchen Pei
Author Date: 2023-11-14 01:48:52 UTC

MDEV-27576 Use reverse index for max/min optimization

We use a bool to indicate that the key part used is a descending
index, which will flip the functions and flags used in
get_index_max_value() and get_index_min_value(), that allows correct
optimization for max/min for descending index.

Author: Yuchen Pei
Author Date: 2023-11-14 01:13:04 UTC

MDEV-32753 Make spider init queries compatible with oracle sql mode

Remove ORACLE from the (session) sql_mode in connections made with sql
service to run init queries

The connection is new and the global variable value takes effect
rather than the session value from the caller of spider_db_init.

Author: Daniel Bartholomew
Author Date: 2023-11-13 22:42:11 UTC

Merge branch 'bb-11.0-bumpversion' of github.com:MariaDB/server into bb-11.0-bumpversion

Author: Daniel Bartholomew
Author Date: 2023-11-13 19:41:25 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2023-11-13 19:39:41 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2023-11-13 18:22:44 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2023-11-13 18:21:35 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2023-11-13 18:19:39 UTC

bump the VERSION

Author: Dmitry Shulga
Author Date: 2023-11-13 12:18:51 UTC

MDEV-32788: Building of server fails on MacOS on compilation of trx0purge.cc

The method srw_lock_impl of the template class srw_lock_impl
is conditionally defined under #ifndef SUX_LOCK_GENERIC but
called unconditionally that could result in compilation failure

Author: Alexey Botchkov
Author Date: 2023-11-13 11:58:17 UTC

MDEV-27666 User variable not parsed as geometry variable in geometry function.

Author: Alexey Botchkov
Author Date: 2023-11-13 10:48:02 UTC

MDEV-32454 JSON test has problem in view protocol.

Author: Oleg Smirnov
Author Date: 2023-11-10 13:03:06 UTC

MDEV-15656 Assertion `is_last_prefix <= 0' failed in QUICK_GROUP_MIN_MAX_SELECT::get_next

When QUICK_GROUP_MIN_MAX_SELECT is initialized or being reset
it stores the prefix of the last group of the index chosen for
retrieving data (last_value). Later, when looping through records
at get_next() method, the server checks whether the retrieved
group is the last, and if so, it finishes processing.

At the same time, it looks like there is no need for that additional
check since method next_prefix() returns HA_ERR_KEY_NOT_FOUND
or HA_ERR_END_OF_FILE when there are no more satisfying records.
If we do not perform the check, we do not need to retrieve and
store last_value either.

This commit removes using of last_value from QUICK_GROUP_MIN_MAX_SELECT

Author: Julius Goryavsky
Author Date: 2023-11-11 14:37:42 UTC

galera: cleanup of the lists of disabled tests

Author: Alexander Barkov
Author Date: 2023-11-13 07:18:16 UTC

MDEV-32645 CAST(AS UNSIGNED) fails with --view-protocol

Item_float::neg() did not preserve the "presentation" from "this".
So
  CAST(-1e0 AS UNSIGNED) -- cast from double to unsigned
changes its meaning to:
  CAST(-1 AS UNSIGNED) -- cast signed to undigned

Fixing Item_float::neg() to construct the new value for
Item_float::presentation as minus sign followed by its old value:
  '1e0' -> '-1e0'

Author: Rucha Deodhar
Author Date: 2023-11-10 10:31:15 UTC

MDEV-32726: Fix failing test fir freebsd for json

Json test about max statement time fails with freebsd because on some
architectures the test might execute faster and the statement may not fail.

To simulate failure regardless of architecture, introduce a wait of seconds
longer than the max_statement_time.

Author: midenok
Author Date: 2023-11-09 13:26:11 UTC

MDEV-29932 Invalid expr in cleanup_session_expr() upon INSERT DELAYED

There are two TABLE objects in each thread: first one is created in
delayed thread by Delayed_insert::open_and_lock_table(), second one is
created in connection thread by Delayed_insert::get_local_table(). It
is copied from the delayed thread table.

When the second table is copied copy-assignment operator copies
vcol_refix_list which is already filled with an item from delayed
thread. Then get_local_table() adds its own item. Thus both tables
contains the same list with two items which is wrong. Then connection
thread finishes and its item freed. Then delayed thread tries to
access it in vcol_cleanup_expr().

The fix just clears vcol_refix_list in the copied table.

Another problem is that copied table contains the same mem_root, any
allocations on it will be invalid if the original table is freed (and
that is indeterministic as it is done in another thread). Since copied
table is allocated in connection THD and lives not longer than
thd->mem_root we may assign its mem_root from thd->mem_root.

Third, it doesn't make sense to do open_and_lock_tables() on NULL
pointer.

Author: Alexander Barkov
Author Date: 2023-09-04 11:28:50 UTC

MDEV-32101 CREATE PACKAGE [BODY] for sql_mode=DEFAULT

This patch adds PACKAGE support with SQL/PSM dialect for sql_mode=DEFAULT:

- CREATE PACKAGE
- DROP PACKAGE
- CREATE PACKAGE BODY
- DROP PACKAGE BODY
- Package function and procedure invocation from outside of the package:
    -- using two step identifiers
    SELECT pkg.f1();
    CALL pkg.p1()

    -- using three step identifiers
    SELECT db.pkg.f1();
    CALL db.pkg.p1();

This is a non-standard MariaDB extension.

However, later this code can be used to implement
the SQL Standard and DB2 dialects of CREATE MODULE.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-10 10:55:15 UTC

MDEV-28613 LeakSanitizer caused by I_S query using LIMIT ROWS EXAMINED

Problem:
========
- InnoDB fails to free the allocated buffer of stored cursor
when information schema query is interrupted.

Solution:
=========
- In case of error handling, information schema query should free
the allocated buffer to store the cursor.

Author: Alexander Barkov
Author Date: 2023-09-04 11:28:50 UTC

MDEV-32101 CREATE PACKAGE [BODY] for sql_mode=DEFAULT

This patch adds PACKAGE support with SQL/PSM dialect for sql_mode=DEFAULT:

- CREATE PACKAGE
- DROP PACKAGE
- CREATE PACKAGE BODY
- DROP PACKAGE BODY
- Package function and procedure invocation from outside of the package:
    -- using two step identifiers
    SELECT pkg.f1();
    CALL pkg.p1()

    -- using three step identifiers
    SELECT db.pkg.f1();
    CALL db.pkg.p1();

This is a non-standard MariaDB extension.

However, later this code can be used to implement
the SQL Standard and DB2 dialects of CREATE MODULE.

Author: Alexander Barkov
Author Date: 2023-09-25 17:48:01 UTC

MDEV-12252 ROW data type for stored function return values

Adding support for the ROW data type in the stored function RETURNS clause:

- explicit ROW(..members...) for both sql_mode=DEFAULT and sql_mode=ORACLE

  CREATE FUNCTION f1() RETURNS ROW(a INT, b VARCHAR(32)) ...

- anchored "ROW TYPE OF [db1.]table1" declarations for sql_mode=DEFAULT

  CREATE FUNCTION f1() RETURNS ROW TYPE OF test.t1 ...

- anchored "[db1.]table1%ROWTYPE" declarations for sql_mode=ORACLE

  CREATE FUNCTION f1() RETURN test.t1%ROWTYPE ...

Adding support for anchored scalar data types in RETURNS clause:

- "TYPE OF [db1.]table1.column1" for sql_mode=DEFAULT

  CREATE FUNCTION f1() RETURNS TYPE OF test.t1.column1;

- "[db1.]table1.column1" for sql_mode=ORACLE

  CREATE FUNCTION f1() RETURN test.t1.column1%TYPE;

Details:

- Adding a new sql_mode_t parameter to
    sp_head::create()
    sp_head::sp_head()
    sp_package::create()
    sp_package::sp_package()
  to guarantee early initialization of sp_head::m_sql_mode.
  Before this change, this member was not initialized at all during
  CREATE FUNCTION/PROCEDURE/PACKAGE statements, and was not used.
  Now it needs to be initialized to write properly the
  mysql.proc.returns column, according to the create time sql_mode.

- Moving the code from the loop body in sp_rcontext::init_var_items()
  into a separate member Spvar_definition::make_item_field_row(),
  to reuse this code for variable/parameter definitions
  and for the RETURNS clause.

- Changing the data type of sp_head::m_return_field_def
  from Column_definition to Spvar_definition.
  So now it supports not only SQL column field types,
  but also explicit ROW and anchored ROW data types.

- Modifying sql_yacc.yy to support the new grammar.
  Adding new helper methods:
    * sf_return_fill_definition_row()
    * sf_return_fill_definition_rowtype_of()
    * sf_return_fill_definition_type_of()

- Fixing tests in:
  * Virtual_tmp_table::setup_field_pointers() in sql_select.cc
  * Send_field::normalize() in field.h
  to prevent calling Type_handler_row::field_type(),
  which is implemented a DBUG_ASSERT(0).
  Before this patch the affected methods were called only
  for scalar data types. Now ROW is also possible.

- Adding a new virtual method Field::cols()

- Overriding methods:
   Item_func_sp::cols()
   Item_func_sp::element_index()
   Item_func_sp::check_cols()
   Item_func_sp::bring_value()
  to support the ROW data type.

- Extending sp_returns_type() to support
  * explicit ROW and anchored ROW data types
  * anchored scalar data types

- Overriding Field_row::sql_type() to print
  the data type of an explicit ROW.

Author: Alexander Barkov
Author Date: 2023-11-10 08:45:55 UTC

Cleanup: Adding "const" to the member Spvar_definition::m_column_type_ref

and to related methods and their parameters:
- The return value of Spvar_definition::m_column_type_ref()
- The parameter of Spvar_definition::set_column_type_ref()
- The method Qualified_column_ident::resolve_type_ref()
- The parameter of LEX::sp_variable_declarations_column_type_finalize()

Author: Yuchen Pei
Author Date: 2023-11-10 05:40:45 UTC

[demo] mysql.spider_table_sts is empty, but it returns something when read

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-09 07:51:09 UTC

MDEV-32452 InnoDB system tablespace is not shrunk on slow shutdown

- Truncate the system tablespace during slow shutdown process.

Author: Dave Gosselin
Author Date: 2023-11-06 21:43:23 UTC

MDEV-28509 Dereferenced null pointer of type 'struct JOIN_TAB' in add_key_field

Do not attempt to merge keys to a non-existent join table; verify
that the REGINFO's join_tab field is not NULL.

The newly added tests will crash without this fix, but with this fix
they produce same results as when run on MySQL 8.2.

Author: Kristian Nielsen
Author Date: 2023-11-09 13:54:44 UTC

MDEV-4991: Few fixes post-rebase on 11.3

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Kristian Nielsen
Author Date: 2023-11-09 13:20:03 UTC

MDEV-4991: Minor self-review fixes

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Andrei
Author Date: 2023-11-09 12:36:46 UTC

MDEV-32365: post-fixes to rpl_semi_sync_slave_reply_fail

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2023-11-09 10:25:07 UTC

MDEV-29913 Assertion `thd->stmt_arena != thd->progress.arena' failed in thd_progress_init upon bulk load

Load data statement triggers bulk insert operation and leads to
abort of server. Initialize the progress report for the thread
already done during load statement and bulk insert operation
inside InnoDB again tries to initialize progress report and
it leads to this failure.

row_merge_bulk_t::write_to_index(): Don't report the progress
for bulk load operation

Author: Yuchen Pei
Author Date: 2023-11-09 06:42:49 UTC

MDEV-28739 [demo] Commenting out spider sts and crd table updates fixes the problem

see https://jira.mariadb.org/browse/MDEV-28739?focusedCommentId=273704&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-273704

Author: Yuchen Pei
Author Date: 2023-11-09 00:43:27 UTC

MDEV-27576 Use reverse index for max/min optimization

We use a bool to indicate that the key part used is a descending
index, which will flip the functions and flags used in
get_index_max_value() and get_index_min_value(), that allows correct
optimization for max/min for descending index.

Author: Brandon Nesterenko
Author Date: 2023-11-08 23:57:58 UTC

MDEV-32551: ‘Read semi-sync reply magic number error’ warnings on master

A test to show how to induce the ‘Read semi-sync reply magic number
error’ message on a primary. In short, if semi-sync is turned on
during the hand-shake process between a primary and replica, but
later a user negates the rpl_semi_sync_slave_enabled variable while
the replica’s IO thread is running; if the io thread exits, the
replica can skip a necessary call to kill_connection() in
repl_semisync_slave.slave_stop() due to its reliance on a global
variable. Then, the replica will send a COM_QUIT packet to the
primary on an active semi-sync connection, causing the magic number
error.

The test in this patch exits the IO thread by forcing an error;
though note a call to STOP SLAVE could also do this, but it ends up
needing more synchronization. That is, the STOP SLAVE command also
tries to kill the vio of the replica, which makes a race with the IO
thread to try and send the COM_QUIT before this happens (which would
need more debug_sync to get around). See THD::awake_no_mutex for
details as to the killing of the replica’s vio.

Author: Sergei Golubchik
Author Date: 2023-11-08 14:44:18 UTC

MDEV-32617 deprecate secure_auth=0