binlog | A high performance C++ log library

There are options for filtering the binlog by table, but not the query logs.

There are no options for filtering the general query log. It is either enabled for all queries, or else it's disabled.

There are options for filtering the slow query log, but not by table. For example, to log only queries that take longer than N seconds, or queries that don't use an index. Percona Server adds some options to filter the slow query log based on sampling.

You can use a session variable to disable either slow query or general query logging for queries run in a given session. This is a dynamic setting, so you can change it at will. But you would need to change your client code to do this every time you query that specific table.

Another option is to implement log rotation for the slow query log, so it never grows too large. See https://www.percona.com/blog/2013/04/18/rotating-mysql-slow-logs-safely/

take a look at https://stackoverflow.blog/2015/07/23/announcing-stackexchange-precompilation/

That blog post is old, but the github source is, hmm, less old.

Anyway, it directly addresses your concern.

Summary of the comment thread above:

A combination of using OPTIMIZE TABLE to defragment the InnoDB tables after deleting lots of data, and PURGE BINARY LOGS to prune the accumulated binary logs helped to reduce the storage usage.

If you haven't configured the replication manually, you can use the following command inside the Maxscale container to set up replication between the servers:

I have solved the problem. Turns out I tweaked some of my first uploaded code because there was a syntax error and typos:

We finally found the issue. Mariadb was not able to correctly replicate the @SESSION timestamp. We changed the getLamportMicros() function like this for making it work:

pt-online-schema change copies the table structure and applies the desired ALTER TABLE to the zero-row table. This is virtually instantaneous. Then it creates triggers to mirror changes against the original table. Then it starts copying old data from the original table to the new table.

What happens to the binlog? It gets quite huge. The CREATE TABLE and ALTER TABLE and CREATE TRIGGER are pretty small. DDL is always statement-based in the binlog. The DML changes created by the triggers and the process of copying old data become transactions in the binlog. We prefer row-based binlogs, so these end up being pretty bulky.

gh-ost is similar, but without the triggers. gh-ost reads the binlog to find events that applied to the old table, and it applies those to the new table. Meanwhile, it also copies old data. Together these actions result in a similar volume of extra events in the binlog as occur when using pt-online-schema-change.

So you should check the amount of free disk space before you begin either of these online schema change operations. It will expand the binlogs approximately in proportion to the amount of data to be copied. And of course you need to store two copies of the whole table — the original and the altered version — temporarily, until the original table can be dropped at the end of the process.

I have had to run pt-online-schema change on large tables (500GB+) when I had a disk that was close to being full. It causes some tense moments. I had to PURGE BINARY LOGS periodically to get some more free space, because the schema change would fill the disk to 100% if I didn't! This is not a situation I recommend.

use:

Untested, but I'd grab the mysqlbinlog from MariaDB and use flashback to generate the SQL.

In theory, being just DML, this should be MySQL compatible or only require small modification to achieve the final result.