vacuum | Amazon Product Advertising API client | AWS library

I have master-slave (primary-standby) streaming replication set up on 2 physical nodes. Although the replication is working correctly and walsender and walreceiver both work fine, the files in the pg_wal folder on the slave node are not getting removed. This is a problem I have been facing every time I try to bring the slave node back after a crash. Here are the details of the problem:

postgresql.conf on master and slave/standby node

My goal is to create a stored procedure that loops through a select statement that will identify tables requiring a vacuum. I will call it from Lambda if I can get it to work. These are my ideas and code so far.

I'm trying to get some insight in this room for optimization for a SQL query (BigQuery). I have this segment of a WHERE clause that needs to include all instances where h.isEntrance is TRUE or where h.hitNumber = 1. I've tested it back and forth with CASE statements, and with OR statements for them, and the results aren't wholly conclusive.

It seems like the CASE is faster for shorter data pulls, and the OR is faster for longer data pulls, but that doesn't make sense to me. Is there a difference between these or is it likely something else driving this difference? Is one faster/is there another better option for incorporating this logical requirement into my query? Below the statement is my full query for context in case that's helpful.

Also open to any other optimizations I may have overlooked within this query as lowering the runtime for this query is paramount to its usefulness.

Thanks!

The goal is to have a Spark Streaming application that read data from Kafka and use Delta Lake to create store data. The granularity of the delta table is pretty granular, the first partition is the organization_id (there are more than 5000 organizations) and the second partition is the date.

The application has a expected latency, but it does not last more than one day up. The error is always about memory as I'll show below.

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006f8000000, 671088640, 0) failed; error='Cannot allocate memory' (errno=12)

There is no persistence and the memory is already high for the whole application.

Increasing memory and workes were the first things I've tried, but the number of partitions were changed as well, from 4 to 16.

I have identified the below script as being really useful for anyone running Amazon Redshift:

I have a Flask web application which I tested locally on my PC. Locally, everything is working correctly - no issues. So, I have moved on and deployed my Flask application on a Ubuntu 20.04 server behind uWSGI and Nginx. The issue I am facing now is, that my custom filter for Jinja2 template is 'Undefined'. I am not sure where the issue is, at all.

The error I get is:

When I use st_dwithin on a geography column with a GIST index, the explain plan shows that the narrowing condition on the index scan is changed into a && condition. With this different condition, the result set is missing rows that are included if a st_dwithin is performed using a full table scan when there is no index.

The condition in the explain plan without an index is

I'm sorry I don't have code because I really don't know / didn't find how this is possible.

I've created a database from big TXT files that were separated by character length, making so that if the limit for - let's say - address field was 200 chars, but for 1 value it only had 50, the other 150 will be whitespace...example:

I have a Postgres table T with 3 columns (w1, w2, occurrences), all three are integers and w1 and w2 are referencing to another table. So basically, there are two foreign keys and a value.
There are about 15 million unique values (indices) that w1 and w2 can become in each row. Currently, T holds around 1.8 billion rows, which are more or less permutations of the indices with the limitation, that - if you think of a symmetric matrix - only one value pair may exist. For example, there may be (252, 13, x) but not (13, 252, x). However, there is no sorting, so (5, 900, x) also may be in T (sorting is done during insert and depends on the value in the referenced table). These tuples (w1, w2) are unique. At the moment, there are 3 different indices on that table, a UNIQUE INDEX tdc_idx_1 on T (w1, w2), a INDEX tdc_idx_w1 on T (w1) and a INDEX tdc_idx_w2 on T (w2).

Basically, I want to run two different queries. The difficulty for me is to figure out where the bad performance comes from, or to accept the runtime given the "complexity" of the query and the table size (I'll guess that is not the case..). Short, I could need help in the structure of the queries and probably the handling of the table indices (or maybe the table design in general).
The most straightforward query for my desired result is