wal-e | Continuous Archiving for Postgres | Cloud Storage library

by wal-e Python Version: 1.1.1 License: BSD-3-Clause

X-Ray Key Features Code Snippets(1)Community Discussions(5)Vulnerabilities Install Support

kandi X-RAY | wal-e Summary

wal-e is a Python library typically used in Storage, Cloud Storage, Amazon S3 applications. wal-e has no vulnerabilities, it has build file available, it has a Permissive License and it has high support. However wal-e has 8 bugs. You can install using 'pip install wal-e' or download it from GitHub, PyPI.

Continuous Archiving for Postgres

Support

Quality

Security

License

Reuse

Support

wal-e has a highly active ecosystem.

It has 3380 star(s) with 325 fork(s). There are 73 watchers for this library.

It had no major release in the last 12 months.

There are 87 open issues and 163 have been closed. On average issues are closed in 374 days. There are 7 open pull requests and 0 closed requests.

It has a negative sentiment in the developer community.

The latest version of wal-e is 1.1.1

Quality

wal-e has 8 bugs (3 blocker, 0 critical, 5 major, 0 minor) and 79 code smells.

Security

wal-e has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

wal-e code analysis shows 0 unresolved vulnerabilities.

There are 8 security hotspots that need review.

License

wal-e is licensed under the BSD-3-Clause License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

wal-e releases are not available. You will need to build from source code and install.

Deployable package is available in PyPI.

Build file is available. You can build the component from source.

wal-e saves you 3290 person hours of effort in developing the same functionality from scratch.

It has 7065 lines of code, 659 functions and 112 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed wal-e and discovered the below as its top functions. This is intended to give you an instant insight into wal-e implemented functionality, and help decide if they suit your requirements.

Configure the Backup C extension
Generate a configuration hint
Format a log record
Run external programs
Partition a PostgreSQL directory
Segment a list of tar members into partitions
Ensures that a directory does not exist
Wrapper for lzop - retrieve
Convert a URI to a key
Extract a tarball from a file object
Uploads a file to the given URI
Run psql check
Build the argument parser
Iterate over the tasks in the queue
Read data from the stream
Create a CallInfo object from a bucket name
Creates a monkey patch for ssl
Return a connection to S3
Download a wal
Create directories for a segment
Uploads a wal archive to the server
Restore a wal
Fetch database from database
Uploads PostgreSQL
Configure logging
Delete num_to_retain backups

Get all kandi verified functions for this library.

wal-e Key Features

No Key Features are available at this moment for wal-e.

wal-e Examples and Code Snippets

Postgres docker container with wale

Shell

Lines of Code : 4

License : No License

Copy

WALE_AWS_ACCESS_KEY_ID`
WALE_AWS_SECRET_ACCESS_KEY`
WALE_S3_PREFIX="s3:///"
WALE_AWS_REGION=eu-west-1

Community Discussions

Trending Discussions on wal-e

How to drop the oldest entries from a remote TimescaleDb, maintaining the full local backup of the database?

Postgresql: How to take incremental backup with wal-e

Python: Permission denied error while setting up archive database

Running command line results in module not found as user postgres

wal-e backup-push error on Azure

QUESTION

How to drop the oldest entries from a remote TimescaleDb, maintaining the full local backup of the database?

Asked 2020-Jan-27 at 00:52

I've got a remote ever growing TimescaleDb database. I would like to keep only the most recent entries in the that Db, backing up the rest of the data to local drive, to achieve constant Db size on the server.

I thought of making full pg_dump backups before retaining and rebuilding the base locally from these backups.

Also, I could use WAL-E to create a continuous copy, somehow ignoring the deletions on the remote database.

What would be the most efficient way to achieve that?

...

ANSWER

Answered 2020-Jan-26 at 18:04

(TimescaleDB person here)

There are two main approaches here:

Use a backup system like WAL-E or pgBackRest to continuously replicate data to some other source (like S3).
Integrate your use of TimescaleDB's drop_chunks with your data extraction process.

The answer somewhat depends on how complex your data/database is.

If you are looking to primary archive your data in a single hypertable, I would recommend the latter: Use show_chunks to determine which chunks are over a certain range, compute a select over their range and write the data wherever, and then execute drop_chunks over the same range.

Source https://stackoverflow.com/questions/59920564

QUESTION

Postgresql: How to take incremental backup with wal-e

Asked 2020-Jan-10 at 06:50

I am new to postgresql. I've connected to my demo database with psycopg2 and inserted data with python faker. I want to take incremental backup with the tool WAL-E and want to see how it actually works. But the few tutorials aren't helping that much as they're not for naive users.

Can someone help explaining the steps simply how to backup data with WAL-E? That would be a great help ! I'm using psql 10.4 in Ubuntu 18.04.

Thanks

...

ANSWER

Answered 2018-Dec-04 at 08:51

Install Python & Dependencies

Source https://stackoverflow.com/questions/51186620

QUESTION

Python: Permission denied error while setting up archive database

Asked 2019-Feb-04 at 06:14

I was setting up scheduled backup of my postgresql database on my server using wal-e. I was following this link. (Just have a look at the article from Step 3)
When I execute the below command after initial setup .
sudo -u postgres /usr/bin/envdir /etc/wal-e.d/env /usr/local/bin/wal-e backup-push /var/lib/postgresql/9.5/main.

I get following error:-

...

ANSWER

Answered 2019-Feb-04 at 06:14

Faced the same issue. Your postgres user is not able to access the distribution packages.

Try running the below command. It should resolve the issue.

Source https://stackoverflow.com/questions/54093222

QUESTION

Running command line results in module not found as user postgres

Asked 2017-Sep-22 at 11:08

Im trying to run a package wal-e which I have installed as user root with sudo python3 -m pip install wal-e[aws,azure,google,swift].

I can run this command perfectly as user root using envdir /etc/wal-e.d/env wal-e backup-fetch /var/lib/postgresql/9.6/main LATEST.

However, when I sudo su - postgres and then run envdir /etc/wal-e.d/env wal-e backup-fetch /var/lib/postgresql/9.6/main LATEST, I get the error

...

ANSWER

Answered 2017-Sep-22 at 11:08

I had to run a strict setup process for wal-e in order for the package to function properly.

Virtually what it boiled down to was installing all necessary dependencies on the machine that I was working with before installing and creating the user postgres. If the user was created before all the dependencies were installed, I got permissions errors.

Source https://stackoverflow.com/questions/46338745

QUESTION

wal-e backup-push error on Azure

Asked 2017-Jun-15 at 07:37

I am getting the following error on executing the backup-push command:

...

ANSWER

Answered 2017-Jun-15 at 07:37

I tried to reproduce your issue successfully, and I reviewed the source code of the wal-e version 1.0.3, then I discovered the issue was caused by the wal-e version 1.0.3 actually required the azure-storage version less than 0.20.3, but installed the latest azure-storage version 0.34.3 when you were installing via pip3 install wal-e[azure]. Please check your current version of azure-storage via pip3 freeze | grep azure-storage.

So there are two solution for resolving the issue, as below.

Downgrade the azure-storage version to 0.20.3 via pip3 install azure-storage==0.20.3.
Or just upgrade the wal-e version to 1.1.0b1 which required the latest azure-storage version and be compatible with the old version, via pip3 install wal-e==1.1.0b1.

Hope it helps.

Source https://stackoverflow.com/questions/44555236

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install wal-e

You can install using 'pip install wal-e' or download it from GitHub, PyPI.
You can use wal-e like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: