rclone | cloud storage '' - Google Drive | Cloud Storage library

I am currently working on SFTP load to GCS bucket. However, I am able to do it for a limited number of files in any given SFTP directory by getting the list of files & iterating the absolute path of files. However, if the directory has too many files (or files within another folder), I am not able to do a simple ls & get the list of files to download from SFTP. Following is the working code to get the list of files in any given directory recursively from sftp:

Windows 11/ Powershell 7.2.1

I've added the following variables to user PATH and system PATH. C:\Program Files\rclone\rclone-v1.57.0-windows-amd64\rclone.exe

When I try to run rclone from Powershell or cmd I get the following message:

PS C:\Windows\System32> rclone rclone: The term 'rclone' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Sucessfully ran refreshenv and just to be sure I restarted Windows. After running $env:path -split ";" I can see C:\Program Files\rclone\rclone-v1.57.0-windows-amd64\rclone.exe is set correctly.

I can run rclone from within the program folder I get this notice.

PS C:\Program Files\rclone\rclone-v1.57.0-windows-amd64> rclone rclone: The term 'rclone' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Suggestion [3,General]: The command rclone was not found, but does exist in the current location. PowerShell does not load commands from the current location by default. If you trust this command, instead type: ".\rclone". See "get-help about_Command_Precedence" for more details.

After setting rclone on PATH it still isn't "seen", what am I doing wrong here?

I would like to copy a directory (with all its sub files, folders etc.) from azure file storage (not azure blob storage) to an aws s3 bucket on powershell.

So : Azure Files -> Amazon Web Services (AWS) S3

What I tried :

• using Rclone but rclone only takes into account blob and not file storage for the moment (see here)

• use of azcopy but azcopy does not allow the following combination Azure Files (SAS) -> Amazon Web Services (AWS) S3 (Access Key)

The process must not go through a local location (Virtual machine).

Any Ideas ? Thanks !

Question: I have two Google Shared Drives (Team Drives), let's say Movies and MoviesBackup. I need to backup whatever I upload to Movies into MoviesBackup. For this, I need an unidirectional sync from Movies to MoviesBackup, once daily.

What I have tried: I know of of Rclone, and have used its Command Line Interface to sync between two Shared Drives. Maybe if Rclone can be used from Google AppScript, I would set a daily trigger. But I seem to find no way to do so.

Any other solutions that work will be appreciated.

This question assumes you have used Google Drive Sync or at least have knowledge of what files it creates in your cloud drive

While using rclone to sync a local ubuntu directory to a Google Drive (a.k.a. gdrive) location, I found that rclone wasn't able to (error googleapi: Error 500: Internal Error, internalError; the Google Cloud Platform API console revealed that the gdrive API call drive.files.create was failing)

By location I mean the root of the directory structure that the Google Drive Sync app creates on the cloud (eg. emboldened of say: Computers/laptopName/(syncedFolder1,syncedFolder2,...)). In the current case, the gdrive sync app (famously unavailable on Linux) was running from a separate Windows machine. It was in this location that rclone wasn't able to create a dir.

Forget rclone. Trying to manually create the folder in the web app also fails as follows.

Working...

Could not execute action

Why is this happening and how to achieve this - making a directory in the cloud region where gdrive sync has put all my synced folders?

Before this, I checked this, snakemake's documentation, this,and this. Maybe they actually answered this question but I just didn't understand it.

In short, I create in one rule a number of files from other files, that both conform to a wildcard format. I don't know how many of these I create, since I don't know how many I originally download.

In all of the examples I've read so far, the output is directory("the/path"), while I have a "the/path/{id}.txt. So this I guess modifies how I call the checkpoints in the function itself. And the use of expand.

The rules in question are:

download_mv

textgrid_to_ctm_txt

get_MV_IDs

merge_ctms

The order of the rules should be:

download_mv (creates {MV_ID}.TEX and .wav (though not necessarily the same amount)

textgrid_to_ctm_txt (creates from {MV_ID}.TEX matching .txt and .ctm)

get_MV_IDs (should make a list of the .ctm files)

merge_ctms (should concatenate the ctm files)

kaldi_align (from the .wav and .txt directories creates one ctm file)

analyse_align (compares ctm file from kaldi_align the the merge_ctms)

upload_print_results

I have tried with the outputs of download_mv being directories, and then trying to get the IDs but I had different errors then. Now with snakemake --dryrun I get

I am trying to exec an rclone command via a PHP script. The plain text version of the call looks like this:

I have an Alpine Docker image I'm deploying using Fargate. This is the command it runs:

I'm on OSX Big Sur, previously using Google Backup and Sync to sync files between my computer and google drive.

I have setup Backup and Sync to sync any files in the folder /Users/doe/ODrive which contains 16GB file size.

After migrating to the new google drive since backup and sync got deprecated, I see a different behaviour.

The new google drive by default works like rclone. It creates a virtual drive under /Volumes/GoogleDrive and at the same time makes a symbolic link to /Users/doe/Google Drive for quick access.

Here's my problem:

1. If I choose to access any files offline it starts downloading them on disk taking unnecessary disk space since I already have all files downloaded on disk but on a different location /Users/doe/ODrive. How do I tell google drive to use those files and not download anything?

2. Theres a preference settings in the new google drive allowing to choose your desired directory location for google drive. If I set up that preference from the current setting /Volumes/GoogleDrive ---> /Users/doe/ODrive will that mess my ODrive folder and its content? I'd rather die than loose its content.

3. What's the difference between Folders from my computer and Folders from Drive. Isn't this two way communication like backup and sync was?