gcsfuse | space file system for interacting with Google Cloud Storage | Cloud Storage library

by GoogleCloudPlatform Go Version: v0.42.5 License: Apache-2.0

X-Ray Key Features Code Snippets Community Discussions(10)Vulnerabilities Install Support

kandi X-RAY | gcsfuse Summary

gcsfuse is a Go library typically used in Storage, Cloud Storage, Amazon S3 applications. gcsfuse has no bugs, it has no vulnerabilities, it has a Permissive License and it has medium support. You can download it from GitHub.

gcsfuse is a user-space file system for interacting with Google Cloud Storage.

Support

Quality

Security

License

Reuse

Support

gcsfuse has a medium active ecosystem.

It has 1707 star(s) with 395 fork(s). There are 57 watchers for this library.

It had no major release in the last 12 months.

There are 54 open issues and 517 have been closed. On average issues are closed in 271 days. There are 16 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of gcsfuse is v0.42.5

Quality

gcsfuse has no bugs reported.

Security

gcsfuse has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

License

gcsfuse is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

gcsfuse releases are available to install and integrate.

Installation instructions are not available. Examples and code snippets are available.

Top functions reviewed by kandi - BETA

kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of gcsfuse

Get all kandi verified functions for this library.

gcsfuse Key Features

No Key Features are available at this moment for gcsfuse.

gcsfuse Examples and Code Snippets

No Code Snippets are available at this moment for gcsfuse.

Community Discussions

Trending Discussions on gcsfuse

Issue building from a Dockerfile that used to work

Install gcsfuse in Google Cloud Container Optimized OS (COS)

"server doesn't have a resource type "pods"" while installing NVIDIA Clara Deploy

Read/Write data to/from Cloud Storage Bucket using gcsfuse

gcsfuse command fail with gcsfuse takes exactly two arguments

google colab with google cloud storage data egress

Cloud storage directory disappears when restarting instance

GCSfuse "Unknown bucket" when Gcloud can see it. Service account seems correct

Mounting GCS over FUSE

How to Use Cloud Storage Fuse to backup files when deleted in local machine

QUESTION

Issue building from a Dockerfile that used to work

Asked 2021-May-20 at 14:13

I'm having an issue building a docker image from a dockerfile that used to work:

(My dockerfile has more steps, but this is enough to reproduce)

...

ANSWER

Answered 2021-May-20 at 14:13

This is a known issue. Read this for more info.

You can first add the correct repository GPG key using the following command.

Source https://stackoverflow.com/questions/67621790

QUESTION

Install gcsfuse in Google Cloud Container Optimized OS (COS)

Asked 2020-Dec-17 at 08:09

Does anyone know how do I install gcsfuse into a Google Container Optimized OS so that I could mount bucket in the VM instance itself.

I tried running a docker with gcsfuse mounted volume from host. The docker container successfully mounted the bucket into the host volume. When I view from the host volume, it is empty, but the container has the bucket data.

...

ANSWER

Answered 2020-Dec-17 at 01:30

The issue explained on this git, the docker-default apparmor profile denies all mounts. This '--security-opt apparmor=unconfined' flag make it possible to mount a GCS bucket.

Source https://stackoverflow.com/questions/65333210

QUESTION

"server doesn't have a resource type "pods"" while installing NVIDIA Clara Deploy

Asked 2020-Oct-19 at 18:56

I am trying to install the latest version of NVIDIA Clara Deploy Bootstrap following the official documentations (this & this). At one step of the installation, these is a shellscript named "bootstrap.sh" - which is meant to install all the dependencies including Kubernetes & kubectl, along with cluster creation. But upon running sudo ./bootstrap.sh, I am getting this error: error: the server doesn't have a resource type "pods".

What I have done so far: I am fairly new to Kubernetes. So I've tried solution from this answer, tried to run kubectl get pods which gives me No resources found.. I have also tried kubectl auth can-i get podswhich gives me yes. Inside etc/kubernetes/manifests, it was empty which is supposed to have conf files that I have looked from the answer, so I ran sudo kubeadm init.

Here is the full error message:

...

ANSWER

Answered 2020-Oct-19 at 18:56

1. Instance:

Source https://stackoverflow.com/questions/64407408

QUESTION

Read/Write data to/from Cloud Storage Bucket using gcsfuse

Asked 2020-Oct-09 at 07:04

To mount Google Cloud Storage Bucket onto a directory on a local machine for processing. Using a manjaro environment and installed gcsfuse manually.

in the gs://bucket01, there are directories containing jpg and json files

...

ANSWER

Answered 2020-Oct-09 at 07:04

Please try using Implicit directories

As mentioned above, by default there is no allowance for the implicit existence of directories. Since the usual file system operations like mkdir will do the right thing, if you set up a bucket's structure using only gcsfuse then you will not notice anything odd about this. If, however, you use some other tool to set up objects in GCS (such as the storage browser in the Google Developers Console), you may notice that not all objects are visible until you create leading directories for them.

gcsfuse supports a flag called --implicit-dirs that changes the behavio

Source https://stackoverflow.com/questions/64273277

QUESTION

gcsfuse command fail with gcsfuse takes exactly two arguments

Asked 2020-Aug-11 at 14:57

I am using GCSFuse for mounting the GCS bucket to my user pod in JupyterHub, but it always fail with the error message gcsfuse takes exactly two arguments.

Here is my DockerFile:

...

ANSWER

Answered 2020-Aug-08 at 20:51

I'm not an expert (and even a user) of JupyterHub. My answer is generic

I'm seeing 2 way to solve your issue

You can mount your secret file (if you have your json key in a file) into the container at runtime. However I don't know the jupyterhub syntax for achieving this
You can try this

In your jupyterhub yaml file, change the env var of your json key file content

Source https://stackoverflow.com/questions/63317861

QUESTION

google colab with google cloud storage data egress

Asked 2020-Jul-29 at 18:54

I'm trying to teach my network on data that is stored in google cloud storage. I'm teaching using google colab pro resources and when I do that, I got around 50$ bill a day for "egress between NA and EU". I'm located in Russia and data storage is in Germany so I have absolutely no idea why this data egress to NA. How can I stop this behavior and why does it happen, because I don't want to pay for something I don't really need.

Link between storage and colab looks like this:

...

ANSWER

Answered 2020-Jun-10 at 11:18

Google Colab server is run in USA (North America).

So, to avoid network cost, you should host your gcs bucket in USA as well (instead of Germany).

Source https://stackoverflow.com/questions/62301970

QUESTION

Cloud storage directory disappears when restarting instance

Asked 2020-Jun-16 at 15:02

I have attached a standard persistent disk to the instance. On that disk I have created a directory 'cloud-storage' (as always mkdir) in which I mount the bucket (Google Cloud Storage). Also I have added to fstab the command to mount it:

...

ANSWER

Answered 2020-Jun-16 at 15:02

You are right, whenever you reboot your instance, you must mount your GCS bucket again. I know this can be a bummer, but there is a workaround for this:

Startup scripts let you set scripts that will automatically run each time your GCE instance starts or reboots. What you can do is to add the commands to mount the GCS bucket in this startup script and when the VM is up and ready to serve, you should see your GCS bucket mounted and ready to work. These scripts are written in Bash.

You can add the logic in this script about the directory after the standard disk and before mounting your bucket without going too-manual.

Also, make sure that the flow of your startup script meets your other tech requirements. Not adding a dependence after running the script that uses it or something similar.

I know some of the last scenarios might not suit to yours but I just want to make sure you have a wider insight for possible future implementations.

Hope this is helpful! :)

Source https://stackoverflow.com/questions/62395551

QUESTION

GCSfuse "Unknown bucket" when Gcloud can see it. Service account seems correct

Asked 2020-Apr-29 at 14:53

I'm trying to mount a bucket to my gcp instance, but it fails both with a newly created bucket and my existing one. I can see the buckets with gsutil ls, and I can copy files to them with gsutil cp.

However, when I try to mount using GCSFuse, the following happens:

...

ANSWER

Answered 2019-Nov-01 at 18:06

Looking at the documentation here:

https://cloud.google.com/storage/docs/gcs-fuse

It seems that one should specify the bucket name and not a bucket URL.

For example:

Source https://stackoverflow.com/questions/58663731

QUESTION

Mounting GCS over FUSE

Asked 2020-Apr-25 at 14:59

When mounting GCS through FUSE, gcsfuse does the file/files stored in the mount point are saved on the local disk file system (meaning does it consume actual disk space) or all data is stored directly to the cloud?

...

ANSWER

Answered 2018-Oct-01 at 22:40

gcsfuse downloads files to a temporary location, and keeps a cache. This is usually the right thing because otherwise you can use up all your available ram. If you want, you can prevent storing a local copy on disk by setting --temp-dir to a ramdisk.

Source https://stackoverflow.com/questions/52594122

QUESTION

How to Use Cloud Storage Fuse to backup files when deleted in local machine

Asked 2020-Jan-16 at 08:55

I installed gcsfuse on my local macOS system and mounted a folder to cloud storage bucket. everythings works fine.

but, If deleted a file from mounted folder also deleting on bucket. I don't want this to be happen.

when ever I delete any files, It should only delete on my local machine.

Can anyone help me to do it.

Thanks.

...

ANSWER

Answered 2020-Jan-16 at 08:55

You can't do this with official version of gcsfuse.

As workaround, you can activate the object versioning. Thereby, even is you delete a file, a versioned copy still live in your Bucket. You lost nothing.

This video is also great for explaining the versioning

If you really want to use gcsfuse with your special feature, you can fork the project and remove the delete part in the code of the open source project

Source https://stackoverflow.com/questions/59764554

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install gcsfuse

You can download it from GitHub.

Support

gcsfuse is open source software, released under the Apache license. It is distributed as-is, without warranties or conditions of any kind. For support, visit Server Fault. Tag your questions with gcsfuse and google-cloud-platform, and make sure to look at previous questions and answers before asking a new one. For bugs and feature requests, please file an issue.

Find more information at: