google-cloud-java | Google Cloud Client Library for Java | Cloud Storage library

by googleapis Java Version: v1.4.0 License: Apache-2.0

kandi X-RAY | google-cloud-java Summary

google-cloud-java is a Java library typically used in Storage, Cloud Storage applications. google-cloud-java has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has medium support. You can download it from GitHub, Maven.

Java idiomatic client for Google Cloud Platform services.

Support

Quality

Security

License

Reuse

kandi-support Support

google-cloud-java has a medium active ecosystem.
It has 1692 star(s) with 1057 fork(s). There are 185 watchers for this library.
There were 10 major release(s) in the last 6 months.
There are 44 open issues and 2178 have been closed. On average issues are closed in 16 days. There are 9 open pull requests and 0 closed requests.
It has a neutral sentiment in the developer community.
The latest version of google-cloud-java is v1.4.0

google-cloud-java Support

Best in #Cloud Storage

Average in #Cloud Storage

google-cloud-java Support

Best in #Cloud Storage

Average in #Cloud Storage

quality kandi Quality

google-cloud-java has 0 bugs and 0 code smells.

google-cloud-java Quality

Best in #Cloud Storage

Average in #Cloud Storage

google-cloud-java Quality

Best in #Cloud Storage

Average in #Cloud Storage

security Security

google-cloud-java has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
google-cloud-java code analysis shows 0 unresolved vulnerabilities.
There are 0 security hotspots that need review.

google-cloud-java Security

Best in #Cloud Storage

Average in #Cloud Storage

google-cloud-java Security

Best in #Cloud Storage

Average in #Cloud Storage

license License

google-cloud-java is licensed under the Apache-2.0 License. This license is Permissive.
Permissive licenses have the least restrictions, and you can use them in most projects.

google-cloud-java License

Best in #Cloud Storage

Average in #Cloud Storage

google-cloud-java License

Best in #Cloud Storage

Average in #Cloud Storage

build Reuse

google-cloud-java releases are available to install and integrate.
Deployable package is available in Maven.
Build file is available. You can build the component from source.
Installation instructions are not available. Examples and code snippets are available.
google-cloud-java saves you 762628 person hours of effort in developing the same functionality from scratch.
It has 369047 lines of code, 39256 functions and 1428 files.
It has high code complexity. Code complexity directly impacts maintainability of the code.

google-cloud-java Reuse

Best in #Cloud Storage

Average in #Cloud Storage

google-cloud-java Reuse

Best in #Cloud Storage

Average in #Cloud Storage

Top functions reviewed by kandi - BETA

kandi has reviewed google-cloud-java and discovered the below as its top functions. This is intended to give you an instant insight into google-cloud-java implemented functionality, and help decide if they suit your requirements.

Checks the current operating system .

Gets the metadata for the specified project .

Sets the CORS configuration for a bucket .

Creates a conditional binding between a bucket and bucket .

Removes the conditional binding on a bucket .

Counts the size of a file .

Gets metadata for a project .

Entry point for translation .

Creates a finding with the given source properties .

Generates a signed POST policy .

Get all kandi verified functions for this library.

google-cloud-java Key Features

Java on Google Cloud

google-cloud-java Examples and Code Snippets

Community Discussions

Trending Discussions on google-cloud-java

multi-part upload Google Storage
Where the file get transferred while using LocalStorageHelper service for google cloud storage?
ERROR i.g.i.ManagedChannelOrphanWrapper Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true
How to migrate from GoogleCredential to GoogleCredentials and still get access to People API?
How to get the number of rows inserted using BigQuery Streaming

Trending Discussions on google-cloud-java

QUESTION

multi-part upload Google Storage

Asked 2022-Feb-26 at 23:14

I'm trying to implement multi-part upload to Google Storage but to my surprise it does not seem to be straightforward (I could not find java example).

Only mention I found was in the XML API https://cloud.google.com/storage/docs/multipart-uploads

Also found some discussion around a compose API StorageExample.java#L446 mentioned here google-cloud-java issues 1440

Any recommendations how to do multipart upload?

ANSWER

Answered 2022-Feb-25 at 12:32

Currently Java Client library for multi part upload in Cloud Storage is not available. You can raise a feature request for the same in this link. As mentioned by John Hanley, the next best thing you can do is, do a parallel composite upload with gsutil (CLI), JSON and XML support/ resumable upload with Java libraries.

In parallel compose, the parallel writes can be done by using the JSON or XML API for Google Cloud Storage. Specifically, you would write a number of smaller objects in parallel and then (once all of those objects have been written) call the Compose request to compose them into one larger object.

If you're using the JSON API the compose documentation is at : https://cloud.google.com/storage/docs/json_api/v1/objects/compose

If you're using the XML API the compose documentation is at : https://cloud.google.com/storage/docs/reference-methods#putobject (see the compose query parameter).

Also there is an interesting document link provided by Kolban which you can try and work out. Also I would like to mention that you can have multi part uploads in Java, if you use the Google Drive API(v3). Here is the code example where we use the files.create method with uploadType=multipart.

Source https://stackoverflow.com/questions/71260411

QUESTION

ERROR i.g.i.ManagedChannelOrphanWrapper Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true

Asked 2021-Aug-17 at 12:27

I create subscribers to pull synchronously messages from a pubSub Topic. I used the exact setting as from the documentation example.

However, I am facing the following problem when I am trying to shut down subscribers and create new ones.

[pool-1-thread-8] ERROR i.g.i.ManagedChannelOrphanWrapper - *~*~*~ Channel ManagedChannelImpl{logId=47567, target=pubsub.googleapis.com:443} was not shutdown properly!!! ~*~*~*
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init> 
(ManagedChannelOrphanWrapper.java:93)
at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:53)
at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:44)
at io.grpc.internal.ManagedChannelImplBuilder.build(ManagedChannelImplBuilder.java:612)
at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.
java:261) at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel
(InstantiatingGrpcChannelProvider.java:340)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.access$1600
(InstantiatingGrpcChannelProvider.java:73)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$1.createSingleChannel
(InstantiatingGrpcChannelProvider.java:214)
at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:72)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.
 createChannel(InstantiatingGrpcChannelProvider.java:221)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider
.getTransportChannel(InstantiatingGrpcChannelProvider.java:204)
at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:169)
at com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:272)
at eu.hermes.esb.cloud.service.SubService.getSubscriber(SubService.java:48)
at eu.hermes.esb.cloud.runnables.SubTask.run(SubTask.java:31)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

my code for creating and shutting down subscribers:

   while (compositeConfigurationElement.getSubscriber().isEnabled()) {
        SubscriberStub subscriber;
        try {
            subscriber = subService.getSubscriber(compositeConfigurationElement);
            subService.pullAndSend(compositeConfigurationElement, subscriptionName, subscriber);
            subscriber.shutdown();
            subscriber.awaitTermination(20, TimeUnit.SECONDS);

        } catch (DeadlineExceededException e) {
            // this is a know issue at the moment ->
            // 1- https://github.com/googleapis/google-cloud-java/issues/4220
            // 2- https://stackoverflow.com/questions/60012138/google-cloud-function-pulling-from-pub-sub-subscription-throws-exception-deadl
            // 3- https://github.com/googleapis/google-cloud-java/issues/3648
            log.warn("Deadline exceeded for subscription");
        } catch (Exception e) {
            log.error("Exception with the subscription service", e);
            try {
                TimeUnit.SECONDS.sleep(10);
            } catch (Exception e2) {
                log.error("Exception during wait", e2);
            }
        }
    }

so, how to get rid of this error message ? and how on earth is this exception thrown although I am catching all exceptions.

Iam using google-cloud-pubsub:1.110.3

ANSWER

Answered 2021-Aug-17 at 12:27

so, I was missing that when an empty subscription happens the pull request throws a DeadlineExceededException which I am already catching, but I missed that in this case the subscriber is not shutdown.

so the solution is to shutdown the subscriber in the catch or in a finally block.

Source https://stackoverflow.com/questions/68804692

QUESTION

How to migrate from GoogleCredential to GoogleCredentials and still get access to People API?

Asked 2021-Jun-09 at 13:42

Background

For an app I'm working on, which uses People API using credentials (user login). Once the user gives the credentials, I can access various Google APIs, such as People API. An example is one to get a list of contacts:

https://developers.google.com/people/api/rest/v1/people.connections/list

I've noticed that the class com.google.api.client.googleapis.auth.oauth2.GoogleCredential has become deprecated:

https://googleapis.dev/java/google-api-client/latest/com/google/api/client/googleapis/auth/oauth2/GoogleCredential.html

The problem

The app has old code that is based on some old G+ code (here) to reach contacts via the Google account. Here's a snippet of the most important part of it, which causes me trouble of migrating away from it:

object GoogleOuthHelper {
    @WorkerThread
    fun setUp(context: Context, serverAuthCode: String?): Services {
        val httpTransport: HttpTransport = NetHttpTransport()
        val jsonFactory = JacksonFactory.getDefaultInstance()
        // Redirect URL for web based applications. Can be empty too.
        val redirectUrl = "urn:ietf:wg:oauth:2.0:oob"
        // Exchange auth code for access token
        val tokenResponse = GoogleAuthorizationCodeTokenRequest(
            httpTransport, jsonFactory, GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET,
            serverAuthCode, redirectUrl)
            .execute()
        // Then, create a GoogleCredential object using the tokens from GoogleTokenResponse
        val credential = GoogleCredential.Builder()
            .setClientSecrets(GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET)
            .setTransport(httpTransport)
            .setJsonFactory(jsonFactory)
            .build()
        val accessToken = tokenResponse.accessToken
        getDefaultSecuredSharedPreferences(context).edit()
            .putString(SecuredSharedPreferences.KEY__GOOGLE_ACCESS_TOKEN, accessToken).apply()
        credential.setFromTokenResponse(tokenResponse)
        val appPackageName = context.packageName
        val peopleServiceApi = PeopleService.Builder(httpTransport, jsonFactory, credential)
            .setApplicationName(appPackageName)
            .build()
        val peopleService = peopleServiceApi.people()
        val otherContactsService = peopleServiceApi.otherContacts()
        val contactGroups = peopleServiceApi.contactGroups()
        return Services(peopleService, otherContactsService, contactGroups)
    }

    class Services(
        /**https://developers.google.com/people/api/rest/v1/people*/
        val peopleService: PeopleService.People,
        /**https://developers.google.com/people/api/rest/v1/otherContacts*/
        val otherContactsService: OtherContacts,
        /**https://developers.google.com/people/api/rest/v1/contactGroups*/
        val contactGroups: ContactGroups)
}

The problem is even from the very beginning:

The class GoogleCredentials doesn't seem to accept anything I got above it for the GoogleCredential class.

To add more to it, this function takes "serverAuthCode" as parameter, which is from GoogleSignInAccount, but to get it, I need to use the deprecated GoogleApiClient class:

        fun prepareGoogleApiClient(someContext: Context): GoogleApiClient {
            val context = someContext.applicationContext ?: someContext
            val gso = GoogleSignInOptions.Builder(GoogleSignInOptions.DEFAULT_SIGN_IN)
                .requestServerAuthCode(GOOGLE_CLIENT_ID)
                .requestEmail()
                .requestScopes(
                    Scope(PeopleServiceScopes.CONTACTS_READONLY),
                    Scope(PeopleServiceScopes.USERINFO_PROFILE),
                    Scope(PeopleServiceScopes.USER_EMAILS_READ),
                    Scope(PeopleServiceScopes.CONTACTS),
                    Scope(PeopleServiceScopes.CONTACTS_OTHER_READONLY)
                )
                .build()
            return GoogleApiClient.Builder(context)
                .addApi(Auth.GOOGLE_SIGN_IN_API, gso)
                .build()
        }

And this is what I do with it:

val connectionResult = googleApiClient!!.blockingConnect()
if (!connectionResult.isSuccess)
    return
val operation = Auth.GoogleSignInApi.silentSignIn(googleApiClient)
val googleSignInResult: GoogleSignInResult = operation.await()
val googleSignInAccount = googleSignInResult.signInAccount
//use googleSignInAccount.serverAuthCode in setUp() function above

Gradle file has these dependencies:

// https://mvnrepository.com/artifact/com.google.auth/google-auth-library-oauth2-http
implementation 'com.google.auth:google-auth-library-oauth2-http:0.26.0'
// https://github.com/googleapis/google-api-java-client-services/tree/master/clients/google-api-services-people/v1#gradle   https://mvnrepository.com/artifact/com.google.apis/google-api-services-people
implementation 'com.google.apis:google-api-services-people:v1-rev20210515-1.31.0'

What I've tried

Other than looking at the docs (and failing to see a resemblance to what I have to handle), I tried to write about this here.

Sadly I can't find how to migrate away from the old code yet.

I tried to ask there how can I migrate (here and here) but didn't get an answer yet.

The questions

How can I migrate away from GoogleCredential to GoogleCredentials while still using the various APIs such as People API?

In other words: How can I avoid using any of those deprecated classes (GoogleCredential and GoogleApiClient), on Android, while still being able to use the various APIs?

ANSWER

Answered 2021-Jun-09 at 13:42

How can I migrate away from GoogleCredential to GoogleCredentials while still using the various APIs such as People API?

In other words: How can I avoid using any of those deprecated classes (GoogleCredential and GoogleApiClient), on Android, while still being able to use the various APIs?

Although you can make GoogleCredentials work directly, it will be better to use a class derived from GoogleCredentials such as UserCredentials which will accommodate token refresh as GoogleCredential does. GoogleCredentials is more of a foundational class.

The following code makes use of UserCredentials. This is mostly what you have presented, but I have changed some credential storing logic for the purpose of the demo. This code has no deprecated methods except startActivityForResult().

serverAuthCode is available from GoogleSignInAccount. Take a look at Moving Past GoogleApiClient on how to remove the dependency on GoogleApiClient. I have update my public gist of RestApiActivity from the Google signin quickstart which shows how to use GoogleOauthHelper as well as GoogleApi.

GoogleOauthHelper.kt

object GoogleOauthHelper {
    @WorkerThread
    fun setUp(context: Context, serverAuthCode: String?): Services {
        val httpTransport: HttpTransport = NetHttpTransport()
        val jsonFactory = GsonFactory.getDefaultInstance()
        // Redirect URL for web based applications. Can be empty too.
        val redirectUrl = "urn:ietf:wg:oauth:2.0:oob"

        // Patch for demo
        val GOOGLE_CLIENT_ID = context.getString(R.string.GOOGLE_CLIENT_ID)
        val GOOGLE_CLIENT_SECRET = context.getString(R.string.GOOGLE_CLIENT_SECRET)

        var accessToken: AccessToken? = null
        var refreshToken =
            getDefaultSecuredSharedPreferences(context).getString(
                SecuredSharedPreferences.KEY_GOOGLE_REFRESH_TOKEN,
                null
            )

        if (refreshToken == null) {
            /*  Did we lose the refresh token, or is this the first time? Refresh tokens are only
                returned the first time after the user grants us permission to use the API. So, if
                this is the first time doing this, we should get a refresh token. If it's not the
                first time, we will not get a refresh token, so we will proceed with the access
                token alone. If the access token expires (in about an hour), we will get an error.
                What we should do is to ask the user to reauthorize the app and go through the
                OAuth flow again to recover a refresh token.

                See https://developers.google.com/identity/protocols/oauth2#expiration regarding
                how a refresh token can become invalid.
            */
            val tokenResponse = GoogleAuthorizationCodeTokenRequest(
                httpTransport, jsonFactory, GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET,
                serverAuthCode, redirectUrl
            ).execute()

            refreshToken = tokenResponse.refreshToken
            if (refreshToken != null) {
                getDefaultSecuredSharedPreferences(context).edit()
                    .putString(SecuredSharedPreferences.KEY_GOOGLE_REFRESH_TOKEN, refreshToken)
                    .apply()
            } else {
                Log.d("Applog", "No refresh token. Going with access token alone.")
                val expiresAtMilliseconds =
                    Clock.SYSTEM.currentTimeMillis() + tokenResponse.expiresInSeconds * 1000
                accessToken = AccessToken(tokenResponse.accessToken, Date(expiresAtMilliseconds))
            }
        }

        Log.d("Applog", "Refresh token: $refreshToken")
        // UserCredentials extends GoogleCredentials and permits token refreshing.
        val googleCredentials = UserCredentials.newBuilder().run {
            clientId = GOOGLE_CLIENT_ID
            clientSecret = GOOGLE_CLIENT_SECRET
            setRefreshToken(refreshToken)
            setAccessToken(accessToken)
            build()
        }

        // Save access token on change
        googleCredentials.addChangeListener { oAuth2Credentials ->
            saveAccessToken(oAuth2Credentials.accessToken)
        }

        val requestInitializer: HttpRequestInitializer = HttpCredentialsAdapter(googleCredentials)
        val appPackageName = context.packageName

        val peopleServiceApi = PeopleService.Builder(httpTransport, jsonFactory, requestInitializer)
            .setApplicationName(appPackageName)
            .build()

        return peopleServiceApi.run { Services(people(), otherContacts(), contactGroups()) }
    }

    private fun saveAccessToken(accessToken: AccessToken) {
        // Persist the token securely.
        Log.d("Applog", "Access token has changed: ${accessToken.tokenValue}")
    }

    // Warning insecure!: Patch for demo.
    private fun getDefaultSecuredSharedPreferences(context: Context): SharedPreferences {
        return PreferenceManager.getDefaultSharedPreferences(context)
    }

    // Warning insecure!: Patch for demo.
    object SecuredSharedPreferences {
        const val KEY_GOOGLE_REFRESH_TOKEN = "GOOGLE_REFRESH_TOKEN"
    }

    class Services(
        /**https://developers.google.com/people/api/rest/v1/people*/
        val peopleService: PeopleService.People,
        /**https://developers.google.com/people/api/rest/v1/otherContacts*/
        val otherContactsService: PeopleService.OtherContacts,
        /**https://developers.google.com/people/api/rest/v1/contactGroups*/
        val contactGroups: PeopleService.ContactGroups,
    )
}

I have posted a demo project on GitHub.

Source https://stackoverflow.com/questions/67500479

QUESTION

How to get the number of rows inserted using BigQuery Streaming

Asked 2020-Oct-26 at 12:48

I am reading data from a CSV file, inserting the data to a Big Query table using insertAll() method from Streaming Insert as shown below:

InsertAllResponse response = dfsf.insertAll(InsertAllRequest.newBuilder(tableId).setRows(rows).build());

rows here is an Iterable declared like this:

Iterable<InsertAllRequest.RowToInsert> rows

Now, I am actually batching the rows to insert into a size of 500 as suggested here - link to suggestion

After all the data has been inserted, how do I count the total number of rows that were inserted? I want to find that out and log it to log4j.

ANSWER

Answered 2020-Oct-26 at 12:48

This can be done one of two ways

The BigQuery Jobs API via the getQueryResults https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/getQueryResults
Cloud Logging, the output you want in the tableDataChange field.

Here is a sample output:

{
  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
    "status": {},
    "authenticationInfo": {
      "principalEmail": "service_account"
    },
    "requestMetadata": {
      "callerIp": "2600:1900:2000:1b:400::27",
      "callerSuppliedUserAgent": "gl-python/3.7.1 grpc/1.22.0 gax/1.14.2 gapic/1.12.1 gccl/1.12.1,gzip(gfe)"
    },
    "serviceName": "bigquery.googleapis.com",
    "methodName": "google.cloud.bigquery.v2.JobService.InsertJob",
    "authorizationInfo": [
      {
        "resource": "projects/project_id/datasets/dataset/tables/table",
        "permission": "bigquery.tables.updateData",
        "granted": true
      }
    ],
    "resourceName": "projects/project_id/datasets/dataset/tables/table",
    "metadata": {
      "tableDataChange": {
        "deletedRowsCount": "2",
        "insertedRowsCount": "2",
        "reason": "QUERY",
        "jobName": "projects/PRJOECT_ID/jobs/85f19bdd-aff5-4abe-9283-9f0bc9ed3ce8"
      },
      "@type": "type.googleapis.com/google.cloud.audit.BigQueryAuditMetadata"
    }
  },
  "insertId": "7x7ye390qm",
  "resource": {
    "type": "bigquery_dataset",
    "labels": {
      "project_id": "PRJOECT_ID",
      "dataset_id": "dataset-id"
    }
  },
  "timestamp": "2020-10-26T07:00:22.960735Z",
  "severity": "INFO",
  "logName": "projects/PRJOECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access",
  "receiveTimestamp": "2020-10-26T07:00:23.763159336Z"
}

Source https://stackoverflow.com/questions/64535139

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install google-cloud-java

You can download it from GitHub, Maven.
You can use google-cloud-java like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the google-cloud-java component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

Libraries are available on GitHub and Maven Central for developing Java applications that interact with individual Google Cloud services:. If the service is not listed, google-api-java-client interfaces with additional Google Cloud APIs using a legacy REST interface.

Find more information at: