Explore all Cloud Functions open source software, libraries, packages, source code, cloud functions and APIs.

Popular New Releases in Cloud Functions

serverless-offline

v8.6.0

serverless-next.js

v3.8.0-alpha.0

aws-lambda-rust-runtime

lambda_runtime release v0.5.1

Klayers

Multi Python Builds

azure-functions-core-tools

3.0.4484

Popular Libraries in Cloud Functions

functions-samples

by firebase doticonjavascriptdoticon

star image 10874 doticonApache-2.0

Collection of sample apps showcasing popular use cases using Cloud Functions for Firebase

aws-doc-sdk-examples

by awsdocs doticonjavadoticon

star image 5156 doticonApache-2.0

Welcome to the AWS Code Examples Repository. This repo contains code examples used in the AWS documentation, AWS SDK Developer Guides, and more. For more information, see the Readme.rst file below.

serverless-offline

by dherault doticonjavascriptdoticon

star image 4486 doticonMIT

Emulate AWS λ and API Gateway locally when developing your Serverless project

serverless-next.js

by serverless-nextjs doticontypescriptdoticon

star image 3445 doticonMIT

⚡ Deploy your Next.js apps on AWS Lambda@Edge via Serverless Components

chrome-aws-lambda

by alixaxel doticontypescriptdoticon

star image 2267 doticonMIT

Chromium Binary for AWS Lambda and Google Cloud Functions

aws-lambda-rust-runtime

by awslabs doticonrustdoticon

star image 2029 doticonApache-2.0

A Rust runtime for AWS Lambda

AutoSpotting

by AutoSpotting doticongodoticon

star image 1937 doticonNOASSERTION

Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups. Installs in minutes using CloudFormation or Terraform. Convenient to deploy at scale using StackSets. Uses tagging to avoid launch configuration changes. Automated spot termination handling. Reliable fallback to on-demand instances.

goad

by goadapp doticongodoticon

star image 1857 doticonMIT

Goad is an AWS Lambda powered, highly distributed, load testing tool

aws-lambda-developer-guide

by awsdocs doticonjavadoticon

star image 1588 doticonNOASSERTION

The AWS Lambda Developer Guide

Trending New libraries in Cloud Functions

terraform-aws-next-js

by milliHQ doticontypescriptdoticon

star image 796 doticonApache-2.0

Terraform module for building and deploying Next.js apps to AWS. Supports SSR (Lambda), Static (S3) and API (Lambda) pages.

aws-lambda-runtime-interface-emulator

by aws doticongodoticon

star image 481 doticonApache-2.0

llama

by nelhage doticongodoticon

star image 387 doticonApache-2.0

terraform-aws-next-js

by dealmore doticontypescriptdoticon

star image 263 doticonApache-2.0

Terraform module for building and deploying Next.js apps to AWS. Supports SSR (Lambda), Static (S3) and API (Lambda) pages.

aws-lambda-extensions

by aws-samples doticongodoticon

star image 223 doticonNOASSERTION

A collection of sample extensions to help you get started with AWS Lambda Extensions

web2img

by EtherDream doticonjavascriptdoticon

star image 199 doticon

bundle web files into a single image

saga-orchestration-serverless

by Azure-Samples doticoncsharpdoticon

star image 186 doticonMIT

An orchestration-based saga implementation reference in a serverless architecture

micro-aws-lambda

by Albert-Gao doticontypescriptdoticon

star image 184 doticonMIT

A 7KB and 0 dependencies AWS Lambda library which supports middleware and easy debug.

azure-functions-dotnet-worker

by Azure doticoncsharpdoticon

star image 181 doticonMIT

Azure Functions out-of-process .NET language worker

Top Authors in Cloud Functions

1

aws-samples

75 Libraries

star icon3122

2

awslabs

23 Libraries

star icon4301

3

Azure-Samples

14 Libraries

star icon796

4

theburningmonk

14 Libraries

star icon351

5

Azure

13 Libraries

star icon2633

6

SamVerschueren

10 Libraries

star icon170

7

amazon-archives

10 Libraries

star icon155

8

aws

10 Libraries

star icon2941

9

SHAHAZADISHAGUFTHA

9 Libraries

star icon18

10

DataDog

7 Libraries

star icon429

1

75 Libraries

star icon3122

2

23 Libraries

star icon4301

3

14 Libraries

star icon796

4

14 Libraries

star icon351

5

13 Libraries

star icon2633

6

10 Libraries

star icon170

7

10 Libraries

star icon155

8

10 Libraries

star icon2941

9

9 Libraries

star icon18

10

7 Libraries

star icon429

Trending Kits in Cloud Functions

The AWS SDK for JavaScript offers access to the AWS Lambda service through APIs, allowing developers to create, manage, and invoke Lambda functions from within their JavaScript applications. JavaScript AWS Lambda Libraries are a robust set of software components to develop and deploy applications. They enable developers to use the same code for client and server-side applications and provide a wide range of features and functionalities.

 

Lambda functions are serverless functions that are triggered by events. These events can be anything from a new data record being added to a database to a user clicking a button in an application. When an event is triggered, the Lambda function is executed. Lambda functions can be written in Node.js, Python, JavaScript, or C# and perform various tasks such as image processing, data analysis, or even machine learning. AWS Lambda is an Amazon Web Services (AWS) computing service that empowers you to execute the code without provisioning or managing servers. Without worrying about server provisioning or management, this "pay-per-use" cloud computing service enables developers to quickly and easily design and manage apps that respond to events and run code responding to those events. Applications of any size may be built, tested, deployed, and scaled quickly and efficiently while only incurring costs for the compute time required. 

 

The AWS Lambda Library contains a set of ready-made libraries that developers can use to develop their Lambda functions. These libraries are pre-packaged and contain functions, classes, and other tools designed to make it easier for developers to develop their Lambda functions. Some commonly used JavaScript AWS libraries that facilitate developers are shim, lambda-image, slack-commands, nodejs-lambda-middleware, synthetic-tests-pings, template-mailer-aws-lambda-client, lambda-oauth2, aws-lambda-toolkit, serverless-response, swf-hook.


Check out the list below to find more popular Node JS AWS libraries for your app development: 

Trending Discussions on Cloud Functions

Automatic delete Cloud Logging logs

Firebase function failing to deploy

Implementation of Distributed-Counter-Extension for local emulator

Build a container image from inside a cloud function

Error when import firebase-functions-test when testing with mocha

Parsing error: Cannot read file '\tsconfig.json' eslint after following Firebase Cloud Functions initialization instructions

firebase function with realtime database error

Firebase Cloud Functions min-instances setting seems to be ignored

Receiving inconsistent data in Go api

Get the client_id of the IAM proxy on GCP Cloud composer

QUESTION

Automatic delete Cloud Logging logs

Asked 2022-Mar-23 at 16:41

I would like to know if there is a way to periodically delete logs from inside Cloud Logging.

I have setup Firebase with Cloud Functions and i have an automatic Cloud Logging logs injection done for each function call.
I don't want especially to stop sending logs to Cloud Logging, but i would like to be able to manage my costs by deleting older logs.

ANSWER

Answered 2022-Mar-23 at 16:41

You can set a retention policy on your Cloud Logging bucket to match with your requirements, which can auto-delete logs after between 1 day and 10 years.

Source https://stackoverflow.com/questions/71590582

QUESTION

Firebase function failing to deploy

Asked 2022-Feb-17 at 15:31

I'm trying to create a Firebase Function but I'm running into a deploy error, even when deploying the default helloworld function.

The firebase-debug.log file mentions this: Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld.

I have been trying to debug and so far have not been able to solve it...

firebase-debug.log

1[info] Functions deploy had errors with the following functions:
2    helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i  functions: cleaning up build files... 
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10    at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11    at processTicksAndRejections (internal/process/task_queues.js:93:5)
12    at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13    at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14    at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error] 
16[error] Error: There was an error deploying functions
17

index.js:

1[info] Functions deploy had errors with the following functions:
2    helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i  functions: cleaning up build files... 
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10    at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11    at processTicksAndRejections (internal/process/task_queues.js:93:5)
12    at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13    at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14    at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error] 
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23  functions.logger.info("Hello logs!", {structuredData: true});
24  response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29

Package.json:

1[info] Functions deploy had errors with the following functions:
2    helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i  functions: cleaning up build files... 
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10    at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11    at processTicksAndRejections (internal/process/task_queues.js:93:5)
12    at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13    at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14    at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error] 
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23  functions.logger.info("Hello logs!", {structuredData: true});
24  response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29{
30  "name": "functions",
31  "description": "Cloud Functions for Firebase",
32  "scripts": {
33    "lint": "eslint",
34    "serve": "firebase emulators:start --only functions",
35    "shell": "firebase functions:shell",
36    "start": "npm run shell",
37    "deploy": "firebase deploy --only functions",
38    "logs": "firebase functions:log"
39  },
40  "engines": {
41    "node": "14"
42  },
43  "main": "index.js",
44  "dependencies": {
45    "firebase-admin": "^9.8.0",
46    "firebase-functions": "^3.14.1",
47    "blurhash": "^1.1.4"
48  },
49  "devDependencies": {
50    "eslint": "^7.6.0",
51    "eslint-config-google": "^0.14.0",
52    "firebase-functions-test": "^0.2.0"
53  },
54  "private": true
55}
56

My version of node: v14.16.0

Appreciate your help.

ANSWER

Answered 2022-Feb-06 at 14:36

Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld.

The Firebase Function deployment failed because it cannot find the image built based on your function app. There might be a problem building in your app, it could be your dependencies or files.

I replicated your issue, received the same error and solved it. There's a problem with the package.json file and package-lock.json. If you just add(without installing) your dependency in package.json you should delete or remove your package-lock.json that will be found in function directory before you deploy it again using the deployment command:

1[info] Functions deploy had errors with the following functions:
2    helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i  functions: cleaning up build files... 
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10    at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11    at processTicksAndRejections (internal/process/task_queues.js:93:5)
12    at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13    at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14    at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error] 
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23  functions.logger.info("Hello logs!", {structuredData: true});
24  response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29{
30  "name": "functions",
31  "description": "Cloud Functions for Firebase",
32  "scripts": {
33    "lint": "eslint",
34    "serve": "firebase emulators:start --only functions",
35    "shell": "firebase functions:shell",
36    "start": "npm run shell",
37    "deploy": "firebase deploy --only functions",
38    "logs": "firebase functions:log"
39  },
40  "engines": {
41    "node": "14"
42  },
43  "main": "index.js",
44  "dependencies": {
45    "firebase-admin": "^9.8.0",
46    "firebase-functions": "^3.14.1",
47    "blurhash": "^1.1.4"
48  },
49  "devDependencies": {
50    "eslint": "^7.6.0",
51    "eslint-config-google": "^0.14.0",
52    "firebase-functions-test": "^0.2.0"
53  },
54  "private": true
55}
56firebase deploy --only functions
57

or you can just install your dependency to make sure it will be added to your package.json and package-lock.json file, deploy again. For example:

1[info] Functions deploy had errors with the following functions:
2    helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i  functions: cleaning up build files... 
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10    at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11    at processTicksAndRejections (internal/process/task_queues.js:93:5)
12    at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13    at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14    at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error] 
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23  functions.logger.info("Hello logs!", {structuredData: true});
24  response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29{
30  "name": "functions",
31  "description": "Cloud Functions for Firebase",
32  "scripts": {
33    "lint": "eslint",
34    "serve": "firebase emulators:start --only functions",
35    "shell": "firebase functions:shell",
36    "start": "npm run shell",
37    "deploy": "firebase deploy --only functions",
38    "logs": "firebase functions:log"
39  },
40  "engines": {
41    "node": "14"
42  },
43  "main": "index.js",
44  "dependencies": {
45    "firebase-admin": "^9.8.0",
46    "firebase-functions": "^3.14.1",
47    "blurhash": "^1.1.4"
48  },
49  "devDependencies": {
50    "eslint": "^7.6.0",
51    "eslint-config-google": "^0.14.0",
52    "firebase-functions-test": "^0.2.0"
53  },
54  "private": true
55}
56firebase deploy --only functions
57npm install --save blurhash
58

Source https://stackoverflow.com/questions/70027316

QUESTION

Implementation of Distributed-Counter-Extension for local emulator

Asked 2022-Jan-24 at 17:55

The firebase extension for a distributed counter can be directly installed for the cloud and works just fine. To develop new features for an app I need to do this on the emulator to not interrupt the running server.

As the firebase extensions simply are cloud Functions*, I thought about implementing the cloud function in my emulator by getting the source code from the extension itself. This worked fine for other extentions so far...

Error and Disfunction when implementing

When implementing the javaScript version that i get the following error:

function ignored because the unknown emulator does not exist or is not running.

This problem can be fixed by rewriting the export line of the index.jsfunctions, but is wont provide the expected functionality of the extension anyhow:

1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2

to

1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3
Question

Am i missing some implementaion or is an emulator missing? How can i implement this firestore extention?

There are some similar questions, but dont really talk about the implementation of the whole extension:

ANSWER

Answered 2022-Jan-24 at 17:55

firebaser here

Firebase Extensions normally declare their triggers in the extension.yaml file, instead of in the code itself. Therefore, in order to emulate an extension in this way, you'd need to move the triggers over to the code.

For your specific example of the 'worker' function, the extension declares what document to listen to here, so we'll copy the document over to the code:

1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3exports.worker = functions.firestore.document(`${process.env.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {...});
4

You'll have to do the same for each other background triggered function in the extension - find the trigger in extension.yaml, and move it over to the code.

You'll also need to provide the param values that you normally configure the extension with as environment variables. To do this, you'll have to add a .env file to your functions directory containing the environment variables you want to set. For the your case, that will look something like:

1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3exports.worker = functions.firestore.document(`${process.env.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {...});
4LOCATION=//Put your location here
5INTERNAL_STATE_PATH=//Put your path here
6SCHEDULE_FREQUENCY=1
7

Then, you'll need to enable the emulator to read this file:

1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3exports.worker = functions.firestore.document(`${process.env.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {...});
4LOCATION=//Put your location here
5INTERNAL_STATE_PATH=//Put your path here
6SCHEDULE_FREQUENCY=1
7firebase --open-sesame dotenv
8

Then, you should be able to run the extension code in the emulator.

As you might be thinking, this is not a very easy process right now. We are currently working on adding native support for Extensions to the Firebase Emulator Suite. As soon as that is available, I'll make sure to come back here and update this answer with how to use it, and hopefully this will be much easier for you!

PS: If you are interested in potentially helping us test out this feature early, let me know and we'll reach out as soon as it is ready to test.

Source https://stackoverflow.com/questions/70836970

QUESTION

Build a container image from inside a cloud function

Asked 2021-Dec-22 at 00:59

Context: I am training a very similar model per bigquery dataset in Google Vertex AI, but I want to have a custom training image for each existing dataset (in Google BigQuery). In that sense, I need to programatically build a custom Docker Image in the container registry on demand. My idea was to have a Google Cloud Function do it, being triggered by PubSub topic with information regarding which dataset I want to build the training container for. So naturally, the function will write the Dockerfile and pertinent scripts to a /tmp folder within Cloud Functions (the only writable place as per my knowledge). However, when I try to actually build the container within this script, apparently, it doesn't find the /tmp folder or its contents, even though they are there (checked with logging operations).

The troubling code so far:

1def build_container(dataset=str):
2
3    with open('container_template/Dockerfile','r') as f:
4        dockerfile = f.read()
5    dockerfile = dockerfile.replace('@dataset',dataset)
6    f.close()
7
8    os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10    with open('/tmp/Dockerfile','w') as docker_config:
11        docker_config.write(dockerfile)
12
13    docker_config.close()
14
15    shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17    build_client = cloudbuild_v1.CloudBuildClient()
18
19    build = cloudbuild_v1.Build()
20
21    build.steps = [{'name':'gcr.io/cloud-builders/docker',
22                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23                    {'name':'gcr.io/cloud-builders/docker',
24                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
25
26    build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28    build_result = build_operation.result()
29
30    logger.info('Build Result: {}'.format(build_result.status))
31

When I check the cloud build logs I get: Step #0: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/Dockerfile: no such file or directory

ANSWER

Answered 2021-Dec-21 at 11:07

I've locally tested building a container image using Cloud Build Client Python library. It turns out to have the same error even the Dockerfile file is existing in current directory:

error:

Step #0: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /workspace/Dockerfile: no such file or directory

build steps:

1def build_container(dataset=str):
2
3    with open('container_template/Dockerfile','r') as f:
4        dockerfile = f.read()
5    dockerfile = dockerfile.replace('@dataset',dataset)
6    f.close()
7
8    os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10    with open('/tmp/Dockerfile','w') as docker_config:
11        docker_config.write(dockerfile)
12
13    docker_config.close()
14
15    shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17    build_client = cloudbuild_v1.CloudBuildClient()
18
19    build = cloudbuild_v1.Build()
20
21    build.steps = [{'name':'gcr.io/cloud-builders/docker',
22                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23                    {'name':'gcr.io/cloud-builders/docker',
24                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
25
26    build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28    build_result = build_operation.result()
29
30    logger.info('Build Result: {}'.format(build_result.status))
31    build_client = cloudbuild_v1.CloudBuildClient()
32
33    build = cloudbuild_v1.Build()
34
35    build.steps = [{'name':'gcr.io/cloud-builders/docker',
36                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37                    {'name':'gcr.io/cloud-builders/docker',
38                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
39
40    build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42    build_result = build_operation.result()
43

Since it uses the API method I followed this documentation. You will see the source is present in API method. It is the missing key to move forward the problem. In StorageSource, you must to specify the bucket and object_. You need to compress your source code and upload it in Cloud Storage bucket. For example:

  1. Run the following command to compress your source code:
1def build_container(dataset=str):
2
3    with open('container_template/Dockerfile','r') as f:
4        dockerfile = f.read()
5    dockerfile = dockerfile.replace('@dataset',dataset)
6    f.close()
7
8    os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10    with open('/tmp/Dockerfile','w') as docker_config:
11        docker_config.write(dockerfile)
12
13    docker_config.close()
14
15    shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17    build_client = cloudbuild_v1.CloudBuildClient()
18
19    build = cloudbuild_v1.Build()
20
21    build.steps = [{'name':'gcr.io/cloud-builders/docker',
22                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23                    {'name':'gcr.io/cloud-builders/docker',
24                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
25
26    build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28    build_result = build_operation.result()
29
30    logger.info('Build Result: {}'.format(build_result.status))
31    build_client = cloudbuild_v1.CloudBuildClient()
32
33    build = cloudbuild_v1.Build()
34
35    build.steps = [{'name':'gcr.io/cloud-builders/docker',
36                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37                    {'name':'gcr.io/cloud-builders/docker',
38                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
39
40    build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42    build_result = build_operation.result()
43tar -cvzf sourcecode.tar.gz .
44
  1. Upload to Cloud Storage bucket(you can use Cloud Build bucket):
1def build_container(dataset=str):
2
3    with open('container_template/Dockerfile','r') as f:
4        dockerfile = f.read()
5    dockerfile = dockerfile.replace('@dataset',dataset)
6    f.close()
7
8    os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10    with open('/tmp/Dockerfile','w') as docker_config:
11        docker_config.write(dockerfile)
12
13    docker_config.close()
14
15    shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17    build_client = cloudbuild_v1.CloudBuildClient()
18
19    build = cloudbuild_v1.Build()
20
21    build.steps = [{'name':'gcr.io/cloud-builders/docker',
22                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23                    {'name':'gcr.io/cloud-builders/docker',
24                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
25
26    build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28    build_result = build_operation.result()
29
30    logger.info('Build Result: {}'.format(build_result.status))
31    build_client = cloudbuild_v1.CloudBuildClient()
32
33    build = cloudbuild_v1.Build()
34
35    build.steps = [{'name':'gcr.io/cloud-builders/docker',
36                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37                    {'name':'gcr.io/cloud-builders/docker',
38                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
39
40    build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42    build_result = build_operation.result()
43tar -cvzf sourcecode.tar.gz .
44gsutil cp sourcecode.tar.gz gs://myproject_cloudbuild
45
  1. build.source:
1def build_container(dataset=str):
2
3    with open('container_template/Dockerfile','r') as f:
4        dockerfile = f.read()
5    dockerfile = dockerfile.replace('@dataset',dataset)
6    f.close()
7
8    os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10    with open('/tmp/Dockerfile','w') as docker_config:
11        docker_config.write(dockerfile)
12
13    docker_config.close()
14
15    shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17    build_client = cloudbuild_v1.CloudBuildClient()
18
19    build = cloudbuild_v1.Build()
20
21    build.steps = [{'name':'gcr.io/cloud-builders/docker',
22                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23                    {'name':'gcr.io/cloud-builders/docker',
24                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
25
26    build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28    build_result = build_operation.result()
29
30    logger.info('Build Result: {}'.format(build_result.status))
31    build_client = cloudbuild_v1.CloudBuildClient()
32
33    build = cloudbuild_v1.Build()
34
35    build.steps = [{'name':'gcr.io/cloud-builders/docker',
36                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37                    {'name':'gcr.io/cloud-builders/docker',
38                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
39
40    build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42    build_result = build_operation.result()
43tar -cvzf sourcecode.tar.gz .
44gsutil cp sourcecode.tar.gz gs://myproject_cloudbuild
45    build_client = cloudbuild_v1.CloudBuildClient()
46
47    build = cloudbuild_v1.Build()
48    
49    build.source = {"storage_source":{"bucket":"myproject_cloudbuild", "object_":"gs://myproject_cloudbuild/sourcecode.tar.gz"}}
50
51    build.steps = [{'name':'gcr.io/cloud-builders/docker',
52                    'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
53                    {'name':'gcr.io/cloud-builders/docker',
54                    'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]  
55
56    build_operation = build_client.create_build(project_id=myprojectid,build=build)
57
58    build_result = build_operation.result()
59

As a result it solved the problem building an image using Client Library. I recommended to do all of this inside your Cloud Function.

Source https://stackoverflow.com/questions/70428362

QUESTION

Error when import firebase-functions-test when testing with mocha

Asked 2021-Dec-02 at 09:53

I am trying to setup a Firebase Cloud Functions repo to run mocha test. However, it throws the following error when I use import * as firebase from "firebase-functions-test"; or const firebase = require("firebase-functions-test")();. You can see in my code that I haven't even called the actual firebase functions yet so I think this a setup issue.

Question: What change do I need to make mocha test running for Firebase Functions testing using import syntax?

Working test code

1import { assert } from "chai";
2describe("Sanity Check", () => {
3  it("should pass", () => {
4    assert.equal(0, 0);
5  });
6});
7
8

Failed Test Code using require

1import { assert } from "chai";
2describe("Sanity Check", () => {
3  it("should pass", () => {
4    assert.equal(0, 0);
5  });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11  it("should pass", () => {
12    assert.equal(0, 0);
13    test.cleanup();
14  });
15});
16

Failed code using import

1import { assert } from "chai";
2describe("Sanity Check", () => {
3  it("should pass", () => {
4    assert.equal(0, 0);
5  });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11  it("should pass", () => {
12    assert.equal(0, 0);
13    test.cleanup();
14  });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21  it("should pass", () => {
22    assert.equal(0, 0);
23    test.cleanup();
24  });
25});
26

Error for the failure

1import { assert } from "chai";
2describe("Sanity Check", () => {
3  it("should pass", () => {
4    assert.equal(0, 0);
5  });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11  it("should pass", () => {
12    assert.equal(0, 0);
13    test.cleanup();
14  });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21  it("should pass", () => {
22    assert.equal(0, 0);
23    test.cleanup();
24  });
25});
26> functions@ test /Users/cupidchan/temp/functions
27> mocha -r ts-node/register test/**/*.spec.ts
28
29
30Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/encoder' is not defined by "exports" in /Users/cupidchan/temp/functions/node_modules/firebase-functions/package.json
31    at new NodeError (internal/errors.js:322:7)
32    at throwExportsNotFound (internal/modules/esm/resolve.js:322:9)
33    at packageExportsResolve (internal/modules/esm/resolve.js:545:3)
34    at resolveExports (internal/modules/cjs/loader.js:450:36)
35    at Function.Module._findPath (internal/modules/cjs/loader.js:490:31)
36    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:888:27)
37    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/cupidchan/temp/functions/node_modules/@cspotcode/source-map-support/source-map-support.js:679:30)
38    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
39    at Module.require (internal/modules/cjs/loader.js:974:19)
40    at require (internal/modules/cjs/helpers.js:93:18)
41    at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/providers/firestore.js:26:19)
42    at Module._compile (internal/modules/cjs/loader.js:1085:14)
43    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
44    at Module.load (internal/modules/cjs/loader.js:950:32)
45    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
46    at Module.require (internal/modules/cjs/loader.js:974:19)
47    at require (internal/modules/cjs/helpers.js:93:18)
48    at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/features.js:9:19)
49    at Module._compile (internal/modules/cjs/loader.js:1085:14)
50    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
51    at Module.load (internal/modules/cjs/loader.js:950:32)
52    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
53    at Module.require (internal/modules/cjs/loader.js:974:19)
54    at require (internal/modules/cjs/helpers.js:93:18)
55    at module.exports (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/index.js:30:20)
56    at Object.<anonymous> (/Users/cupidchan/temp/functions/test/index.spec.ts:9:14)
57    at Module._compile (internal/modules/cjs/loader.js:1085:14)
58    at Module.m._compile (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1371:23)
59    at Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
60    at Object.require.extensions.<computed> [as .ts] (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1374:12)
61    at Module.load (internal/modules/cjs/loader.js:950:32)
62    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
63    at Module.require (internal/modules/cjs/loader.js:974:19)
64    at require (internal/modules/cjs/helpers.js:93:18)
65    at Object.exports.requireOrImport (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:56:20)
66    at async Object.exports.loadFilesAsync (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:88:20)
67    at async singleRun (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run-helpers.js:125:3)
68    at async Object.exports.handler (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run.js:374:5)
69npm ERR! Test failed.  See above for more details.
70

package.json

1import { assert } from "chai";
2describe("Sanity Check", () => {
3  it("should pass", () => {
4    assert.equal(0, 0);
5  });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11  it("should pass", () => {
12    assert.equal(0, 0);
13    test.cleanup();
14  });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21  it("should pass", () => {
22    assert.equal(0, 0);
23    test.cleanup();
24  });
25});
26> functions@ test /Users/cupidchan/temp/functions
27> mocha -r ts-node/register test/**/*.spec.ts
28
29
30Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/encoder' is not defined by "exports" in /Users/cupidchan/temp/functions/node_modules/firebase-functions/package.json
31    at new NodeError (internal/errors.js:322:7)
32    at throwExportsNotFound (internal/modules/esm/resolve.js:322:9)
33    at packageExportsResolve (internal/modules/esm/resolve.js:545:3)
34    at resolveExports (internal/modules/cjs/loader.js:450:36)
35    at Function.Module._findPath (internal/modules/cjs/loader.js:490:31)
36    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:888:27)
37    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/cupidchan/temp/functions/node_modules/@cspotcode/source-map-support/source-map-support.js:679:30)
38    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
39    at Module.require (internal/modules/cjs/loader.js:974:19)
40    at require (internal/modules/cjs/helpers.js:93:18)
41    at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/providers/firestore.js:26:19)
42    at Module._compile (internal/modules/cjs/loader.js:1085:14)
43    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
44    at Module.load (internal/modules/cjs/loader.js:950:32)
45    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
46    at Module.require (internal/modules/cjs/loader.js:974:19)
47    at require (internal/modules/cjs/helpers.js:93:18)
48    at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/features.js:9:19)
49    at Module._compile (internal/modules/cjs/loader.js:1085:14)
50    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
51    at Module.load (internal/modules/cjs/loader.js:950:32)
52    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
53    at Module.require (internal/modules/cjs/loader.js:974:19)
54    at require (internal/modules/cjs/helpers.js:93:18)
55    at module.exports (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/index.js:30:20)
56    at Object.<anonymous> (/Users/cupidchan/temp/functions/test/index.spec.ts:9:14)
57    at Module._compile (internal/modules/cjs/loader.js:1085:14)
58    at Module.m._compile (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1371:23)
59    at Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
60    at Object.require.extensions.<computed> [as .ts] (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1374:12)
61    at Module.load (internal/modules/cjs/loader.js:950:32)
62    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
63    at Module.require (internal/modules/cjs/loader.js:974:19)
64    at require (internal/modules/cjs/helpers.js:93:18)
65    at Object.exports.requireOrImport (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:56:20)
66    at async Object.exports.loadFilesAsync (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:88:20)
67    at async singleRun (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run-helpers.js:125:3)
68    at async Object.exports.handler (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run.js:374:5)
69npm ERR! Test failed.  See above for more details.
70{
71  "name": "functions",
72  "scripts": {
73    "lint": "eslint --ext .js,.ts .",
74    "build": "tsc",
75    "serve": "npm run build && firebase emulators:start --only functions",
76    "shell": "npm run build && firebase functions:shell",
77    "start": "npm run shell",
78    "deploy": "firebase deploy --only functions",
79    "logs": "firebase functions:log",
80    "test": "mocha -r ts-node/register test/**/*.spec.ts --reporter spec"
81  },
82  "engines": {
83    "node": "14"
84  },
85  "main": "lib/index.js",
86  "dependencies": {
87    "firebase-admin": "^9.8.0",
88    "firebase-functions": "^3.14.1"
89  },
90  "devDependencies": {
91    "@types/chai": "^4.2.22",
92    "@types/mocha": "^9.0.0",
93    "@types/node": "^16.11.11",
94    "@typescript-eslint/eslint-plugin": "^3.9.1",
95    "@typescript-eslint/parser": "^3.8.0",
96    "chai": "^4.3.4",
97    "eslint": "^7.6.0",
98    "eslint-config-google": "^0.14.0",
99    "eslint-config-prettier": "^8.3.0",
100    "eslint-plugin-import": "^2.22.0",
101    "eslint-plugin-prettier": "^4.0.0",
102    "esm": "^3.2.25",
103    "firebase-functions-test": "^0.2.3",
104    "mocha": "^9.1.3",
105    "prettier": "^2.5.0",
106    "ts-node": "^10.4.0",
107    "typescript": "^3.8.0"
108  },
109  "private": true
110}
111

tsconfig.json

1import { assert } from "chai";
2describe("Sanity Check", () => {
3  it("should pass", () => {
4    assert.equal(0, 0);
5  });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11  it("should pass", () => {
12    assert.equal(0, 0);
13    test.cleanup();
14  });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21  it("should pass", () => {
22    assert.equal(0, 0);
23    test.cleanup();
24  });
25});
26> functions@ test /Users/cupidchan/temp/functions
27> mocha -r ts-node/register test/**/*.spec.ts
28
29
30Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/encoder' is not defined by "exports" in /Users/cupidchan/temp/functions/node_modules/firebase-functions/package.json
31    at new NodeError (internal/errors.js:322:7)
32    at throwExportsNotFound (internal/modules/esm/resolve.js:322:9)
33    at packageExportsResolve (internal/modules/esm/resolve.js:545:3)
34    at resolveExports (internal/modules/cjs/loader.js:450:36)
35    at Function.Module._findPath (internal/modules/cjs/loader.js:490:31)
36    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:888:27)
37    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/cupidchan/temp/functions/node_modules/@cspotcode/source-map-support/source-map-support.js:679:30)
38    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
39    at Module.require (internal/modules/cjs/loader.js:974:19)
40    at require (internal/modules/cjs/helpers.js:93:18)
41    at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/providers/firestore.js:26:19)
42    at Module._compile (internal/modules/cjs/loader.js:1085:14)
43    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
44    at Module.load (internal/modules/cjs/loader.js:950:32)
45    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
46    at Module.require (internal/modules/cjs/loader.js:974:19)
47    at require (internal/modules/cjs/helpers.js:93:18)
48    at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/features.js:9:19)
49    at Module._compile (internal/modules/cjs/loader.js:1085:14)
50    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
51    at Module.load (internal/modules/cjs/loader.js:950:32)
52    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
53    at Module.require (internal/modules/cjs/loader.js:974:19)
54    at require (internal/modules/cjs/helpers.js:93:18)
55    at module.exports (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/index.js:30:20)
56    at Object.<anonymous> (/Users/cupidchan/temp/functions/test/index.spec.ts:9:14)
57    at Module._compile (internal/modules/cjs/loader.js:1085:14)
58    at Module.m._compile (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1371:23)
59    at Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
60    at Object.require.extensions.<computed> [as .ts] (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1374:12)
61    at Module.load (internal/modules/cjs/loader.js:950:32)
62    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
63    at Module.require (internal/modules/cjs/loader.js:974:19)
64    at require (internal/modules/cjs/helpers.js:93:18)
65    at Object.exports.requireOrImport (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:56:20)
66    at async Object.exports.loadFilesAsync (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:88:20)
67    at async singleRun (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run-helpers.js:125:3)
68    at async Object.exports.handler (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run.js:374:5)
69npm ERR! Test failed.  See above for more details.
70{
71  "name": "functions",
72  "scripts": {
73    "lint": "eslint --ext .js,.ts .",
74    "build": "tsc",
75    "serve": "npm run build && firebase emulators:start --only functions",
76    "shell": "npm run build && firebase functions:shell",
77    "start": "npm run shell",
78    "deploy": "firebase deploy --only functions",
79    "logs": "firebase functions:log",
80    "test": "mocha -r ts-node/register test/**/*.spec.ts --reporter spec"
81  },
82  "engines": {
83    "node": "14"
84  },
85  "main": "lib/index.js",
86  "dependencies": {
87    "firebase-admin": "^9.8.0",
88    "firebase-functions": "^3.14.1"
89  },
90  "devDependencies": {
91    "@types/chai": "^4.2.22",
92    "@types/mocha": "^9.0.0",
93    "@types/node": "^16.11.11",
94    "@typescript-eslint/eslint-plugin": "^3.9.1",
95    "@typescript-eslint/parser": "^3.8.0",
96    "chai": "^4.3.4",
97    "eslint": "^7.6.0",
98    "eslint-config-google": "^0.14.0",
99    "eslint-config-prettier": "^8.3.0",
100    "eslint-plugin-import": "^2.22.0",
101    "eslint-plugin-prettier": "^4.0.0",
102    "esm": "^3.2.25",
103    "firebase-functions-test": "^0.2.3",
104    "mocha": "^9.1.3",
105    "prettier": "^2.5.0",
106    "ts-node": "^10.4.0",
107    "typescript": "^3.8.0"
108  },
109  "private": true
110}
111{
112  "compilerOptions": {
113    "module": "commonjs",
114    "noImplicitReturns": true,
115    "noUnusedLocals": true,
116    "outDir": "lib",
117    "sourceMap": true,
118    "strict": true,
119    "target": "es2017"
120  },
121  "compileOnSave": true,
122  "include": ["src", "test"]
123}
124

ANSWER

Answered 2021-Dec-02 at 09:53

This error should be resolved after specifying the latest version of the

  • firebase-functions, v3.16.0, and

  • firebase-functions-test, v0.3.3.

Source https://stackoverflow.com/questions/70183270

QUESTION

Parsing error: Cannot read file '\tsconfig.json' eslint after following Firebase Cloud Functions initialization instructions

Asked 2021-Nov-16 at 16:17
Problem

Right after my TypeScript project initialization in VSCode using firebase tools for composing Firebase Cloud Functions following the official documentation the very first line of the index.ts file displays an error:

Parsing error: Cannot read file '\tsconfig.json' eslint [1,1] enter image description here

and the .eslintrc.js displays an error:

File is a CommonJS module; it may be converted to an ES6 module.ts(80001) enter image description here

Since all files are auto-generated these errors are a complete surprise and I want to get rid of them.

Versions

For the record, here are the versions installed:

1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5
Installation process

These are the commands I used in the powershell in VSCode with some info/warnings:

1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE   package: undefined,
15npm WARN EBADENGINE   required: { node: '14' },
16npm WARN EBADENGINE   current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30  1:13  warning  'functions' is defined but never used  @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33
Files tsconfig.json
1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE   package: undefined,
15npm WARN EBADENGINE   required: { node: '14' },
16npm WARN EBADENGINE   current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30  1:13  warning  'functions' is defined but never used  @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34  "compilerOptions": {
35    "module": "commonjs",
36    "noImplicitReturns": true,
37    "noUnusedLocals": true,
38    "outDir": "lib",
39    "sourceMap": true,
40    "strict": true,
41    "target": "es2017"
42  },
43  "compileOnSave": true,
44  "include": [
45    "src"
46  ]
47}
48
.eslintrc.js
1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE   package: undefined,
15npm WARN EBADENGINE   required: { node: '14' },
16npm WARN EBADENGINE   current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30  1:13  warning  'functions' is defined but never used  @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34  "compilerOptions": {
35    "module": "commonjs",
36    "noImplicitReturns": true,
37    "noUnusedLocals": true,
38    "outDir": "lib",
39    "sourceMap": true,
40    "strict": true,
41    "target": "es2017"
42  },
43  "compileOnSave": true,
44  "include": [
45    "src"
46  ]
47}
48module.exports = {
49  root: true,
50  env: {
51    es6: true,
52    node: true,
53  },
54  extends: [
55    "eslint:recommended",
56    "plugin:import/errors",
57    "plugin:import/warnings",
58    "plugin:import/typescript",
59    "google",
60    "plugin:@typescript-eslint/recommended",
61  ],
62  parser: "@typescript-eslint/parser",
63  parserOptions: {
64    project: ["tsconfig.json", "tsconfig.dev.json"],
65    sourceType: "module",
66  },
67  ignorePatterns: [
68    "/lib/**/*", // Ignore built files.
69  ],
70  plugins: [
71    "@typescript-eslint",
72    "import",
73  ],
74  rules: {
75    "quotes": ["error", "double"],
76    "import/no-unresolved": 0,
77  },
78};
79
80
index.ts
1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE   package: undefined,
15npm WARN EBADENGINE   required: { node: '14' },
16npm WARN EBADENGINE   current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30  1:13  warning  'functions' is defined but never used  @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34  "compilerOptions": {
35    "module": "commonjs",
36    "noImplicitReturns": true,
37    "noUnusedLocals": true,
38    "outDir": "lib",
39    "sourceMap": true,
40    "strict": true,
41    "target": "es2017"
42  },
43  "compileOnSave": true,
44  "include": [
45    "src"
46  ]
47}
48module.exports = {
49  root: true,
50  env: {
51    es6: true,
52    node: true,
53  },
54  extends: [
55    "eslint:recommended",
56    "plugin:import/errors",
57    "plugin:import/warnings",
58    "plugin:import/typescript",
59    "google",
60    "plugin:@typescript-eslint/recommended",
61  ],
62  parser: "@typescript-eslint/parser",
63  parserOptions: {
64    project: ["tsconfig.json", "tsconfig.dev.json"],
65    sourceType: "module",
66  },
67  ignorePatterns: [
68    "/lib/**/*", // Ignore built files.
69  ],
70  plugins: [
71    "@typescript-eslint",
72    "import",
73  ],
74  rules: {
75    "quotes": ["error", "double"],
76    "import/no-unresolved": 0,
77  },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83  functions.logger.info("Hello logs!", {structuredData: true});
84  response.send("Hello from Firebase!");
85});
86
What I tried
  1. Reinitialization
  2. Creating the .vscode folder with settings.json file with
1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE   package: undefined,
15npm WARN EBADENGINE   required: { node: '14' },
16npm WARN EBADENGINE   current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30  1:13  warning  'functions' is defined but never used  @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34  "compilerOptions": {
35    "module": "commonjs",
36    "noImplicitReturns": true,
37    "noUnusedLocals": true,
38    "outDir": "lib",
39    "sourceMap": true,
40    "strict": true,
41    "target": "es2017"
42  },
43  "compileOnSave": true,
44  "include": [
45    "src"
46  ]
47}
48module.exports = {
49  root: true,
50  env: {
51    es6: true,
52    node: true,
53  },
54  extends: [
55    "eslint:recommended",
56    "plugin:import/errors",
57    "plugin:import/warnings",
58    "plugin:import/typescript",
59    "google",
60    "plugin:@typescript-eslint/recommended",
61  ],
62  parser: "@typescript-eslint/parser",
63  parserOptions: {
64    project: ["tsconfig.json", "tsconfig.dev.json"],
65    sourceType: "module",
66  },
67  ignorePatterns: [
68    "/lib/**/*", // Ignore built files.
69  ],
70  plugins: [
71    "@typescript-eslint",
72    "import",
73  ],
74  rules: {
75    "quotes": ["error", "double"],
76    "import/no-unresolved": 0,
77  },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83  functions.logger.info("Hello logs!", {structuredData: true});
84  response.send("Hello from Firebase!");
85});
86 {
87   "eslint.workingDirectories": [
88       "src" // and "functions"
89   ]
90 }
91
  1. Updating your eslintrc.json file with the following line: "project":"PROJECT_NAME/tsconfig.json"
  2. Updating .eslintrc.js by setting tsconfigRootDir: __dirname in parserOptions
  3. Deleting the ESLint extension. The error was gone, but firebase deploy command didn't allow the code deployement.

So, the related thread didn't really help

ANSWER

Answered 2021-Nov-16 at 16:17

Ok, I have solved the problem with a great help of this github thread False positive Error - TS6133 error (declared but its value is never read) report.

I have changed "noUnusedLocals" setting in the tsconfig.json file from the default true to false, so the file becomes:

1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE   package: undefined,
15npm WARN EBADENGINE   required: { node: '14' },
16npm WARN EBADENGINE   current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30  1:13  warning  'functions' is defined but never used  @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34  "compilerOptions": {
35    "module": "commonjs",
36    "noImplicitReturns": true,
37    "noUnusedLocals": true,
38    "outDir": "lib",
39    "sourceMap": true,
40    "strict": true,
41    "target": "es2017"
42  },
43  "compileOnSave": true,
44  "include": [
45    "src"
46  ]
47}
48module.exports = {
49  root: true,
50  env: {
51    es6: true,
52    node: true,
53  },
54  extends: [
55    "eslint:recommended",
56    "plugin:import/errors",
57    "plugin:import/warnings",
58    "plugin:import/typescript",
59    "google",
60    "plugin:@typescript-eslint/recommended",
61  ],
62  parser: "@typescript-eslint/parser",
63  parserOptions: {
64    project: ["tsconfig.json", "tsconfig.dev.json"],
65    sourceType: "module",
66  },
67  ignorePatterns: [
68    "/lib/**/*", // Ignore built files.
69  ],
70  plugins: [
71    "@typescript-eslint",
72    "import",
73  ],
74  rules: {
75    "quotes": ["error", "double"],
76    "import/no-unresolved": 0,
77  },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83  functions.logger.info("Hello logs!", {structuredData: true});
84  response.send("Hello from Firebase!");
85});
86 {
87   "eslint.workingDirectories": [
88       "src" // and "functions"
89   ]
90 }
91"compilerOptions": {
92    ...
93    "noUnusedLocals": false,
94    ...
95  }
96

However, the strange thing is that after successfully deploying functions to Firebase Cloud Fuctions using this setting and afterwards reverting the changes, the redeployment doesn´t show any error and is successful as well.

Update

After retrying it a few time I must admit that the solution is different. Turns out that the related stackoverflow thread did help, in particular @cherryblossom solution. Setting tsconfigRootDir: __dirname in the .eslintrc.js file and restarting the VSCode solves the problem.

.eslintrc.js:

1npm      --version 8.1.3
2tsc      --version 4.4.4
3node     --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE   package: undefined,
15npm WARN EBADENGINE   required: { node: '14' },
16npm WARN EBADENGINE   current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30  1:13  warning  'functions' is defined but never used  @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34  "compilerOptions": {
35    "module": "commonjs",
36    "noImplicitReturns": true,
37    "noUnusedLocals": true,
38    "outDir": "lib",
39    "sourceMap": true,
40    "strict": true,
41    "target": "es2017"
42  },
43  "compileOnSave": true,
44  "include": [
45    "src"
46  ]
47}
48module.exports = {
49  root: true,
50  env: {
51    es6: true,
52    node: true,
53  },
54  extends: [
55    "eslint:recommended",
56    "plugin:import/errors",
57    "plugin:import/warnings",
58    "plugin:import/typescript",
59    "google",
60    "plugin:@typescript-eslint/recommended",
61  ],
62  parser: "@typescript-eslint/parser",
63  parserOptions: {
64    project: ["tsconfig.json", "tsconfig.dev.json"],
65    sourceType: "module",
66  },
67  ignorePatterns: [
68    "/lib/**/*", // Ignore built files.
69  ],
70  plugins: [
71    "@typescript-eslint",
72    "import",
73  ],
74  rules: {
75    "quotes": ["error", "double"],
76    "import/no-unresolved": 0,
77  },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83  functions.logger.info("Hello logs!", {structuredData: true});
84  response.send("Hello from Firebase!");
85});
86 {
87   "eslint.workingDirectories": [
88       "src" // and "functions"
89   ]
90 }
91"compilerOptions": {
92    ...
93    "noUnusedLocals": false,
94    ...
95  }
96module.exports = {
97  // ...
98  parserOptions: {
99    ...
100    tsconfigRootDir: __dirname,
101    ...
102  },
103  // ...
104}
105

Source https://stackoverflow.com/questions/69897000

QUESTION

firebase function with realtime database error

Asked 2021-Nov-08 at 12:28

I am new to firebase function and trying to use firebase function with Realtime database (Emulator suite).But when i try to set the value in firebase using the firebase function,it gives an error and doesn't set the value in database.

Error:

117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
17

firebase function code :

117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp(); 
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24    response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28    var tex = request.query.text;
29    var textLength = tex.length;
30    console.log(textLength);
31    database.ref().child('test').set("op");
32    response.send(String(textLength));
33});
34

dependencies :

117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp(); 
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24    response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28    var tex = request.query.text;
29    var textLength = tex.length;
30    console.log(textLength);
31    database.ref().child('test').set("op");
32    response.send(String(textLength));
33});
34{
35  "name": "functions",
36  "description": "Cloud Functions for Firebase",
37  "scripts": {
38    "serve": "firebase emulators:start --only functions",
39    "shell": "firebase functions:shell",
40    "start": "npm run shell",
41    "deploy": "firebase deploy --only functions",
42    "logs": "firebase functions:log"
43  },
44  "engines": {
45    "node": "14"
46  },
47  "main": "index.js",
48  "dependencies": {
49    "firebase-admin": "^9.8.0",
50    "firebase-functions": "^3.14.1",
51    "@firebase/database-compat": "0.1.2" 
52  },
53  "devDependencies": {
54    "firebase-functions-test": "^0.2.0"
55  },
56  "private": true
57}
58

npm installed packages

117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp(); 
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24    response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28    var tex = request.query.text;
29    var textLength = tex.length;
30    console.log(textLength);
31    database.ref().child('test').set("op");
32    response.send(String(textLength));
33});
34{
35  "name": "functions",
36  "description": "Cloud Functions for Firebase",
37  "scripts": {
38    "serve": "firebase emulators:start --only functions",
39    "shell": "firebase functions:shell",
40    "start": "npm run shell",
41    "deploy": "firebase deploy --only functions",
42    "logs": "firebase functions:log"
43  },
44  "engines": {
45    "node": "14"
46  },
47  "main": "index.js",
48  "dependencies": {
49    "firebase-admin": "^9.8.0",
50    "firebase-functions": "^3.14.1",
51    "@firebase/database-compat": "0.1.2" 
52  },
53  "devDependencies": {
54    "firebase-functions-test": "^0.2.0"
55  },
56  "private": true
57}
58+-- @firebase/app-compat@0.1.7
59| +-- @firebase/app@0.7.6
60| | +-- @firebase/component@0.5.8
61| | | +-- @firebase/util@1.4.1 deduped
62| | | `-- tslib@2.3.1 deduped
63| | +-- @firebase/logger@0.3.1
64| | | `-- tslib@2.3.1 deduped
65| | +-- @firebase/util@1.4.1
66| | | `-- tslib@2.3.1 deduped
67| | `-- tslib@2.3.1 deduped
68| +-- @firebase/component@0.5.8
69| | +-- @firebase/util@1.4.1 deduped
70| | `-- tslib@2.3.1 deduped
71| +-- @firebase/logger@0.3.1
72| | `-- tslib@2.3.1 deduped
73| +-- @firebase/util@1.4.1
74| | `-- tslib@2.3.1 deduped
75| `-- tslib@2.3.1
76+-- @firebase/database-compat@0.1.2
77| +-- @firebase/component@0.5.7
78| | +-- @firebase/util@1.4.0 deduped
79| | `-- tslib@2.3.1 deduped
80| +-- @firebase/database@0.12.2
81| | +-- @firebase/auth-interop-types@0.1.6
82| | +-- @firebase/component@0.5.7 deduped
83| | +-- @firebase/logger@0.3.0 deduped
84| | +-- @firebase/util@1.4.0 deduped
85| | +-- faye-websocket@0.11.4
86| | | `-- websocket-driver@0.7.4
87| | |   +-- http-parser-js@0.5.3
88| | |   +-- safe-buffer@5.2.1 deduped
89| | |   `-- websocket-extensions@0.1.4
90| | `-- tslib@2.3.1 deduped
91| +-- @firebase/database-types@0.9.1
92| | +-- @firebase/app-types@0.7.0
93| | `-- @firebase/util@1.4.0 deduped
94| +-- @firebase/logger@0.3.0
95| | `-- tslib@2.3.1 deduped
96| +-- @firebase/util@1.4.0
97| | `-- tslib@2.3.1 deduped
98| `-- tslib@2.3.1 deduped
99`-- firebase-admin@10.0.0
100  +-- @firebase/database-compat@0.1.2 deduped
101  +-- @firebase/database-types@0.7.3
102  | `-- @firebase/app-types@0.6.3
103  +-- @google-cloud/firestore@4.15.1
104  | +-- fast-deep-equal@3.1.3
105  | +-- functional-red-black-tree@1.0.1
106  | +-- google-gax@2.28.0
107

ANSWER

Answered 2021-Nov-05 at 13:59

I'm unsure as to the cause of that log message, but I do see that you are returning a response from your function before it completes all of its work. In a deployed function, as soon as the function returns, all further actions should be treated as if they will never be executed as documented here. An "inactive" function might be terminated at any time, is severely throttled and any network calls you make (like setting data in the RTDB) may never be executed.

I know you are new to this, but its a good habit to get into now: don't assume the person calling your function is you. Check for problems like missing query parameters and dodgy data before you blindly action something. The Admin SDK bypasses your database's security rules and if you are not careful a malicious user can cause some damage (e.g. a user that updates /users/$theirUid/roles/admin to true).

117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z]  @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets.  Trying anyway. 
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp(); 
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24    response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28    var tex = request.query.text;
29    var textLength = tex.length;
30    console.log(textLength);
31    database.ref().child('test').set("op");
32    response.send(String(textLength));
33});
34{
35  "name": "functions",
36  "description": "Cloud Functions for Firebase",
37  "scripts": {
38    "serve": "firebase emulators:start --only functions",
39    "shell": "firebase functions:shell",
40    "start": "npm run shell",
41    "deploy": "firebase deploy --only functions",
42    "logs": "firebase functions:log"
43  },
44  "engines": {
45    "node": "14"
46  },
47  "main": "index.js",
48  "dependencies": {
49    "firebase-admin": "^9.8.0",
50    "firebase-functions": "^3.14.1",
51    "@firebase/database-compat": "0.1.2" 
52  },
53  "devDependencies": {
54    "firebase-functions-test": "^0.2.0"
55  },
56  "private": true
57}
58+-- @firebase/app-compat@0.1.7
59| +-- @firebase/app@0.7.6
60| | +-- @firebase/component@0.5.8
61| | | +-- @firebase/util@1.4.1 deduped
62| | | `-- tslib@2.3.1 deduped
63| | +-- @firebase/logger@0.3.1
64| | | `-- tslib@2.3.1 deduped
65| | +-- @firebase/util@1.4.1
66| | | `-- tslib@2.3.1 deduped
67| | `-- tslib@2.3.1 deduped
68| +-- @firebase/component@0.5.8
69| | +-- @firebase/util@1.4.1 deduped
70| | `-- tslib@2.3.1 deduped
71| +-- @firebase/logger@0.3.1
72| | `-- tslib@2.3.1 deduped
73| +-- @firebase/util@1.4.1
74| | `-- tslib@2.3.1 deduped
75| `-- tslib@2.3.1
76+-- @firebase/database-compat@0.1.2
77| +-- @firebase/component@0.5.7
78| | +-- @firebase/util@1.4.0 deduped
79| | `-- tslib@2.3.1 deduped
80| +-- @firebase/database@0.12.2
81| | +-- @firebase/auth-interop-types@0.1.6
82| | +-- @firebase/component@0.5.7 deduped
83| | +-- @firebase/logger@0.3.0 deduped
84| | +-- @firebase/util@1.4.0 deduped
85| | +-- faye-websocket@0.11.4
86| | | `-- websocket-driver@0.7.4
87| | |   +-- http-parser-js@0.5.3
88| | |   +-- safe-buffer@5.2.1 deduped
89| | |   `-- websocket-extensions@0.1.4
90| | `-- tslib@2.3.1 deduped
91| +-- @firebase/database-types@0.9.1
92| | +-- @firebase/app-types@0.7.0
93| | `-- @firebase/util@1.4.0 deduped
94| +-- @firebase/logger@0.3.0
95| | `-- tslib@2.3.1 deduped
96| +-- @firebase/util@1.4.0
97| | `-- tslib@2.3.1 deduped
98| `-- tslib@2.3.1 deduped
99`-- firebase-admin@10.0.0
100  +-- @firebase/database-compat@0.1.2 deduped
101  +-- @firebase/database-types@0.7.3
102  | `-- @firebase/app-types@0.6.3
103  +-- @google-cloud/firestore@4.15.1
104  | +-- fast-deep-equal@3.1.3
105  | +-- functional-red-black-tree@1.0.1
106  | +-- google-gax@2.28.0
107exports.textToLength = functions.https.onRequest((request, response) => {
108    const { text: queryText } = request.query;
109    
110    if (!queryText)
111      return response.status(400).send("Missing ?text");
112    if (typeof queryText !== 'string')
113      return response.status(400).send("Bad value for ?text");
114
115    const textLength = queryText.length;
116    
117    database.ref()
118      .child('test')
119      .set("op")
120      .then(() => {
121        console.log(`Successfully updated /test/op to ${textLength}`);
122        response.send(String(textLength))
123      })
124      .catch((err) => {
125        console.error(`Failed to update /test/op to ${textLength}: `, error);
126        response.status(500).send(`Failed to update database to ${textLength}`)
127      });
128    ;
129});
130

Note: When deploying HTTPS Request functions, don't forget to handle CORS.

Source https://stackoverflow.com/questions/69853038

QUESTION

Firebase Cloud Functions min-instances setting seems to be ignored

Asked 2021-Oct-30 at 06:35

Firebase has announced in September 2021 that it is possible now to configure its cloud function autoscaling in a way, so that a certain number of instances will always be running (https://firebase.google.com/docs/functions/manage-functions#min-max-instances).

I have tried to set this up, but I can not get it to work: At first I have set the number of minimum instances in Google Cloud Console: Cloud Console Screenshot After doing this I expected that one instance for that cloud function would run at any time. The metrics of that function indicate that it instances were still scaled down to 0: Cloud functions "Active Instances Metric"

So to me it looks a bit as if my setting is ignored here. Am I missing anything? Google Cloud Console shows me that the number of minimum instances has been set to 1 so it seems to know about it but to ignore it. Is this feature only available in certain regions?

I have also tried to set the number of minimum instances using the Firebase SDK for Cloud Functions (https://www.npmjs.com/package/firebase-functions). This gave me the same result, my setting is still ignored.

ANSWER

Answered 2021-Oct-30 at 06:35

According to the Documentation, the Active Instances metrics shows the number of instances that are currently handling the request.

As stated in the Documentation :

Cloud Functions scales by creating new instances of your function. Each of these instances can handle only one request at a time, so large spikes in request volume often causes longer wait times as new instances are created to handle the demand.

Because functions are stateless, your function sometimes initializes the execution environment from scratch, which is called a cold start. Cold starts can take significant amounts of time to complete, so we recommend setting a minimum number of Cloud Functions instances if your application is latency-sensitive.

You can also refer to the Stackoverflow thread where it has been mentioned that

Setting up minInstances does not mean that there will always be that much number of Active Instances. Minimum instances are kept running idle (without CPU > allocated), so are not counted in Active Instances.

Source https://stackoverflow.com/questions/69717844

QUESTION

Receiving inconsistent data in Go api

Asked 2021-Oct-23 at 12:31

I was learning the Go language and tested Google Cloud Functions with go + Google Firestore as the database.

While I was testing the response I got inconsistent responses.

I have used the json Marshaller to convert Firebase data to Json object to return from the API, this API is hosted in the Google Cloud Functions.

1// Package p contains an HTTP Cloud Function.
2package p
3
4import (
5    "context"
6    "fmt"
7    "log"
8    "net/http"
9
10    s "strings"
11
12    "encoding/json"
13
14    "cloud.google.com/go/firestore"
15    "google.golang.org/api/iterator"
16)
17
18
19func HelloWorld(w http.ResponseWriter, r *http.Request) {
20    path := s.Replace(r.URL.Path, "/", "", -1)
21
22    ctx := context.Background()
23    client := createClient(ctx)
24
25    iter := client.Collection("profile").Where("publicUrl", "==", path).Documents(ctx)
26    for {
27        doc, err := iter.Next()
28        if err == iterator.Done {
29            break
30        }
31        var publicDTO PublicDTO
32        var Profile Profile
33        doc.DataTo(&Profile)
34
35        publicDTO.Profile = Profile
36
37        b, err := json.Marshal(publicDTO)
38        if err != nil {
39            fmt.Println(err)
40            return
41        }
42
43        w.Header().Set("Content-Type", "application/json")
44        w.Write(b)
45    }
46
47}
48
49func createClient(ctx context.Context) *firestore.Client {
50    projectID := "projectId"
51
52    client, err := firestore.NewClient(ctx, projectID)
53    if err != nil {
54        log.Fatalf("Failed to create client: %v", err)
55    }
56    return client
57}
58
59type PublicDTO struct {
60    Profile       Profile     `json:"profile"`
61}
62
63type Profile struct {
64    Id           string            `json:"id"`
65    FirstName    string            `json:"firstName"`
66    LastName     string            `json:"lastName"`
67    FullName     string            `json:"fullName"`
68    Email        string            `json:"email"`
69    ImageUrl     string            `json:"imageUrl"`
70    CoverPic     string            `json:"coverPic"`
71    Experience   int               `json:"experience"`
72    PhoneNumber  string            `json:"phoneNumber"`
73    Description  string            `json:"description"`
74    Address      string            `json:"address"`
75    State        string            `json:"state"`
76    Country      string            `json:"country"`
77    Dob          map[string]string `json:"dob"`
78    Website      string            `json:"website"`
79    Reputation   int               `json:"reputation"`
80    MemberFrom   map[string]int    `json:"memberFrom"`
81    Title        string            `json:"title"`
82    Organization string            `json:"organization"`
83    Status       string            `json:"status"`
84    Setup        int               `json:"setup"`
85    Social       map[string]string `json:"social"`
86    PublicUrl    string            `json:"publicUrl"`
87    Language     []string          `json:"language"`
88    Interests    []string          `json:"interests"`
89}
90
91

but each time the response i'm getting is inconsistent, some of the values are missing.

ANSWER

Answered 2021-Oct-23 at 12:31

The solution i got after marshal and unmarshal, it works as expected.

1// Package p contains an HTTP Cloud Function.
2package p
3
4import (
5    "context"
6    "fmt"
7    "log"
8    "net/http"
9
10    s "strings"
11
12    "encoding/json"
13
14    "cloud.google.com/go/firestore"
15    "google.golang.org/api/iterator"
16)
17
18
19func HelloWorld(w http.ResponseWriter, r *http.Request) {
20    path := s.Replace(r.URL.Path, "/", "", -1)
21
22    ctx := context.Background()
23    client := createClient(ctx)
24
25    iter := client.Collection("profile").Where("publicUrl", "==", path).Documents(ctx)
26    for {
27        doc, err := iter.Next()
28        if err == iterator.Done {
29            break
30        }
31        var publicDTO PublicDTO
32        var Profile Profile
33        doc.DataTo(&Profile)
34
35        publicDTO.Profile = Profile
36
37        b, err := json.Marshal(publicDTO)
38        if err != nil {
39            fmt.Println(err)
40            return
41        }
42
43        w.Header().Set("Content-Type", "application/json")
44        w.Write(b)
45    }
46
47}
48
49func createClient(ctx context.Context) *firestore.Client {
50    projectID := "projectId"
51
52    client, err := firestore.NewClient(ctx, projectID)
53    if err != nil {
54        log.Fatalf("Failed to create client: %v", err)
55    }
56    return client
57}
58
59type PublicDTO struct {
60    Profile       Profile     `json:"profile"`
61}
62
63type Profile struct {
64    Id           string            `json:"id"`
65    FirstName    string            `json:"firstName"`
66    LastName     string            `json:"lastName"`
67    FullName     string            `json:"fullName"`
68    Email        string            `json:"email"`
69    ImageUrl     string            `json:"imageUrl"`
70    CoverPic     string            `json:"coverPic"`
71    Experience   int               `json:"experience"`
72    PhoneNumber  string            `json:"phoneNumber"`
73    Description  string            `json:"description"`
74    Address      string            `json:"address"`
75    State        string            `json:"state"`
76    Country      string            `json:"country"`
77    Dob          map[string]string `json:"dob"`
78    Website      string            `json:"website"`
79    Reputation   int               `json:"reputation"`
80    MemberFrom   map[string]int    `json:"memberFrom"`
81    Title        string            `json:"title"`
82    Organization string            `json:"organization"`
83    Status       string            `json:"status"`
84    Setup        int               `json:"setup"`
85    Social       map[string]string `json:"social"`
86    PublicUrl    string            `json:"publicUrl"`
87    Language     []string          `json:"language"`
88    Interests    []string          `json:"interests"`
89}
90
91package p
92
93import (
94    "context"
95    "encoding/json"
96    "fmt"
97    "log"
98    "net/http"
99    s "strings"
100    "time"
101
102    "cloud.google.com/go/firestore"
103    "google.golang.org/api/iterator"
104)
105
106var ctx context.Context
107var client *firestore.Client
108
109func PublicApi(w http.ResponseWriter, r *http.Request) {
110    path := s.Replace(r.URL.Path, "/", "", -1)
111    w.Header().Set("Content-Type", "application/json")
112    w.Header().Set("Access-Control-Allow-Origin", "*")
113    w.Header().Set("Access-Control-Allow-Headers", "*")
114
115    newFsConfigBytes, _ := json.Marshal(getPublicDTO(path))
116    w.Write(newFsConfigBytes)
117}
118
119func getPublicDTO(path string) (publicDTO publicDTO) {
120    ctx = context.Background()
121    client = createClient()
122    profile, id := getProfiles(path)
123    publicDTO.Profile = profile
124    publicDTO.MilestoneDTOS = getMilestone(id)
125    return
126}
127
128func getProfiles(link string) (profile, string) {
129    var retVal profile
130    var id string
131    iter := client.Collection("profile").Where("publicUrl", "==", link).Documents(ctx)
132    for {
133        doc, err := iter.Next()
134        if err == iterator.Done {
135            break
136        }
137        id = doc.Ref.ID
138        b, _ := json.Marshal(doc.Data())
139        json.Unmarshal(b, &retVal)
140    }
141    return retVal, id
142}
143
144func getMilestone(id string) []milestoneDTOS {
145    var retVal []milestoneDTOS
146    iter := client.Collection("milestone").Where("userId", "==", id).Documents(ctx)
147
148    for {
149        var milestoneDTO milestoneDTOS
150        doc, err := iter.Next()
151        if err == iterator.Done {
152            break
153        }
154        b, _ := json.Marshal(doc.Data())
155        err = json.Unmarshal(b, &milestoneDTO)
156        if err != nil {
157            fmt.Println(err)
158        }
159        retVal = append(retVal, milestoneDTO)
160    }
161    return retVal
162}
163
164func createClient() *firestore.Client {
165    projectID := "live-hiveboard"
166
167    client, err := firestore.NewClient(ctx, projectID)
168    if err != nil {
169        log.Fatalf("Failed to create client: %v", err)
170    }
171    return client
172}
173
174type profile struct {
175    Address          string    `json:"address"`
176    City             string    `json:"city"`
177    Country          string    `json:"country"`
178    CoverPic         string    `json:"coverPic"`
179    CreatedBy        string    `json:"createdBy"`
180    CreatedDate      int       `json:"createdDate"`
181    Description      string    `json:"description"`
182    Dob              int64     `json:"dob"`
183    Email            string    `json:"email"`
184    Enabled          bool      `json:"enabled"`
185    Experience       int       `json:"experience"`
186    FirstName        string    `json:"firstName"`
187    FullName         string    `json:"fullName"`
188    FullNameNoSpace  string    `json:"fullNameNoSpace"`
189    ImageURL         string    `json:"imageUrl"`
190    Interests        []string  `json:"interests"`
191    IsEnabled        bool      `json:"isEnabled"`
192    Language         string    `json:"language"`
193    LastModifiedDate int       `json:"lastModifiedDate"`
194    LastName         string    `json:"lastName"`
195    LatLng           string    `json:"latLng"`
196    MemberFrom       time.Time `json:"memberFrom"`
197    ObjectID         string    `json:"objectID"`
198    Organization     string    `json:"organization"`
199    PhoneNumber      string    `json:"phoneNumber"`
200    PlanID           string    `json:"planId"`
201    PublicURL        string    `json:"publicUrl"`
202    Reputation       int       `json:"reputation"`
203    Setup            int       `json:"setup"`
204    Social           string    `json:"social"`
205    State            string    `json:"state"`
206    Status           string    `json:"status"`
207    Title            string    `json:"title"`
208    Website          string    `json:"website"`
209}
210
211type milestoneDTOS struct {
212    Category          string    `json:"category"`
213    CreatedBy         string    `json:"createdBy"`
214    CreatedDate       int       `json:"createdDate"`
215    Description       string    `json:"description"`
216    Enabled           bool      `json:"enabled"`
217    EndDate           time.Time `json:"endDate"`
218    IsCurrentPosition bool      `json:"isCurrentPosition"`
219    IsEnabled         bool      `json:"isEnabled"`
220    LastModifiedBy    time.Time `json:"lastModifiedBy"`
221    LastModifiedDate  int       `json:"lastModifiedDate"`
222    ObjectID          string    `json:"objectID"`
223    Organization      string    `json:"organization"`
224    PictureURL        string    `json:"pictureURL"`
225    Profile           string    `json:"profile"`
226    Score             float64   `json:"score"`
227    StartDate         time.Time `json:"startDate"`
228    Tags              []string  `json:"tags"`
229    Title             string    `json:"title"`
230    URL               string    `json:"url"`
231    UserID            string    `json:"userId"`
232}
233
234type publicDTO struct {
235    Profile       profile         `json:"profile"`
236    MilestoneDTOS []milestoneDTOS `json:"milestoneDTOS"`
237}
238
239

Source https://stackoverflow.com/questions/69072757

QUESTION

Get the client_id of the IAM proxy on GCP Cloud composer

Asked 2021-Oct-15 at 15:02

I'm trying to trigger Airflow DAG inside of a composer environment with cloud functions. In order to do that I need to get the client id as described here. I've tried with curl command but it doesn't return any value. With a python script I keep getting this error:

1Traceback (most recent call last):
2  File "get_client_id.py", line 55, in <module>
3    get_client_id(
4  File "get_client_id.py", line 40, in get_client_id
5    print(query_string['client_id'][0])
6KeyError: 'client_id'
7

Python code:

1Traceback (most recent call last):
2  File "get_client_id.py", line 55, in <module>
3    get_client_id(
4  File "get_client_id.py", line 40, in get_client_id
5    print(query_string['client_id'][0])
6KeyError: 'client_id'
7import google.auth
8import google.auth.transport.requests
9import requests
10import six.moves.urllib.parse
11
12# Authenticate with Google Cloud.
13# See: https://cloud.google.com/docs/authentication/getting-started
14credentials, _ = google.auth.default(
15    scopes=['https://www.googleapis.com/auth/cloud-platform'])
16authed_session = google.auth.transport.requests.AuthorizedSession(
17    credentials)
18
19project_id = 'my-project'
20location = 'my-region'
21composer_environment = 'my-env'
22
23environment_url = (
24    'https://composer.googleapis.com/v1beta1/projects/{}/locations/{}'
25    '/environments/{}').format(project_id, location, composer_environment)
26composer_response = authed_session.request('GET', environment_url)
27environment_data = composer_response.json()
28airflow_uri = environment_data['config']['airflowUri']
29
30
31
32# The Composer environment response does not include the IAP client ID.
33# Make a second, unauthenticated HTTP request to the web server to get the
34# redirect URI.
35redirect_response = requests.get(airflow_uri, allow_redirects=False)
36redirect_location = redirect_response.headers['location']
37
38# Extract the client_id query parameter from the redirect.
39parsed = six.moves.urllib.parse.urlparse(redirect_location)
40query_string = six.moves.urllib.parse.parse_qs(parsed.query)
41print(query_string['client_id'][0])
42

cURL command:

1Traceback (most recent call last):
2  File "get_client_id.py", line 55, in <module>
3    get_client_id(
4  File "get_client_id.py", line 40, in get_client_id
5    print(query_string['client_id'][0])
6KeyError: 'client_id'
7import google.auth
8import google.auth.transport.requests
9import requests
10import six.moves.urllib.parse
11
12# Authenticate with Google Cloud.
13# See: https://cloud.google.com/docs/authentication/getting-started
14credentials, _ = google.auth.default(
15    scopes=['https://www.googleapis.com/auth/cloud-platform'])
16authed_session = google.auth.transport.requests.AuthorizedSession(
17    credentials)
18
19project_id = 'my-project'
20location = 'my-region'
21composer_environment = 'my-env'
22
23environment_url = (
24    'https://composer.googleapis.com/v1beta1/projects/{}/locations/{}'
25    '/environments/{}').format(project_id, location, composer_environment)
26composer_response = authed_session.request('GET', environment_url)
27environment_data = composer_response.json()
28airflow_uri = environment_data['config']['airflowUri']
29
30
31
32# The Composer environment response does not include the IAP client ID.
33# Make a second, unauthenticated HTTP request to the web server to get the
34# redirect URI.
35redirect_response = requests.get(airflow_uri, allow_redirects=False)
36redirect_location = redirect_response.headers['location']
37
38# Extract the client_id query parameter from the redirect.
39parsed = six.moves.urllib.parse.urlparse(redirect_location)
40query_string = six.moves.urllib.parse.parse_qs(parsed.query)
41print(query_string['client_id'][0])
42curl -v my_airflow_url 2>&1 >/dev/null | grep -o "client_id\=[A-Za-z0-9-]*\.apps\.googleusercontent\.com"
43

Does anybody have an idea how to get the Get the client_id of the IAM proxy?

ANSWER

Answered 2021-Sep-28 at 13:00

Posting this Community Wiki for better visibility.

As mentioned in the comment section by @LEC this configuration is compatible with Cloud Composer V1 which can be found in GCP Documentation Triggering DAGs with Cloud Functions.

At the moment there can be found two tabs Cloud Composer 1 Guides and Cloud Composer 2 Guides. Under Cloud Composer 1 is code used by the OP, but if you will check Cloud Composer 2 under Manage DAGs > Triggering DAGs with Cloud Functions you will get information that there is not proper documentation yet.

This documentation page for Cloud Composer 2 is not yet available. Please use the page for Cloud Composer 1.

As solution, please use Cloud Composer V1.

Source https://stackoverflow.com/questions/69269929

Community Discussions contain sources that include Stack Exchange Network

Tutorials and Learning Resources in Cloud Functions

Tutorials and Learning Resources are not available at this moment for Cloud Functions

Share this Page

share link

Get latest updates on Cloud Functions