Popular New Releases in Cloud Functions
serverless-offline
v8.6.0
serverless-next.js
v3.8.0-alpha.0
aws-lambda-rust-runtime
lambda_runtime release v0.5.1
Klayers
Multi Python Builds
azure-functions-core-tools
3.0.4484
Popular Libraries in Cloud Functions
by firebase javascript
10874 Apache-2.0
Collection of sample apps showcasing popular use cases using Cloud Functions for Firebase
by awsdocs java
5156 Apache-2.0
Welcome to the AWS Code Examples Repository. This repo contains code examples used in the AWS documentation, AWS SDK Developer Guides, and more. For more information, see the Readme.rst file below.
by dherault javascript
4486 MIT
Emulate AWS λ and API Gateway locally when developing your Serverless project
by serverless-nextjs typescript
3445 MIT
⚡ Deploy your Next.js apps on AWS Lambda@Edge via Serverless Components
by alixaxel typescript
2267 MIT
Chromium Binary for AWS Lambda and Google Cloud Functions
by awslabs rust
2029 Apache-2.0
A Rust runtime for AWS Lambda
by AutoSpotting go
1937 NOASSERTION
Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups. Installs in minutes using CloudFormation or Terraform. Convenient to deploy at scale using StackSets. Uses tagging to avoid launch configuration changes. Automated spot termination handling. Reliable fallback to on-demand instances.
by goadapp go
1857 MIT
Goad is an AWS Lambda powered, highly distributed, load testing tool
by awsdocs java
1588 NOASSERTION
The AWS Lambda Developer Guide
Trending New libraries in Cloud Functions
by milliHQ typescript
796 Apache-2.0
Terraform module for building and deploying Next.js apps to AWS. Supports SSR (Lambda), Static (S3) and API (Lambda) pages.
by aws go
481 Apache-2.0
by nelhage go
387 Apache-2.0
by dealmore typescript
263 Apache-2.0
Terraform module for building and deploying Next.js apps to AWS. Supports SSR (Lambda), Static (S3) and API (Lambda) pages.
by aws-samples go
223 NOASSERTION
A collection of sample extensions to help you get started with AWS Lambda Extensions
by EtherDream javascript
199
bundle web files into a single image
by Azure-Samples csharp
186 MIT
An orchestration-based saga implementation reference in a serverless architecture
by Albert-Gao typescript
184 MIT
A 7KB and 0 dependencies AWS Lambda library which supports middleware and easy debug.
by Azure csharp
181 MIT
Azure Functions out-of-process .NET language worker
Top Authors in Cloud Functions
1
75 Libraries
3122
2
23 Libraries
4301
3
14 Libraries
796
4
14 Libraries
351
5
13 Libraries
2633
6
10 Libraries
170
7
10 Libraries
155
8
10 Libraries
2941
9
9 Libraries
18
10
7 Libraries
429
1
75 Libraries
3122
2
23 Libraries
4301
3
14 Libraries
796
4
14 Libraries
351
5
13 Libraries
2633
6
10 Libraries
170
7
10 Libraries
155
8
10 Libraries
2941
9
9 Libraries
18
10
7 Libraries
429
Trending Kits in Cloud Functions
The AWS SDK for JavaScript offers access to the AWS Lambda service through APIs, allowing developers to create, manage, and invoke Lambda functions from within their JavaScript applications. JavaScript AWS Lambda Libraries are a robust set of software components to develop and deploy applications. They enable developers to use the same code for client and server-side applications and provide a wide range of features and functionalities.
Lambda functions are serverless functions that are triggered by events. These events can be anything from a new data record being added to a database to a user clicking a button in an application. When an event is triggered, the Lambda function is executed. Lambda functions can be written in Node.js, Python, JavaScript, or C# and perform various tasks such as image processing, data analysis, or even machine learning. AWS Lambda is an Amazon Web Services (AWS) computing service that empowers you to execute the code without provisioning or managing servers. Without worrying about server provisioning or management, this "pay-per-use" cloud computing service enables developers to quickly and easily design and manage apps that respond to events and run code responding to those events. Applications of any size may be built, tested, deployed, and scaled quickly and efficiently while only incurring costs for the compute time required.
The AWS Lambda Library contains a set of ready-made libraries that developers can use to develop their Lambda functions. These libraries are pre-packaged and contain functions, classes, and other tools designed to make it easier for developers to develop their Lambda functions. Some commonly used JavaScript AWS libraries that facilitate developers are shim, lambda-image, slack-commands, nodejs-lambda-middleware, synthetic-tests-pings, template-mailer-aws-lambda-client, lambda-oauth2, aws-lambda-toolkit, serverless-response, swf-hook.
Check out the list below to find more popular Node JS AWS libraries for your app development:
Trending Discussions on Cloud Functions
Automatic delete Cloud Logging logs
Firebase function failing to deploy
Implementation of Distributed-Counter-Extension for local emulator
Build a container image from inside a cloud function
Error when import firebase-functions-test when testing with mocha
Parsing error: Cannot read file '\tsconfig.json' eslint after following Firebase Cloud Functions initialization instructions
firebase function with realtime database error
Firebase Cloud Functions min-instances setting seems to be ignored
Receiving inconsistent data in Go api
Get the client_id of the IAM proxy on GCP Cloud composer
QUESTION
Automatic delete Cloud Logging logs
Asked 2022-Mar-23 at 16:41I would like to know if there is a way to periodically delete logs from inside Cloud Logging
.
I have setup Firebase with Cloud Functions
and i have an automatic Cloud Logging
logs injection done for each function call.
I don't want especially to stop sending logs to Cloud Logging
, but i would like to be able to manage my costs by deleting older logs.
ANSWER
Answered 2022-Mar-23 at 16:41You can set a retention policy on your Cloud Logging bucket to match with your requirements, which can auto-delete logs after between 1 day and 10 years.
QUESTION
Firebase function failing to deploy
Asked 2022-Feb-17 at 15:31I'm trying to create a Firebase Function but I'm running into a deploy error, even when deploying the default helloworld
function.
The firebase-debug.log file mentions this:
Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld.
I have been trying to debug and so far have not been able to solve it...
firebase-debug.log
1[info] Functions deploy had errors with the following functions:
2 helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i functions: cleaning up build files...
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10 at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11 at processTicksAndRejections (internal/process/task_queues.js:93:5)
12 at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13 at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14 at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error]
16[error] Error: There was an error deploying functions
17
index.js:
1[info] Functions deploy had errors with the following functions:
2 helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i functions: cleaning up build files...
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10 at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11 at processTicksAndRejections (internal/process/task_queues.js:93:5)
12 at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13 at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14 at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error]
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23 functions.logger.info("Hello logs!", {structuredData: true});
24 response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29
Package.json:
1[info] Functions deploy had errors with the following functions:
2 helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i functions: cleaning up build files...
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10 at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11 at processTicksAndRejections (internal/process/task_queues.js:93:5)
12 at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13 at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14 at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error]
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23 functions.logger.info("Hello logs!", {structuredData: true});
24 response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29{
30 "name": "functions",
31 "description": "Cloud Functions for Firebase",
32 "scripts": {
33 "lint": "eslint",
34 "serve": "firebase emulators:start --only functions",
35 "shell": "firebase functions:shell",
36 "start": "npm run shell",
37 "deploy": "firebase deploy --only functions",
38 "logs": "firebase functions:log"
39 },
40 "engines": {
41 "node": "14"
42 },
43 "main": "index.js",
44 "dependencies": {
45 "firebase-admin": "^9.8.0",
46 "firebase-functions": "^3.14.1",
47 "blurhash": "^1.1.4"
48 },
49 "devDependencies": {
50 "eslint": "^7.6.0",
51 "eslint-config-google": "^0.14.0",
52 "firebase-functions-test": "^0.2.0"
53 },
54 "private": true
55}
56
My version of node:
v14.16.0
Appreciate your help.
ANSWER
Answered 2022-Feb-06 at 14:36Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld.
The Firebase Function deployment failed because it cannot find the image built based on your function app. There might be a problem building in your app, it could be your dependencies or files.
I replicated your issue, received the same error and solved it. There's a problem with the package.json
file and package-lock.json
. If you just add(without installing) your dependency in package.json
you should delete or remove your package-lock.json
that will be found in function directory before you deploy it again using the deployment command:
1[info] Functions deploy had errors with the following functions:
2 helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i functions: cleaning up build files...
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10 at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11 at processTicksAndRejections (internal/process/task_queues.js:93:5)
12 at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13 at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14 at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error]
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23 functions.logger.info("Hello logs!", {structuredData: true});
24 response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29{
30 "name": "functions",
31 "description": "Cloud Functions for Firebase",
32 "scripts": {
33 "lint": "eslint",
34 "serve": "firebase emulators:start --only functions",
35 "shell": "firebase functions:shell",
36 "start": "npm run shell",
37 "deploy": "firebase deploy --only functions",
38 "logs": "firebase functions:log"
39 },
40 "engines": {
41 "node": "14"
42 },
43 "main": "index.js",
44 "dependencies": {
45 "firebase-admin": "^9.8.0",
46 "firebase-functions": "^3.14.1",
47 "blurhash": "^1.1.4"
48 },
49 "devDependencies": {
50 "eslint": "^7.6.0",
51 "eslint-config-google": "^0.14.0",
52 "firebase-functions-test": "^0.2.0"
53 },
54 "private": true
55}
56firebase deploy --only functions
57
or you can just install your dependency to make sure it will be added to your package.json
and package-lock.json
file, deploy again. For example:
1[info] Functions deploy had errors with the following functions:
2 helloWorld(us-central1)
3[debug] [2021-11-18T21:54:08.946Z] Missing URI for HTTPS function in printTriggerUrls. This shouldn't happen
4[info] i functions: cleaning up build files...
5[debug] [2021-11-18T21:54:08.948Z] >>> [apiv2][query] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list [none]
6[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][status] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list 200
7[debug] [2021-11-18T21:54:09.407Z] <<< [apiv2][body] GET https://us.gcr.io/v2/picci-e030e/gcf/us-central1/tags/list {"child":[],"manifest":{},"name":"picci-e030e/gcf/us-central1","tags":[]}
8[debug] [2021-11-18T21:54:09.407Z] Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld
9[debug] [2021-11-18T21:54:09.481Z] Error: Failed to create function helloWorld in region us-central1
10 at /usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:38:11
11 at processTicksAndRejections (internal/process/task_queues.js:93:5)
12 at async Fabricator.createV1Function (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:161:32)
13 at async Fabricator.createEndpoint (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:116:13)
14 at async handle (/usr/local/lib/node_modules/firebase-tools/lib/deploy/functions/release/fabricator.js:75:17)
15[error]
16[error] Error: There was an error deploying functions
17const functions = require("firebase-functions");
18
19// Create and Deploy Your First Cloud Functions
20// https://firebase.google.com/docs/functions/write-firebase-functions
21
22exports.helloWorld = functions.https.onRequest((request, response) => {
23 functions.logger.info("Hello logs!", {structuredData: true});
24 response.send("Hello from Firebase!");
25});
26
27// const getBlurhash = require("./get_blurhash");
28// exports.getBlurhash = getBlurhash.generateHash;
29{
30 "name": "functions",
31 "description": "Cloud Functions for Firebase",
32 "scripts": {
33 "lint": "eslint",
34 "serve": "firebase emulators:start --only functions",
35 "shell": "firebase functions:shell",
36 "start": "npm run shell",
37 "deploy": "firebase deploy --only functions",
38 "logs": "firebase functions:log"
39 },
40 "engines": {
41 "node": "14"
42 },
43 "main": "index.js",
44 "dependencies": {
45 "firebase-admin": "^9.8.0",
46 "firebase-functions": "^3.14.1",
47 "blurhash": "^1.1.4"
48 },
49 "devDependencies": {
50 "eslint": "^7.6.0",
51 "eslint-config-google": "^0.14.0",
52 "firebase-functions-test": "^0.2.0"
53 },
54 "private": true
55}
56firebase deploy --only functions
57npm install --save blurhash
58
QUESTION
Implementation of Distributed-Counter-Extension for local emulator
Asked 2022-Jan-24 at 17:55The firebase extension for a distributed counter can be directly installed for the cloud and works just fine. To develop new features for an app I need to do this on the emulator to not interrupt the running server.
As the firebase extensions simply are cloud Functions*, I thought about implementing the cloud function in my emulator by getting the source code from the extension itself. This worked fine for other extentions so far...
Error and Disfunction when implementingWhen implementing the javaScript version that i get the following error:
function ignored because the unknown emulator does not exist or is not running.
This problem can be fixed by rewriting the export line of the index.js
functions, but is wont provide the expected functionality of the extension anyhow:
1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2
to
1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3
Am i missing some implementaion or is an emulator missing? How can i implement this firestore extention?
There are some similar questions, but dont really talk about the implementation of the whole extension:
ANSWER
Answered 2022-Jan-24 at 17:55firebaser here
Firebase Extensions normally declare their triggers in the extension.yaml file, instead of in the code itself. Therefore, in order to emulate an extension in this way, you'd need to move the triggers over to the code.
For your specific example of the 'worker' function, the extension declares what document to listen to here, so we'll copy the document over to the code:
1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3exports.worker = functions.firestore.document(`${process.env.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {...});
4
You'll have to do the same for each other background triggered function in the extension - find the trigger in extension.yaml, and move it over to the code.
You'll also need to provide the param values that you normally configure the extension with as environment variables. To do this, you'll have to add a .env
file to your functions directory containing the environment variables you want to set. For the your case, that will look something like:
1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3exports.worker = functions.firestore.document(`${process.env.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {...});
4LOCATION=//Put your location here
5INTERNAL_STATE_PATH=//Put your path here
6SCHEDULE_FREQUENCY=1
7
Then, you'll need to enable the emulator to read this file:
1exports.worker = functions.handler.firestore.document.onWrite(async (change, context) => {...});
2exports.worker = functions.firestore.document("").onWrite(async (change, context) => {...});
3exports.worker = functions.firestore.document(`${process.env.INTERNAL_STATE_PATH}/workers/{workerId}`).onWrite(async (change, context) => {...});
4LOCATION=//Put your location here
5INTERNAL_STATE_PATH=//Put your path here
6SCHEDULE_FREQUENCY=1
7firebase --open-sesame dotenv
8
Then, you should be able to run the extension code in the emulator.
As you might be thinking, this is not a very easy process right now. We are currently working on adding native support for Extensions to the Firebase Emulator Suite. As soon as that is available, I'll make sure to come back here and update this answer with how to use it, and hopefully this will be much easier for you!
PS: If you are interested in potentially helping us test out this feature early, let me know and we'll reach out as soon as it is ready to test.
QUESTION
Build a container image from inside a cloud function
Asked 2021-Dec-22 at 00:59Context: I am training a very similar model per bigquery dataset in Google Vertex AI, but I want to have a custom training image for each existing dataset (in Google BigQuery). In that sense, I need to programatically build a custom Docker Image in the container registry on demand. My idea was to have a Google Cloud Function do it, being triggered by PubSub topic with information regarding which dataset I want to build the training container for. So naturally, the function will write the Dockerfile and pertinent scripts to a /tmp folder within Cloud Functions (the only writable place as per my knowledge). However, when I try to actually build the container within this script, apparently, it doesn't find the /tmp folder or its contents, even though they are there (checked with logging operations).
The troubling code so far:
1def build_container(dataset=str):
2
3 with open('container_template/Dockerfile','r') as f:
4 dockerfile = f.read()
5 dockerfile = dockerfile.replace('@dataset',dataset)
6 f.close()
7
8 os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10 with open('/tmp/Dockerfile','w') as docker_config:
11 docker_config.write(dockerfile)
12
13 docker_config.close()
14
15 shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17 build_client = cloudbuild_v1.CloudBuildClient()
18
19 build = cloudbuild_v1.Build()
20
21 build.steps = [{'name':'gcr.io/cloud-builders/docker',
22 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23 {'name':'gcr.io/cloud-builders/docker',
24 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
25
26 build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28 build_result = build_operation.result()
29
30 logger.info('Build Result: {}'.format(build_result.status))
31
When I check the cloud build logs I get: Step #0: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /tmp/Dockerfile: no such file or directory
ANSWER
Answered 2021-Dec-21 at 11:07I've locally tested building a container image using Cloud Build Client Python library. It turns out to have the same error even the Dockerfile
file is existing in current directory:
error:
Step #0: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /workspace/Dockerfile: no such file or directory
build steps:
1def build_container(dataset=str):
2
3 with open('container_template/Dockerfile','r') as f:
4 dockerfile = f.read()
5 dockerfile = dockerfile.replace('@dataset',dataset)
6 f.close()
7
8 os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10 with open('/tmp/Dockerfile','w') as docker_config:
11 docker_config.write(dockerfile)
12
13 docker_config.close()
14
15 shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17 build_client = cloudbuild_v1.CloudBuildClient()
18
19 build = cloudbuild_v1.Build()
20
21 build.steps = [{'name':'gcr.io/cloud-builders/docker',
22 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23 {'name':'gcr.io/cloud-builders/docker',
24 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
25
26 build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28 build_result = build_operation.result()
29
30 logger.info('Build Result: {}'.format(build_result.status))
31 build_client = cloudbuild_v1.CloudBuildClient()
32
33 build = cloudbuild_v1.Build()
34
35 build.steps = [{'name':'gcr.io/cloud-builders/docker',
36 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37 {'name':'gcr.io/cloud-builders/docker',
38 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
39
40 build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42 build_result = build_operation.result()
43
Since it uses the API method I followed this documentation. You will see the source
is present in API method. It is the missing key to move forward the problem. In StorageSource
, you must to specify the bucket
and object_
. You need to compress your source code and upload it in Cloud Storage bucket. For example:
- Run the following command to compress your source code:
1def build_container(dataset=str):
2
3 with open('container_template/Dockerfile','r') as f:
4 dockerfile = f.read()
5 dockerfile = dockerfile.replace('@dataset',dataset)
6 f.close()
7
8 os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10 with open('/tmp/Dockerfile','w') as docker_config:
11 docker_config.write(dockerfile)
12
13 docker_config.close()
14
15 shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17 build_client = cloudbuild_v1.CloudBuildClient()
18
19 build = cloudbuild_v1.Build()
20
21 build.steps = [{'name':'gcr.io/cloud-builders/docker',
22 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23 {'name':'gcr.io/cloud-builders/docker',
24 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
25
26 build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28 build_result = build_operation.result()
29
30 logger.info('Build Result: {}'.format(build_result.status))
31 build_client = cloudbuild_v1.CloudBuildClient()
32
33 build = cloudbuild_v1.Build()
34
35 build.steps = [{'name':'gcr.io/cloud-builders/docker',
36 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37 {'name':'gcr.io/cloud-builders/docker',
38 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
39
40 build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42 build_result = build_operation.result()
43tar -cvzf sourcecode.tar.gz .
44
- Upload to Cloud Storage bucket(you can use Cloud Build bucket):
1def build_container(dataset=str):
2
3 with open('container_template/Dockerfile','r') as f:
4 dockerfile = f.read()
5 dockerfile = dockerfile.replace('@dataset',dataset)
6 f.close()
7
8 os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10 with open('/tmp/Dockerfile','w') as docker_config:
11 docker_config.write(dockerfile)
12
13 docker_config.close()
14
15 shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17 build_client = cloudbuild_v1.CloudBuildClient()
18
19 build = cloudbuild_v1.Build()
20
21 build.steps = [{'name':'gcr.io/cloud-builders/docker',
22 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23 {'name':'gcr.io/cloud-builders/docker',
24 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
25
26 build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28 build_result = build_operation.result()
29
30 logger.info('Build Result: {}'.format(build_result.status))
31 build_client = cloudbuild_v1.CloudBuildClient()
32
33 build = cloudbuild_v1.Build()
34
35 build.steps = [{'name':'gcr.io/cloud-builders/docker',
36 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37 {'name':'gcr.io/cloud-builders/docker',
38 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
39
40 build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42 build_result = build_operation.result()
43tar -cvzf sourcecode.tar.gz .
44gsutil cp sourcecode.tar.gz gs://myproject_cloudbuild
45
- build.source:
1def build_container(dataset=str):
2
3 with open('container_template/Dockerfile','r') as f:
4 dockerfile = f.read()
5 dockerfile = dockerfile.replace('@dataset',dataset)
6 f.close()
7
8 os.makedirs(os.path.dirname('/tmp/script-location'), exist_ok=True)
9
10 with open('/tmp/Dockerfile','w') as docker_config:
11 docker_config.write(dockerfile)
12
13 docker_config.close()
14
15 shutil.copy('container_template/script-location/script.py','/tmp/script-location/script.py')
16
17 build_client = cloudbuild_v1.CloudBuildClient()
18
19 build = cloudbuild_v1.Build()
20
21 build.steps = [{'name':'gcr.io/cloud-builders/docker',
22 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','/tmp']},
23 {'name':'gcr.io/cloud-builders/docker',
24 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
25
26 build_operation = build_client.create_build(project_id=myprojectid,build=build)
27
28 build_result = build_operation.result()
29
30 logger.info('Build Result: {}'.format(build_result.status))
31 build_client = cloudbuild_v1.CloudBuildClient()
32
33 build = cloudbuild_v1.Build()
34
35 build.steps = [{'name':'gcr.io/cloud-builders/docker',
36 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
37 {'name':'gcr.io/cloud-builders/docker',
38 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
39
40 build_operation = build_client.create_build(project_id=myprojectid,build=build)
41
42 build_result = build_operation.result()
43tar -cvzf sourcecode.tar.gz .
44gsutil cp sourcecode.tar.gz gs://myproject_cloudbuild
45 build_client = cloudbuild_v1.CloudBuildClient()
46
47 build = cloudbuild_v1.Build()
48
49 build.source = {"storage_source":{"bucket":"myproject_cloudbuild", "object_":"gs://myproject_cloudbuild/sourcecode.tar.gz"}}
50
51 build.steps = [{'name':'gcr.io/cloud-builders/docker',
52 'args':['build', '-t', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest','.']},
53 {'name':'gcr.io/cloud-builders/docker',
54 'args':['push', 'us-central1-docker.pkg.dev/myproject/myrepo/imagename:latest']}]
55
56 build_operation = build_client.create_build(project_id=myprojectid,build=build)
57
58 build_result = build_operation.result()
59
As a result it solved the problem building an image using Client Library. I recommended to do all of this inside your Cloud Function.
QUESTION
Error when import firebase-functions-test when testing with mocha
Asked 2021-Dec-02 at 09:53I am trying to setup a Firebase Cloud Functions repo to run mocha test. However, it throws the following error when I use import * as firebase from "firebase-functions-test";
or const firebase = require("firebase-functions-test")();
. You can see in my code that I haven't even called the actual firebase functions yet so I think this a setup issue.
Question: What change do I need to make mocha test running for Firebase Functions testing using import syntax?
Working test code
1import { assert } from "chai";
2describe("Sanity Check", () => {
3 it("should pass", () => {
4 assert.equal(0, 0);
5 });
6});
7
8
Failed Test Code using require
1import { assert } from "chai";
2describe("Sanity Check", () => {
3 it("should pass", () => {
4 assert.equal(0, 0);
5 });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11 it("should pass", () => {
12 assert.equal(0, 0);
13 test.cleanup();
14 });
15});
16
Failed code using import
1import { assert } from "chai";
2describe("Sanity Check", () => {
3 it("should pass", () => {
4 assert.equal(0, 0);
5 });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11 it("should pass", () => {
12 assert.equal(0, 0);
13 test.cleanup();
14 });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21 it("should pass", () => {
22 assert.equal(0, 0);
23 test.cleanup();
24 });
25});
26
Error for the failure
1import { assert } from "chai";
2describe("Sanity Check", () => {
3 it("should pass", () => {
4 assert.equal(0, 0);
5 });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11 it("should pass", () => {
12 assert.equal(0, 0);
13 test.cleanup();
14 });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21 it("should pass", () => {
22 assert.equal(0, 0);
23 test.cleanup();
24 });
25});
26> functions@ test /Users/cupidchan/temp/functions
27> mocha -r ts-node/register test/**/*.spec.ts
28
29
30Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/encoder' is not defined by "exports" in /Users/cupidchan/temp/functions/node_modules/firebase-functions/package.json
31 at new NodeError (internal/errors.js:322:7)
32 at throwExportsNotFound (internal/modules/esm/resolve.js:322:9)
33 at packageExportsResolve (internal/modules/esm/resolve.js:545:3)
34 at resolveExports (internal/modules/cjs/loader.js:450:36)
35 at Function.Module._findPath (internal/modules/cjs/loader.js:490:31)
36 at Function.Module._resolveFilename (internal/modules/cjs/loader.js:888:27)
37 at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/cupidchan/temp/functions/node_modules/@cspotcode/source-map-support/source-map-support.js:679:30)
38 at Function.Module._load (internal/modules/cjs/loader.js:746:27)
39 at Module.require (internal/modules/cjs/loader.js:974:19)
40 at require (internal/modules/cjs/helpers.js:93:18)
41 at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/providers/firestore.js:26:19)
42 at Module._compile (internal/modules/cjs/loader.js:1085:14)
43 at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
44 at Module.load (internal/modules/cjs/loader.js:950:32)
45 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
46 at Module.require (internal/modules/cjs/loader.js:974:19)
47 at require (internal/modules/cjs/helpers.js:93:18)
48 at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/features.js:9:19)
49 at Module._compile (internal/modules/cjs/loader.js:1085:14)
50 at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
51 at Module.load (internal/modules/cjs/loader.js:950:32)
52 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
53 at Module.require (internal/modules/cjs/loader.js:974:19)
54 at require (internal/modules/cjs/helpers.js:93:18)
55 at module.exports (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/index.js:30:20)
56 at Object.<anonymous> (/Users/cupidchan/temp/functions/test/index.spec.ts:9:14)
57 at Module._compile (internal/modules/cjs/loader.js:1085:14)
58 at Module.m._compile (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1371:23)
59 at Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
60 at Object.require.extensions.<computed> [as .ts] (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1374:12)
61 at Module.load (internal/modules/cjs/loader.js:950:32)
62 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
63 at Module.require (internal/modules/cjs/loader.js:974:19)
64 at require (internal/modules/cjs/helpers.js:93:18)
65 at Object.exports.requireOrImport (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:56:20)
66 at async Object.exports.loadFilesAsync (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:88:20)
67 at async singleRun (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run-helpers.js:125:3)
68 at async Object.exports.handler (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run.js:374:5)
69npm ERR! Test failed. See above for more details.
70
package.json
1import { assert } from "chai";
2describe("Sanity Check", () => {
3 it("should pass", () => {
4 assert.equal(0, 0);
5 });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11 it("should pass", () => {
12 assert.equal(0, 0);
13 test.cleanup();
14 });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21 it("should pass", () => {
22 assert.equal(0, 0);
23 test.cleanup();
24 });
25});
26> functions@ test /Users/cupidchan/temp/functions
27> mocha -r ts-node/register test/**/*.spec.ts
28
29
30Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/encoder' is not defined by "exports" in /Users/cupidchan/temp/functions/node_modules/firebase-functions/package.json
31 at new NodeError (internal/errors.js:322:7)
32 at throwExportsNotFound (internal/modules/esm/resolve.js:322:9)
33 at packageExportsResolve (internal/modules/esm/resolve.js:545:3)
34 at resolveExports (internal/modules/cjs/loader.js:450:36)
35 at Function.Module._findPath (internal/modules/cjs/loader.js:490:31)
36 at Function.Module._resolveFilename (internal/modules/cjs/loader.js:888:27)
37 at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/cupidchan/temp/functions/node_modules/@cspotcode/source-map-support/source-map-support.js:679:30)
38 at Function.Module._load (internal/modules/cjs/loader.js:746:27)
39 at Module.require (internal/modules/cjs/loader.js:974:19)
40 at require (internal/modules/cjs/helpers.js:93:18)
41 at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/providers/firestore.js:26:19)
42 at Module._compile (internal/modules/cjs/loader.js:1085:14)
43 at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
44 at Module.load (internal/modules/cjs/loader.js:950:32)
45 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
46 at Module.require (internal/modules/cjs/loader.js:974:19)
47 at require (internal/modules/cjs/helpers.js:93:18)
48 at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/features.js:9:19)
49 at Module._compile (internal/modules/cjs/loader.js:1085:14)
50 at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
51 at Module.load (internal/modules/cjs/loader.js:950:32)
52 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
53 at Module.require (internal/modules/cjs/loader.js:974:19)
54 at require (internal/modules/cjs/helpers.js:93:18)
55 at module.exports (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/index.js:30:20)
56 at Object.<anonymous> (/Users/cupidchan/temp/functions/test/index.spec.ts:9:14)
57 at Module._compile (internal/modules/cjs/loader.js:1085:14)
58 at Module.m._compile (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1371:23)
59 at Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
60 at Object.require.extensions.<computed> [as .ts] (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1374:12)
61 at Module.load (internal/modules/cjs/loader.js:950:32)
62 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
63 at Module.require (internal/modules/cjs/loader.js:974:19)
64 at require (internal/modules/cjs/helpers.js:93:18)
65 at Object.exports.requireOrImport (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:56:20)
66 at async Object.exports.loadFilesAsync (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:88:20)
67 at async singleRun (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run-helpers.js:125:3)
68 at async Object.exports.handler (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run.js:374:5)
69npm ERR! Test failed. See above for more details.
70{
71 "name": "functions",
72 "scripts": {
73 "lint": "eslint --ext .js,.ts .",
74 "build": "tsc",
75 "serve": "npm run build && firebase emulators:start --only functions",
76 "shell": "npm run build && firebase functions:shell",
77 "start": "npm run shell",
78 "deploy": "firebase deploy --only functions",
79 "logs": "firebase functions:log",
80 "test": "mocha -r ts-node/register test/**/*.spec.ts --reporter spec"
81 },
82 "engines": {
83 "node": "14"
84 },
85 "main": "lib/index.js",
86 "dependencies": {
87 "firebase-admin": "^9.8.0",
88 "firebase-functions": "^3.14.1"
89 },
90 "devDependencies": {
91 "@types/chai": "^4.2.22",
92 "@types/mocha": "^9.0.0",
93 "@types/node": "^16.11.11",
94 "@typescript-eslint/eslint-plugin": "^3.9.1",
95 "@typescript-eslint/parser": "^3.8.0",
96 "chai": "^4.3.4",
97 "eslint": "^7.6.0",
98 "eslint-config-google": "^0.14.0",
99 "eslint-config-prettier": "^8.3.0",
100 "eslint-plugin-import": "^2.22.0",
101 "eslint-plugin-prettier": "^4.0.0",
102 "esm": "^3.2.25",
103 "firebase-functions-test": "^0.2.3",
104 "mocha": "^9.1.3",
105 "prettier": "^2.5.0",
106 "ts-node": "^10.4.0",
107 "typescript": "^3.8.0"
108 },
109 "private": true
110}
111
tsconfig.json
1import { assert } from "chai";
2describe("Sanity Check", () => {
3 it("should pass", () => {
4 assert.equal(0, 0);
5 });
6});
7
8const test = require("firebase-functions-test")();
9import { assert } from "chai";
10describe("Sanity Check", () => {
11 it("should pass", () => {
12 assert.equal(0, 0);
13 test.cleanup();
14 });
15});
16import * as firebase from "firebase-functions-test";
17import { assert } from "chai";
18
19const test = firebase();
20describe("Sanity Check", () => {
21 it("should pass", () => {
22 assert.equal(0, 0);
23 test.cleanup();
24 });
25});
26> functions@ test /Users/cupidchan/temp/functions
27> mocha -r ts-node/register test/**/*.spec.ts
28
29
30Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/encoder' is not defined by "exports" in /Users/cupidchan/temp/functions/node_modules/firebase-functions/package.json
31 at new NodeError (internal/errors.js:322:7)
32 at throwExportsNotFound (internal/modules/esm/resolve.js:322:9)
33 at packageExportsResolve (internal/modules/esm/resolve.js:545:3)
34 at resolveExports (internal/modules/cjs/loader.js:450:36)
35 at Function.Module._findPath (internal/modules/cjs/loader.js:490:31)
36 at Function.Module._resolveFilename (internal/modules/cjs/loader.js:888:27)
37 at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/cupidchan/temp/functions/node_modules/@cspotcode/source-map-support/source-map-support.js:679:30)
38 at Function.Module._load (internal/modules/cjs/loader.js:746:27)
39 at Module.require (internal/modules/cjs/loader.js:974:19)
40 at require (internal/modules/cjs/helpers.js:93:18)
41 at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/providers/firestore.js:26:19)
42 at Module._compile (internal/modules/cjs/loader.js:1085:14)
43 at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
44 at Module.load (internal/modules/cjs/loader.js:950:32)
45 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
46 at Module.require (internal/modules/cjs/loader.js:974:19)
47 at require (internal/modules/cjs/helpers.js:93:18)
48 at Object.<anonymous> (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/features.js:9:19)
49 at Module._compile (internal/modules/cjs/loader.js:1085:14)
50 at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
51 at Module.load (internal/modules/cjs/loader.js:950:32)
52 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
53 at Module.require (internal/modules/cjs/loader.js:974:19)
54 at require (internal/modules/cjs/helpers.js:93:18)
55 at module.exports (/Users/cupidchan/temp/functions/node_modules/firebase-functions-test/lib/index.js:30:20)
56 at Object.<anonymous> (/Users/cupidchan/temp/functions/test/index.spec.ts:9:14)
57 at Module._compile (internal/modules/cjs/loader.js:1085:14)
58 at Module.m._compile (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1371:23)
59 at Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
60 at Object.require.extensions.<computed> [as .ts] (/Users/cupidchan/temp/functions/node_modules/ts-node/src/index.ts:1374:12)
61 at Module.load (internal/modules/cjs/loader.js:950:32)
62 at Function.Module._load (internal/modules/cjs/loader.js:790:12)
63 at Module.require (internal/modules/cjs/loader.js:974:19)
64 at require (internal/modules/cjs/helpers.js:93:18)
65 at Object.exports.requireOrImport (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:56:20)
66 at async Object.exports.loadFilesAsync (/Users/cupidchan/temp/functions/node_modules/mocha/lib/nodejs/esm-utils.js:88:20)
67 at async singleRun (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run-helpers.js:125:3)
68 at async Object.exports.handler (/Users/cupidchan/temp/functions/node_modules/mocha/lib/cli/run.js:374:5)
69npm ERR! Test failed. See above for more details.
70{
71 "name": "functions",
72 "scripts": {
73 "lint": "eslint --ext .js,.ts .",
74 "build": "tsc",
75 "serve": "npm run build && firebase emulators:start --only functions",
76 "shell": "npm run build && firebase functions:shell",
77 "start": "npm run shell",
78 "deploy": "firebase deploy --only functions",
79 "logs": "firebase functions:log",
80 "test": "mocha -r ts-node/register test/**/*.spec.ts --reporter spec"
81 },
82 "engines": {
83 "node": "14"
84 },
85 "main": "lib/index.js",
86 "dependencies": {
87 "firebase-admin": "^9.8.0",
88 "firebase-functions": "^3.14.1"
89 },
90 "devDependencies": {
91 "@types/chai": "^4.2.22",
92 "@types/mocha": "^9.0.0",
93 "@types/node": "^16.11.11",
94 "@typescript-eslint/eslint-plugin": "^3.9.1",
95 "@typescript-eslint/parser": "^3.8.0",
96 "chai": "^4.3.4",
97 "eslint": "^7.6.0",
98 "eslint-config-google": "^0.14.0",
99 "eslint-config-prettier": "^8.3.0",
100 "eslint-plugin-import": "^2.22.0",
101 "eslint-plugin-prettier": "^4.0.0",
102 "esm": "^3.2.25",
103 "firebase-functions-test": "^0.2.3",
104 "mocha": "^9.1.3",
105 "prettier": "^2.5.0",
106 "ts-node": "^10.4.0",
107 "typescript": "^3.8.0"
108 },
109 "private": true
110}
111{
112 "compilerOptions": {
113 "module": "commonjs",
114 "noImplicitReturns": true,
115 "noUnusedLocals": true,
116 "outDir": "lib",
117 "sourceMap": true,
118 "strict": true,
119 "target": "es2017"
120 },
121 "compileOnSave": true,
122 "include": ["src", "test"]
123}
124
ANSWER
Answered 2021-Dec-02 at 09:53QUESTION
Parsing error: Cannot read file '\tsconfig.json' eslint after following Firebase Cloud Functions initialization instructions
Asked 2021-Nov-16 at 16:17Right after my TypeScript project initialization in VSCode using firebase tools for composing Firebase Cloud Functions following the official documentation the very first line of the index.ts
file displays an error:
Parsing error: Cannot read file '\tsconfig.json' eslint [1,1]
and the .eslintrc.js
displays an error:
File is a CommonJS module; it may be converted to an ES6 module.ts(80001)
Since all files are auto-generated these errors are a complete surprise and I want to get rid of them.
VersionsFor the record, here are the versions installed:
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5
These are the commands I used in the powershell in VSCode with some info/warnings:
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE package: undefined,
15npm WARN EBADENGINE required: { node: '14' },
16npm WARN EBADENGINE current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30 1:13 warning 'functions' is defined but never used @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE package: undefined,
15npm WARN EBADENGINE required: { node: '14' },
16npm WARN EBADENGINE current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30 1:13 warning 'functions' is defined but never used @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34 "compilerOptions": {
35 "module": "commonjs",
36 "noImplicitReturns": true,
37 "noUnusedLocals": true,
38 "outDir": "lib",
39 "sourceMap": true,
40 "strict": true,
41 "target": "es2017"
42 },
43 "compileOnSave": true,
44 "include": [
45 "src"
46 ]
47}
48
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE package: undefined,
15npm WARN EBADENGINE required: { node: '14' },
16npm WARN EBADENGINE current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30 1:13 warning 'functions' is defined but never used @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34 "compilerOptions": {
35 "module": "commonjs",
36 "noImplicitReturns": true,
37 "noUnusedLocals": true,
38 "outDir": "lib",
39 "sourceMap": true,
40 "strict": true,
41 "target": "es2017"
42 },
43 "compileOnSave": true,
44 "include": [
45 "src"
46 ]
47}
48module.exports = {
49 root: true,
50 env: {
51 es6: true,
52 node: true,
53 },
54 extends: [
55 "eslint:recommended",
56 "plugin:import/errors",
57 "plugin:import/warnings",
58 "plugin:import/typescript",
59 "google",
60 "plugin:@typescript-eslint/recommended",
61 ],
62 parser: "@typescript-eslint/parser",
63 parserOptions: {
64 project: ["tsconfig.json", "tsconfig.dev.json"],
65 sourceType: "module",
66 },
67 ignorePatterns: [
68 "/lib/**/*", // Ignore built files.
69 ],
70 plugins: [
71 "@typescript-eslint",
72 "import",
73 ],
74 rules: {
75 "quotes": ["error", "double"],
76 "import/no-unresolved": 0,
77 },
78};
79
80
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE package: undefined,
15npm WARN EBADENGINE required: { node: '14' },
16npm WARN EBADENGINE current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30 1:13 warning 'functions' is defined but never used @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34 "compilerOptions": {
35 "module": "commonjs",
36 "noImplicitReturns": true,
37 "noUnusedLocals": true,
38 "outDir": "lib",
39 "sourceMap": true,
40 "strict": true,
41 "target": "es2017"
42 },
43 "compileOnSave": true,
44 "include": [
45 "src"
46 ]
47}
48module.exports = {
49 root: true,
50 env: {
51 es6: true,
52 node: true,
53 },
54 extends: [
55 "eslint:recommended",
56 "plugin:import/errors",
57 "plugin:import/warnings",
58 "plugin:import/typescript",
59 "google",
60 "plugin:@typescript-eslint/recommended",
61 ],
62 parser: "@typescript-eslint/parser",
63 parserOptions: {
64 project: ["tsconfig.json", "tsconfig.dev.json"],
65 sourceType: "module",
66 },
67 ignorePatterns: [
68 "/lib/**/*", // Ignore built files.
69 ],
70 plugins: [
71 "@typescript-eslint",
72 "import",
73 ],
74 rules: {
75 "quotes": ["error", "double"],
76 "import/no-unresolved": 0,
77 },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83 functions.logger.info("Hello logs!", {structuredData: true});
84 response.send("Hello from Firebase!");
85});
86
- Reinitialization
- Creating the
.vscode
folder withsettings.json
file with
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE package: undefined,
15npm WARN EBADENGINE required: { node: '14' },
16npm WARN EBADENGINE current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30 1:13 warning 'functions' is defined but never used @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34 "compilerOptions": {
35 "module": "commonjs",
36 "noImplicitReturns": true,
37 "noUnusedLocals": true,
38 "outDir": "lib",
39 "sourceMap": true,
40 "strict": true,
41 "target": "es2017"
42 },
43 "compileOnSave": true,
44 "include": [
45 "src"
46 ]
47}
48module.exports = {
49 root: true,
50 env: {
51 es6: true,
52 node: true,
53 },
54 extends: [
55 "eslint:recommended",
56 "plugin:import/errors",
57 "plugin:import/warnings",
58 "plugin:import/typescript",
59 "google",
60 "plugin:@typescript-eslint/recommended",
61 ],
62 parser: "@typescript-eslint/parser",
63 parserOptions: {
64 project: ["tsconfig.json", "tsconfig.dev.json"],
65 sourceType: "module",
66 },
67 ignorePatterns: [
68 "/lib/**/*", // Ignore built files.
69 ],
70 plugins: [
71 "@typescript-eslint",
72 "import",
73 ],
74 rules: {
75 "quotes": ["error", "double"],
76 "import/no-unresolved": 0,
77 },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83 functions.logger.info("Hello logs!", {structuredData: true});
84 response.send("Hello from Firebase!");
85});
86 {
87 "eslint.workingDirectories": [
88 "src" // and "functions"
89 ]
90 }
91
- Updating your eslintrc.json file with the following line:
"project":"PROJECT_NAME/tsconfig.json"
- Updating
.eslintrc.js
by settingtsconfigRootDir: __dirname
inparserOptions
- Deleting the ESLint extension. The error was gone, but
firebase deploy
command didn't allow the code deployement.
So, the related thread didn't really help
ANSWER
Answered 2021-Nov-16 at 16:17Ok, I have solved the problem with a great help of this github thread False positive Error - TS6133 error (declared but its value is never read) report.
I have changed "noUnusedLocals"
setting in the tsconfig.json
file from the default true
to false
, so the file becomes:
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE package: undefined,
15npm WARN EBADENGINE required: { node: '14' },
16npm WARN EBADENGINE current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30 1:13 warning 'functions' is defined but never used @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34 "compilerOptions": {
35 "module": "commonjs",
36 "noImplicitReturns": true,
37 "noUnusedLocals": true,
38 "outDir": "lib",
39 "sourceMap": true,
40 "strict": true,
41 "target": "es2017"
42 },
43 "compileOnSave": true,
44 "include": [
45 "src"
46 ]
47}
48module.exports = {
49 root: true,
50 env: {
51 es6: true,
52 node: true,
53 },
54 extends: [
55 "eslint:recommended",
56 "plugin:import/errors",
57 "plugin:import/warnings",
58 "plugin:import/typescript",
59 "google",
60 "plugin:@typescript-eslint/recommended",
61 ],
62 parser: "@typescript-eslint/parser",
63 parserOptions: {
64 project: ["tsconfig.json", "tsconfig.dev.json"],
65 sourceType: "module",
66 },
67 ignorePatterns: [
68 "/lib/**/*", // Ignore built files.
69 ],
70 plugins: [
71 "@typescript-eslint",
72 "import",
73 ],
74 rules: {
75 "quotes": ["error", "double"],
76 "import/no-unresolved": 0,
77 },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83 functions.logger.info("Hello logs!", {structuredData: true});
84 response.send("Hello from Firebase!");
85});
86 {
87 "eslint.workingDirectories": [
88 "src" // and "functions"
89 ]
90 }
91"compilerOptions": {
92 ...
93 "noUnusedLocals": false,
94 ...
95 }
96
However, the strange thing is that after successfully deploying functions to Firebase Cloud Fuctions
using this setting and afterwards reverting the changes, the redeployment doesn´t show any error and is successful as well.
After retrying it a few time I must admit that the solution is different.
Turns out that the related stackoverflow thread did help, in particular @cherryblossom solution. Setting tsconfigRootDir: __dirname
in the .eslintrc.js
file and restarting the VSCode solves the problem.
.eslintrc.js
:
1npm --version 8.1.3
2tsc --version 4.4.4
3node --version 17.0.1
4firebase --version 9.22.0
5>npm install -g firebase-tools
6npm WARN deprecated har-validator@5.1.5: this library is no longer supported
7npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
8npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
9...
10
11>firebase init
12...
13npm WARN EBADENGINE Unsupported engine {
14npm WARN EBADENGINE package: undefined,
15npm WARN EBADENGINE required: { node: '14' },
16npm WARN EBADENGINE current: { node: 'v17.0.1', npm: '8.1.3' }
17npm WARN EBADENGINE }
18...
19
20>npm install firebase-functions@latest firebase-admin@latest --save
21...
22npm WARN deprecated har-validator@5.1.5: this library is no longer supported
23npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
24npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
25...
26
27>firebase deploy
28...
29C:\Users\SAMS\firebase_dogout\functions\src\index.ts
30 1:13 warning 'functions' is defined but never used @typescript-eslint/no-unused-vars
31...
32Error: functions predeploy error: Command terminated with non-zero exit code2
33{
34 "compilerOptions": {
35 "module": "commonjs",
36 "noImplicitReturns": true,
37 "noUnusedLocals": true,
38 "outDir": "lib",
39 "sourceMap": true,
40 "strict": true,
41 "target": "es2017"
42 },
43 "compileOnSave": true,
44 "include": [
45 "src"
46 ]
47}
48module.exports = {
49 root: true,
50 env: {
51 es6: true,
52 node: true,
53 },
54 extends: [
55 "eslint:recommended",
56 "plugin:import/errors",
57 "plugin:import/warnings",
58 "plugin:import/typescript",
59 "google",
60 "plugin:@typescript-eslint/recommended",
61 ],
62 parser: "@typescript-eslint/parser",
63 parserOptions: {
64 project: ["tsconfig.json", "tsconfig.dev.json"],
65 sourceType: "module",
66 },
67 ignorePatterns: [
68 "/lib/**/*", // Ignore built files.
69 ],
70 plugins: [
71 "@typescript-eslint",
72 "import",
73 ],
74 rules: {
75 "quotes": ["error", "double"],
76 "import/no-unresolved": 0,
77 },
78};
79
80import * as functions from "firebase-functions";
81
82export const helloWorld = functions.https.onRequest((request, response) => {
83 functions.logger.info("Hello logs!", {structuredData: true});
84 response.send("Hello from Firebase!");
85});
86 {
87 "eslint.workingDirectories": [
88 "src" // and "functions"
89 ]
90 }
91"compilerOptions": {
92 ...
93 "noUnusedLocals": false,
94 ...
95 }
96module.exports = {
97 // ...
98 parserOptions: {
99 ...
100 tsconfigRootDir: __dirname,
101 ...
102 },
103 // ...
104}
105
QUESTION
firebase function with realtime database error
Asked 2021-Nov-08 at 12:28I am new to firebase function and trying to use firebase function with Realtime database (Emulator suite).But when i try to set the value in firebase using the firebase function,it gives an error and doesn't set the value in database.
Error:
117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
17
firebase function code :
117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp();
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24 response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28 var tex = request.query.text;
29 var textLength = tex.length;
30 console.log(textLength);
31 database.ref().child('test').set("op");
32 response.send(String(textLength));
33});
34
dependencies :
117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp();
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24 response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28 var tex = request.query.text;
29 var textLength = tex.length;
30 console.log(textLength);
31 database.ref().child('test').set("op");
32 response.send(String(textLength));
33});
34{
35 "name": "functions",
36 "description": "Cloud Functions for Firebase",
37 "scripts": {
38 "serve": "firebase emulators:start --only functions",
39 "shell": "firebase functions:shell",
40 "start": "npm run shell",
41 "deploy": "firebase deploy --only functions",
42 "logs": "firebase functions:log"
43 },
44 "engines": {
45 "node": "14"
46 },
47 "main": "index.js",
48 "dependencies": {
49 "firebase-admin": "^9.8.0",
50 "firebase-functions": "^3.14.1",
51 "@firebase/database-compat": "0.1.2"
52 },
53 "devDependencies": {
54 "firebase-functions-test": "^0.2.0"
55 },
56 "private": true
57}
58
npm installed packages
117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp();
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24 response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28 var tex = request.query.text;
29 var textLength = tex.length;
30 console.log(textLength);
31 database.ref().child('test').set("op");
32 response.send(String(textLength));
33});
34{
35 "name": "functions",
36 "description": "Cloud Functions for Firebase",
37 "scripts": {
38 "serve": "firebase emulators:start --only functions",
39 "shell": "firebase functions:shell",
40 "start": "npm run shell",
41 "deploy": "firebase deploy --only functions",
42 "logs": "firebase functions:log"
43 },
44 "engines": {
45 "node": "14"
46 },
47 "main": "index.js",
48 "dependencies": {
49 "firebase-admin": "^9.8.0",
50 "firebase-functions": "^3.14.1",
51 "@firebase/database-compat": "0.1.2"
52 },
53 "devDependencies": {
54 "firebase-functions-test": "^0.2.0"
55 },
56 "private": true
57}
58+-- @firebase/app-compat@0.1.7
59| +-- @firebase/app@0.7.6
60| | +-- @firebase/component@0.5.8
61| | | +-- @firebase/util@1.4.1 deduped
62| | | `-- tslib@2.3.1 deduped
63| | +-- @firebase/logger@0.3.1
64| | | `-- tslib@2.3.1 deduped
65| | +-- @firebase/util@1.4.1
66| | | `-- tslib@2.3.1 deduped
67| | `-- tslib@2.3.1 deduped
68| +-- @firebase/component@0.5.8
69| | +-- @firebase/util@1.4.1 deduped
70| | `-- tslib@2.3.1 deduped
71| +-- @firebase/logger@0.3.1
72| | `-- tslib@2.3.1 deduped
73| +-- @firebase/util@1.4.1
74| | `-- tslib@2.3.1 deduped
75| `-- tslib@2.3.1
76+-- @firebase/database-compat@0.1.2
77| +-- @firebase/component@0.5.7
78| | +-- @firebase/util@1.4.0 deduped
79| | `-- tslib@2.3.1 deduped
80| +-- @firebase/database@0.12.2
81| | +-- @firebase/auth-interop-types@0.1.6
82| | +-- @firebase/component@0.5.7 deduped
83| | +-- @firebase/logger@0.3.0 deduped
84| | +-- @firebase/util@1.4.0 deduped
85| | +-- faye-websocket@0.11.4
86| | | `-- websocket-driver@0.7.4
87| | | +-- http-parser-js@0.5.3
88| | | +-- safe-buffer@5.2.1 deduped
89| | | `-- websocket-extensions@0.1.4
90| | `-- tslib@2.3.1 deduped
91| +-- @firebase/database-types@0.9.1
92| | +-- @firebase/app-types@0.7.0
93| | `-- @firebase/util@1.4.0 deduped
94| +-- @firebase/logger@0.3.0
95| | `-- tslib@2.3.1 deduped
96| +-- @firebase/util@1.4.0
97| | `-- tslib@2.3.1 deduped
98| `-- tslib@2.3.1 deduped
99`-- firebase-admin@10.0.0
100 +-- @firebase/database-compat@0.1.2 deduped
101 +-- @firebase/database-types@0.7.3
102 | `-- @firebase/app-types@0.6.3
103 +-- @google-cloud/firestore@4.15.1
104 | +-- fast-deep-equal@3.1.3
105 | +-- functional-red-black-tree@1.0.1
106 | +-- google-gax@2.28.0
107
ANSWER
Answered 2021-Nov-05 at 13:59I'm unsure as to the cause of that log message, but I do see that you are returning a response from your function before it completes all of its work. In a deployed function, as soon as the function returns, all further actions should be treated as if they will never be executed as documented here. An "inactive" function might be terminated at any time, is severely throttled and any network calls you make (like setting data in the RTDB) may never be executed.
I know you are new to this, but its a good habit to get into now: don't assume the person calling your function is you. Check for problems like missing query parameters and dodgy data before you blindly action something. The Admin SDK bypasses your database's security rules and if you are not careful a malicious user can cause some damage (e.g. a user that updates /users/$theirUid/roles/admin
to true
).
117:33:14
2I
3function[us-central1-textToLength]
4[2021-11-05T12:03:14.194Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
517:34:18
6I
7function[us-central1-textToLength]
8[2021-11-05T12:04:18.762Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
917:35:06
10I
11function[us-central1-textToLength]
12[2021-11-05T12:05:06.473Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
1317:35:54
14I
15function[us-central1-textToLength]
16[2021-11-05T12:05:54.409Z] @firebase/database: FIREBASE WARNING: wss:// URL used, but browser isn't known to support websockets. Trying anyway.
17const functions = require('firebase-functions');
18var admin = require("firebase-admin");
19admin.initializeApp();
20
21var database = admin.database();
22
23exports.helloWorld = functions.https.onRequest((request, response) => {
24 response.send("Hello from Firebase!");
25});
26
27exports.textToLength = functions.https.onRequest((request, response) => {
28 var tex = request.query.text;
29 var textLength = tex.length;
30 console.log(textLength);
31 database.ref().child('test').set("op");
32 response.send(String(textLength));
33});
34{
35 "name": "functions",
36 "description": "Cloud Functions for Firebase",
37 "scripts": {
38 "serve": "firebase emulators:start --only functions",
39 "shell": "firebase functions:shell",
40 "start": "npm run shell",
41 "deploy": "firebase deploy --only functions",
42 "logs": "firebase functions:log"
43 },
44 "engines": {
45 "node": "14"
46 },
47 "main": "index.js",
48 "dependencies": {
49 "firebase-admin": "^9.8.0",
50 "firebase-functions": "^3.14.1",
51 "@firebase/database-compat": "0.1.2"
52 },
53 "devDependencies": {
54 "firebase-functions-test": "^0.2.0"
55 },
56 "private": true
57}
58+-- @firebase/app-compat@0.1.7
59| +-- @firebase/app@0.7.6
60| | +-- @firebase/component@0.5.8
61| | | +-- @firebase/util@1.4.1 deduped
62| | | `-- tslib@2.3.1 deduped
63| | +-- @firebase/logger@0.3.1
64| | | `-- tslib@2.3.1 deduped
65| | +-- @firebase/util@1.4.1
66| | | `-- tslib@2.3.1 deduped
67| | `-- tslib@2.3.1 deduped
68| +-- @firebase/component@0.5.8
69| | +-- @firebase/util@1.4.1 deduped
70| | `-- tslib@2.3.1 deduped
71| +-- @firebase/logger@0.3.1
72| | `-- tslib@2.3.1 deduped
73| +-- @firebase/util@1.4.1
74| | `-- tslib@2.3.1 deduped
75| `-- tslib@2.3.1
76+-- @firebase/database-compat@0.1.2
77| +-- @firebase/component@0.5.7
78| | +-- @firebase/util@1.4.0 deduped
79| | `-- tslib@2.3.1 deduped
80| +-- @firebase/database@0.12.2
81| | +-- @firebase/auth-interop-types@0.1.6
82| | +-- @firebase/component@0.5.7 deduped
83| | +-- @firebase/logger@0.3.0 deduped
84| | +-- @firebase/util@1.4.0 deduped
85| | +-- faye-websocket@0.11.4
86| | | `-- websocket-driver@0.7.4
87| | | +-- http-parser-js@0.5.3
88| | | +-- safe-buffer@5.2.1 deduped
89| | | `-- websocket-extensions@0.1.4
90| | `-- tslib@2.3.1 deduped
91| +-- @firebase/database-types@0.9.1
92| | +-- @firebase/app-types@0.7.0
93| | `-- @firebase/util@1.4.0 deduped
94| +-- @firebase/logger@0.3.0
95| | `-- tslib@2.3.1 deduped
96| +-- @firebase/util@1.4.0
97| | `-- tslib@2.3.1 deduped
98| `-- tslib@2.3.1 deduped
99`-- firebase-admin@10.0.0
100 +-- @firebase/database-compat@0.1.2 deduped
101 +-- @firebase/database-types@0.7.3
102 | `-- @firebase/app-types@0.6.3
103 +-- @google-cloud/firestore@4.15.1
104 | +-- fast-deep-equal@3.1.3
105 | +-- functional-red-black-tree@1.0.1
106 | +-- google-gax@2.28.0
107exports.textToLength = functions.https.onRequest((request, response) => {
108 const { text: queryText } = request.query;
109
110 if (!queryText)
111 return response.status(400).send("Missing ?text");
112 if (typeof queryText !== 'string')
113 return response.status(400).send("Bad value for ?text");
114
115 const textLength = queryText.length;
116
117 database.ref()
118 .child('test')
119 .set("op")
120 .then(() => {
121 console.log(`Successfully updated /test/op to ${textLength}`);
122 response.send(String(textLength))
123 })
124 .catch((err) => {
125 console.error(`Failed to update /test/op to ${textLength}: `, error);
126 response.status(500).send(`Failed to update database to ${textLength}`)
127 });
128 ;
129});
130
Note: When deploying HTTPS Request functions, don't forget to handle CORS.
QUESTION
Firebase Cloud Functions min-instances setting seems to be ignored
Asked 2021-Oct-30 at 06:35Firebase has announced in September 2021 that it is possible now to configure its cloud function autoscaling in a way, so that a certain number of instances will always be running (https://firebase.google.com/docs/functions/manage-functions#min-max-instances).
I have tried to set this up, but I can not get it to work: At first I have set the number of minimum instances in Google Cloud Console: Cloud Console Screenshot After doing this I expected that one instance for that cloud function would run at any time. The metrics of that function indicate that it instances were still scaled down to 0: Cloud functions "Active Instances Metric"
So to me it looks a bit as if my setting is ignored here. Am I missing anything? Google Cloud Console shows me that the number of minimum instances has been set to 1 so it seems to know about it but to ignore it. Is this feature only available in certain regions?
I have also tried to set the number of minimum instances using the Firebase SDK for Cloud Functions (https://www.npmjs.com/package/firebase-functions). This gave me the same result, my setting is still ignored.
ANSWER
Answered 2021-Oct-30 at 06:35According to the Documentation, the Active Instances metrics shows the number of instances that are currently handling the request.
As stated in the Documentation :
Cloud Functions scales by creating new instances of your function. Each of these instances can handle only one request at a time, so large spikes in request volume often causes longer wait times as new instances are created to handle the demand.
Because functions are stateless, your function sometimes initializes the execution environment from scratch, which is called a cold start. Cold starts can take significant amounts of time to complete, so we recommend setting a minimum number of Cloud Functions instances if your application is latency-sensitive.
You can also refer to the Stackoverflow thread where it has been mentioned that
Setting up minInstances does not mean that there will always be that much number of Active Instances. Minimum instances are kept running idle (without CPU > allocated), so are not counted in Active Instances.
QUESTION
Receiving inconsistent data in Go api
Asked 2021-Oct-23 at 12:31I was learning the Go language and tested Google Cloud Functions with go + Google Firestore as the database.
While I was testing the response I got inconsistent responses.
I have used the json Marshaller to convert Firebase data to Json object to return from the API, this API is hosted in the Google Cloud Functions.
1// Package p contains an HTTP Cloud Function.
2package p
3
4import (
5 "context"
6 "fmt"
7 "log"
8 "net/http"
9
10 s "strings"
11
12 "encoding/json"
13
14 "cloud.google.com/go/firestore"
15 "google.golang.org/api/iterator"
16)
17
18
19func HelloWorld(w http.ResponseWriter, r *http.Request) {
20 path := s.Replace(r.URL.Path, "/", "", -1)
21
22 ctx := context.Background()
23 client := createClient(ctx)
24
25 iter := client.Collection("profile").Where("publicUrl", "==", path).Documents(ctx)
26 for {
27 doc, err := iter.Next()
28 if err == iterator.Done {
29 break
30 }
31 var publicDTO PublicDTO
32 var Profile Profile
33 doc.DataTo(&Profile)
34
35 publicDTO.Profile = Profile
36
37 b, err := json.Marshal(publicDTO)
38 if err != nil {
39 fmt.Println(err)
40 return
41 }
42
43 w.Header().Set("Content-Type", "application/json")
44 w.Write(b)
45 }
46
47}
48
49func createClient(ctx context.Context) *firestore.Client {
50 projectID := "projectId"
51
52 client, err := firestore.NewClient(ctx, projectID)
53 if err != nil {
54 log.Fatalf("Failed to create client: %v", err)
55 }
56 return client
57}
58
59type PublicDTO struct {
60 Profile Profile `json:"profile"`
61}
62
63type Profile struct {
64 Id string `json:"id"`
65 FirstName string `json:"firstName"`
66 LastName string `json:"lastName"`
67 FullName string `json:"fullName"`
68 Email string `json:"email"`
69 ImageUrl string `json:"imageUrl"`
70 CoverPic string `json:"coverPic"`
71 Experience int `json:"experience"`
72 PhoneNumber string `json:"phoneNumber"`
73 Description string `json:"description"`
74 Address string `json:"address"`
75 State string `json:"state"`
76 Country string `json:"country"`
77 Dob map[string]string `json:"dob"`
78 Website string `json:"website"`
79 Reputation int `json:"reputation"`
80 MemberFrom map[string]int `json:"memberFrom"`
81 Title string `json:"title"`
82 Organization string `json:"organization"`
83 Status string `json:"status"`
84 Setup int `json:"setup"`
85 Social map[string]string `json:"social"`
86 PublicUrl string `json:"publicUrl"`
87 Language []string `json:"language"`
88 Interests []string `json:"interests"`
89}
90
91
but each time the response i'm getting is inconsistent, some of the values are missing.
ANSWER
Answered 2021-Oct-23 at 12:31The solution i got after marshal and unmarshal, it works as expected.
1// Package p contains an HTTP Cloud Function.
2package p
3
4import (
5 "context"
6 "fmt"
7 "log"
8 "net/http"
9
10 s "strings"
11
12 "encoding/json"
13
14 "cloud.google.com/go/firestore"
15 "google.golang.org/api/iterator"
16)
17
18
19func HelloWorld(w http.ResponseWriter, r *http.Request) {
20 path := s.Replace(r.URL.Path, "/", "", -1)
21
22 ctx := context.Background()
23 client := createClient(ctx)
24
25 iter := client.Collection("profile").Where("publicUrl", "==", path).Documents(ctx)
26 for {
27 doc, err := iter.Next()
28 if err == iterator.Done {
29 break
30 }
31 var publicDTO PublicDTO
32 var Profile Profile
33 doc.DataTo(&Profile)
34
35 publicDTO.Profile = Profile
36
37 b, err := json.Marshal(publicDTO)
38 if err != nil {
39 fmt.Println(err)
40 return
41 }
42
43 w.Header().Set("Content-Type", "application/json")
44 w.Write(b)
45 }
46
47}
48
49func createClient(ctx context.Context) *firestore.Client {
50 projectID := "projectId"
51
52 client, err := firestore.NewClient(ctx, projectID)
53 if err != nil {
54 log.Fatalf("Failed to create client: %v", err)
55 }
56 return client
57}
58
59type PublicDTO struct {
60 Profile Profile `json:"profile"`
61}
62
63type Profile struct {
64 Id string `json:"id"`
65 FirstName string `json:"firstName"`
66 LastName string `json:"lastName"`
67 FullName string `json:"fullName"`
68 Email string `json:"email"`
69 ImageUrl string `json:"imageUrl"`
70 CoverPic string `json:"coverPic"`
71 Experience int `json:"experience"`
72 PhoneNumber string `json:"phoneNumber"`
73 Description string `json:"description"`
74 Address string `json:"address"`
75 State string `json:"state"`
76 Country string `json:"country"`
77 Dob map[string]string `json:"dob"`
78 Website string `json:"website"`
79 Reputation int `json:"reputation"`
80 MemberFrom map[string]int `json:"memberFrom"`
81 Title string `json:"title"`
82 Organization string `json:"organization"`
83 Status string `json:"status"`
84 Setup int `json:"setup"`
85 Social map[string]string `json:"social"`
86 PublicUrl string `json:"publicUrl"`
87 Language []string `json:"language"`
88 Interests []string `json:"interests"`
89}
90
91package p
92
93import (
94 "context"
95 "encoding/json"
96 "fmt"
97 "log"
98 "net/http"
99 s "strings"
100 "time"
101
102 "cloud.google.com/go/firestore"
103 "google.golang.org/api/iterator"
104)
105
106var ctx context.Context
107var client *firestore.Client
108
109func PublicApi(w http.ResponseWriter, r *http.Request) {
110 path := s.Replace(r.URL.Path, "/", "", -1)
111 w.Header().Set("Content-Type", "application/json")
112 w.Header().Set("Access-Control-Allow-Origin", "*")
113 w.Header().Set("Access-Control-Allow-Headers", "*")
114
115 newFsConfigBytes, _ := json.Marshal(getPublicDTO(path))
116 w.Write(newFsConfigBytes)
117}
118
119func getPublicDTO(path string) (publicDTO publicDTO) {
120 ctx = context.Background()
121 client = createClient()
122 profile, id := getProfiles(path)
123 publicDTO.Profile = profile
124 publicDTO.MilestoneDTOS = getMilestone(id)
125 return
126}
127
128func getProfiles(link string) (profile, string) {
129 var retVal profile
130 var id string
131 iter := client.Collection("profile").Where("publicUrl", "==", link).Documents(ctx)
132 for {
133 doc, err := iter.Next()
134 if err == iterator.Done {
135 break
136 }
137 id = doc.Ref.ID
138 b, _ := json.Marshal(doc.Data())
139 json.Unmarshal(b, &retVal)
140 }
141 return retVal, id
142}
143
144func getMilestone(id string) []milestoneDTOS {
145 var retVal []milestoneDTOS
146 iter := client.Collection("milestone").Where("userId", "==", id).Documents(ctx)
147
148 for {
149 var milestoneDTO milestoneDTOS
150 doc, err := iter.Next()
151 if err == iterator.Done {
152 break
153 }
154 b, _ := json.Marshal(doc.Data())
155 err = json.Unmarshal(b, &milestoneDTO)
156 if err != nil {
157 fmt.Println(err)
158 }
159 retVal = append(retVal, milestoneDTO)
160 }
161 return retVal
162}
163
164func createClient() *firestore.Client {
165 projectID := "live-hiveboard"
166
167 client, err := firestore.NewClient(ctx, projectID)
168 if err != nil {
169 log.Fatalf("Failed to create client: %v", err)
170 }
171 return client
172}
173
174type profile struct {
175 Address string `json:"address"`
176 City string `json:"city"`
177 Country string `json:"country"`
178 CoverPic string `json:"coverPic"`
179 CreatedBy string `json:"createdBy"`
180 CreatedDate int `json:"createdDate"`
181 Description string `json:"description"`
182 Dob int64 `json:"dob"`
183 Email string `json:"email"`
184 Enabled bool `json:"enabled"`
185 Experience int `json:"experience"`
186 FirstName string `json:"firstName"`
187 FullName string `json:"fullName"`
188 FullNameNoSpace string `json:"fullNameNoSpace"`
189 ImageURL string `json:"imageUrl"`
190 Interests []string `json:"interests"`
191 IsEnabled bool `json:"isEnabled"`
192 Language string `json:"language"`
193 LastModifiedDate int `json:"lastModifiedDate"`
194 LastName string `json:"lastName"`
195 LatLng string `json:"latLng"`
196 MemberFrom time.Time `json:"memberFrom"`
197 ObjectID string `json:"objectID"`
198 Organization string `json:"organization"`
199 PhoneNumber string `json:"phoneNumber"`
200 PlanID string `json:"planId"`
201 PublicURL string `json:"publicUrl"`
202 Reputation int `json:"reputation"`
203 Setup int `json:"setup"`
204 Social string `json:"social"`
205 State string `json:"state"`
206 Status string `json:"status"`
207 Title string `json:"title"`
208 Website string `json:"website"`
209}
210
211type milestoneDTOS struct {
212 Category string `json:"category"`
213 CreatedBy string `json:"createdBy"`
214 CreatedDate int `json:"createdDate"`
215 Description string `json:"description"`
216 Enabled bool `json:"enabled"`
217 EndDate time.Time `json:"endDate"`
218 IsCurrentPosition bool `json:"isCurrentPosition"`
219 IsEnabled bool `json:"isEnabled"`
220 LastModifiedBy time.Time `json:"lastModifiedBy"`
221 LastModifiedDate int `json:"lastModifiedDate"`
222 ObjectID string `json:"objectID"`
223 Organization string `json:"organization"`
224 PictureURL string `json:"pictureURL"`
225 Profile string `json:"profile"`
226 Score float64 `json:"score"`
227 StartDate time.Time `json:"startDate"`
228 Tags []string `json:"tags"`
229 Title string `json:"title"`
230 URL string `json:"url"`
231 UserID string `json:"userId"`
232}
233
234type publicDTO struct {
235 Profile profile `json:"profile"`
236 MilestoneDTOS []milestoneDTOS `json:"milestoneDTOS"`
237}
238
239
QUESTION
Get the client_id of the IAM proxy on GCP Cloud composer
Asked 2021-Oct-15 at 15:02I'm trying to trigger Airflow DAG inside of a composer environment with cloud functions. In order to do that I need to get the client id as described here. I've tried with curl command but it doesn't return any value. With a python script I keep getting this error:
1Traceback (most recent call last):
2 File "get_client_id.py", line 55, in <module>
3 get_client_id(
4 File "get_client_id.py", line 40, in get_client_id
5 print(query_string['client_id'][0])
6KeyError: 'client_id'
7
Python code:
1Traceback (most recent call last):
2 File "get_client_id.py", line 55, in <module>
3 get_client_id(
4 File "get_client_id.py", line 40, in get_client_id
5 print(query_string['client_id'][0])
6KeyError: 'client_id'
7import google.auth
8import google.auth.transport.requests
9import requests
10import six.moves.urllib.parse
11
12# Authenticate with Google Cloud.
13# See: https://cloud.google.com/docs/authentication/getting-started
14credentials, _ = google.auth.default(
15 scopes=['https://www.googleapis.com/auth/cloud-platform'])
16authed_session = google.auth.transport.requests.AuthorizedSession(
17 credentials)
18
19project_id = 'my-project'
20location = 'my-region'
21composer_environment = 'my-env'
22
23environment_url = (
24 'https://composer.googleapis.com/v1beta1/projects/{}/locations/{}'
25 '/environments/{}').format(project_id, location, composer_environment)
26composer_response = authed_session.request('GET', environment_url)
27environment_data = composer_response.json()
28airflow_uri = environment_data['config']['airflowUri']
29
30
31
32# The Composer environment response does not include the IAP client ID.
33# Make a second, unauthenticated HTTP request to the web server to get the
34# redirect URI.
35redirect_response = requests.get(airflow_uri, allow_redirects=False)
36redirect_location = redirect_response.headers['location']
37
38# Extract the client_id query parameter from the redirect.
39parsed = six.moves.urllib.parse.urlparse(redirect_location)
40query_string = six.moves.urllib.parse.parse_qs(parsed.query)
41print(query_string['client_id'][0])
42
cURL command:
1Traceback (most recent call last):
2 File "get_client_id.py", line 55, in <module>
3 get_client_id(
4 File "get_client_id.py", line 40, in get_client_id
5 print(query_string['client_id'][0])
6KeyError: 'client_id'
7import google.auth
8import google.auth.transport.requests
9import requests
10import six.moves.urllib.parse
11
12# Authenticate with Google Cloud.
13# See: https://cloud.google.com/docs/authentication/getting-started
14credentials, _ = google.auth.default(
15 scopes=['https://www.googleapis.com/auth/cloud-platform'])
16authed_session = google.auth.transport.requests.AuthorizedSession(
17 credentials)
18
19project_id = 'my-project'
20location = 'my-region'
21composer_environment = 'my-env'
22
23environment_url = (
24 'https://composer.googleapis.com/v1beta1/projects/{}/locations/{}'
25 '/environments/{}').format(project_id, location, composer_environment)
26composer_response = authed_session.request('GET', environment_url)
27environment_data = composer_response.json()
28airflow_uri = environment_data['config']['airflowUri']
29
30
31
32# The Composer environment response does not include the IAP client ID.
33# Make a second, unauthenticated HTTP request to the web server to get the
34# redirect URI.
35redirect_response = requests.get(airflow_uri, allow_redirects=False)
36redirect_location = redirect_response.headers['location']
37
38# Extract the client_id query parameter from the redirect.
39parsed = six.moves.urllib.parse.urlparse(redirect_location)
40query_string = six.moves.urllib.parse.parse_qs(parsed.query)
41print(query_string['client_id'][0])
42curl -v my_airflow_url 2>&1 >/dev/null | grep -o "client_id\=[A-Za-z0-9-]*\.apps\.googleusercontent\.com"
43
Does anybody have an idea how to get the Get the client_id
of the IAM proxy?
ANSWER
Answered 2021-Sep-28 at 13:00Posting this Community Wiki
for better visibility
.
As mentioned in the comment section by @LEC
this configuration is compatible with Cloud Composer V1
which can be found in GCP Documentation Triggering DAGs with Cloud Functions.
At the moment there can be found two tabs Cloud Composer 1 Guides
and Cloud Composer 2 Guides
.
Under Cloud Composer 1
is code used by the OP, but if you will check Cloud Composer 2
under Manage DAGs
> Triggering DAGs with Cloud Functions you will get information that there is not proper documentation yet.
This documentation page for Cloud Composer 2 is not yet available. Please use the page for Cloud Composer 1.
As solution, please use Cloud Composer V1
.
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Cloud Functions
Tutorials and Learning Resources are not available at this moment for Cloud Functions