kandi background
kandi background
Explore Kits
kandi background
Explore Kits
Explore all Pub Sub open source software, libraries, packages, source code, cloud functions and APIs.

Popular New Releases in Pub Sub

EventBus 3.3.1

5.2.6

release 4.9.3

v2.10.0

RabbitMQ 3.9.8

EventBus

EventBus 3.3.1

celery

5.2.6

rocketmq

release 4.9.3

pulsar

v2.10.0

rabbitmq-server

RabbitMQ 3.9.8

Popular Libraries in Pub Sub

Trending New libraries in Pub Sub

Top Authors in Pub Sub

1

28 Libraries

64288

2

27 Libraries

16029

3

26 Libraries

2587

4

20 Libraries

863

5

19 Libraries

606

6

16 Libraries

104

7

15 Libraries

1238

8

15 Libraries

2608

9

15 Libraries

12976

10

11 Libraries

685

1

28 Libraries

64288

2

27 Libraries

16029

3

26 Libraries

2587

4

20 Libraries

863

5

19 Libraries

606

6

16 Libraries

104

7

15 Libraries

1238

8

15 Libraries

2608

9

15 Libraries

12976

10

11 Libraries

685

Trending Kits in Pub Sub

No Trending Kits are available at this moment for Pub Sub

Trending Discussions on Pub Sub

    Build JSON content in R according Google Cloud Pub Sub message format
    BigQuery Table a Pub Sub Topic not working in Apache Beam Python SDK? Static source to Streaming Sink
    Pub Sub Lite topics with Peak Capacity Throughput option
    How do I add permissions to a NATS User to allow the User to query & create Jestream keyvalue stores?
    MSK vs SQS + SNS
    Dataflow resource usage
    Run code on Python Flask AppEngine startup in GCP
    Is there a way to listen for updates on multiple Google Classroom Courses using Pub Sub?
    Flow.take(ITEM_COUNT) returning all the elements rather then specified amount of elements
    Wrapping Pub-Sub Java API in Akka Streams Custom Graph Stage

QUESTION

Build JSON content in R according Google Cloud Pub Sub message format

Asked 2022-Apr-16 at 09:59

In R, I want to build json content according this Google Cloud Pub Sub message format: https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage

It have to respect :

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11

The message built will be readed from this Python code:

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15

This following R code builds a R dataframe and converts it to json content:

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15library(jsonlite)
16data="Hello World!"
17df <- data.frame(data)
18attributes <- data.frame(file_name=c('gfs_data_temp_FULL.csv'))
19df$attributes <- attributes
20
21msg <- df %>%
22    toJSON(auto_unbox = TRUE, dataframe = 'columns', pretty = T) %>%
23    # Pub/Sub expects a base64 encoded string
24    googlePubsubR::msg_encode() %>%
25    googlePubsubR::PubsubMessage()
26

It seems good but when I visualise it with a json editor :

enter image description here

indexes are added.

Additionally there is the message content: enter image description here

I dont'sure it respects Google Cloud Pub Sub message format...

ANSWER

Answered 2022-Apr-16 at 09:59

Not sure why, but replacing the dataframe by a list seems to work:

copy icondownload icon

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15library(jsonlite)
16data="Hello World!"
17df <- data.frame(data)
18attributes <- data.frame(file_name=c('gfs_data_temp_FULL.csv'))
19df$attributes <- attributes
20
21msg <- df %>%
22    toJSON(auto_unbox = TRUE, dataframe = 'columns', pretty = T) %>%
23    # Pub/Sub expects a base64 encoded string
24    googlePubsubR::msg_encode() %>%
25    googlePubsubR::PubsubMessage()
26library(jsonlite)
27
28df = list(data = "Hello World")
29attributes <- list(file_name=c('toto.csv'))
30df$attributes <- attributes
31
32df %>%
33  toJSON(auto_unbox = TRUE, simplifyVector=TRUE, dataframe = 'columns', pretty = T)
34

Output:

copy icondownload icon

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15library(jsonlite)
16data="Hello World!"
17df <- data.frame(data)
18attributes <- data.frame(file_name=c('gfs_data_temp_FULL.csv'))
19df$attributes <- attributes
20
21msg <- df %>%
22    toJSON(auto_unbox = TRUE, dataframe = 'columns', pretty = T) %>%
23    # Pub/Sub expects a base64 encoded string
24    googlePubsubR::msg_encode() %>%
25    googlePubsubR::PubsubMessage()
26library(jsonlite)
27
28df = list(data = "Hello World")
29attributes <- list(file_name=c('toto.csv'))
30df$attributes <- attributes
31
32df %>%
33  toJSON(auto_unbox = TRUE, simplifyVector=TRUE, dataframe = 'columns', pretty = T)
34{
35  "data": "Hello World",
36  "attributes": {
37    "file_name": "toto.csv"
38  }
39} 
40

Source https://stackoverflow.com/questions/71892778

Community Discussions contain sources that include Stack Exchange Network

    Build JSON content in R according Google Cloud Pub Sub message format
    BigQuery Table a Pub Sub Topic not working in Apache Beam Python SDK? Static source to Streaming Sink
    Pub Sub Lite topics with Peak Capacity Throughput option
    How do I add permissions to a NATS User to allow the User to query & create Jestream keyvalue stores?
    MSK vs SQS + SNS
    Dataflow resource usage
    Run code on Python Flask AppEngine startup in GCP
    Is there a way to listen for updates on multiple Google Classroom Courses using Pub Sub?
    Flow.take(ITEM_COUNT) returning all the elements rather then specified amount of elements
    Wrapping Pub-Sub Java API in Akka Streams Custom Graph Stage

QUESTION

Build JSON content in R according Google Cloud Pub Sub message format

Asked 2022-Apr-16 at 09:59

In R, I want to build json content according this Google Cloud Pub Sub message format: https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage

It have to respect :

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11

The message built will be readed from this Python code:

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15

This following R code builds a R dataframe and converts it to json content:

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15library(jsonlite)
16data="Hello World!"
17df <- data.frame(data)
18attributes <- data.frame(file_name=c('gfs_data_temp_FULL.csv'))
19df$attributes <- attributes
20
21msg <- df %>%
22    toJSON(auto_unbox = TRUE, dataframe = 'columns', pretty = T) %>%
23    # Pub/Sub expects a base64 encoded string
24    googlePubsubR::msg_encode() %>%
25    googlePubsubR::PubsubMessage()
26

It seems good but when I visualise it with a json editor :

enter image description here

indexes are added.

Additionally there is the message content: enter image description here

I dont'sure it respects Google Cloud Pub Sub message format...

ANSWER

Answered 2022-Apr-16 at 09:59

Not sure why, but replacing the dataframe by a list seems to work:

copy icondownload icon

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15library(jsonlite)
16data="Hello World!"
17df <- data.frame(data)
18attributes <- data.frame(file_name=c('gfs_data_temp_FULL.csv'))
19df$attributes <- attributes
20
21msg <- df %>%
22    toJSON(auto_unbox = TRUE, dataframe = 'columns', pretty = T) %>%
23    # Pub/Sub expects a base64 encoded string
24    googlePubsubR::msg_encode() %>%
25    googlePubsubR::PubsubMessage()
26library(jsonlite)
27
28df = list(data = "Hello World")
29attributes <- list(file_name=c('toto.csv'))
30df$attributes <- attributes
31
32df %>%
33  toJSON(auto_unbox = TRUE, simplifyVector=TRUE, dataframe = 'columns', pretty = T)
34

Output:

copy icondownload icon

1{
2  "data": string,
3  "attributes": {
4    string: string,
5    ...
6  },
7  "messageId": string,
8  "publishTime": string,
9  "orderingKey": string
10}
11def pubsub_read(data, context):
12    '''This function is executed from a Cloud Pub/Sub'''
13    message = base64.b64decode(data['data']).decode('utf-8')
14    file_name = data['attributes']['file_name']
15library(jsonlite)
16data="Hello World!"
17df <- data.frame(data)
18attributes <- data.frame(file_name=c('gfs_data_temp_FULL.csv'))
19df$attributes <- attributes
20
21msg <- df %>%
22    toJSON(auto_unbox = TRUE, dataframe = 'columns', pretty = T) %>%
23    # Pub/Sub expects a base64 encoded string
24    googlePubsubR::msg_encode() %>%
25    googlePubsubR::PubsubMessage()
26library(jsonlite)
27
28df = list(data = "Hello World")
29attributes <- list(file_name=c('toto.csv'))
30df$attributes <- attributes
31
32df %>%
33  toJSON(auto_unbox = TRUE, simplifyVector=TRUE, dataframe = 'columns', pretty = T)
34{
35  "data": "Hello World",
36  "attributes": {
37    "file_name": "toto.csv"
38  }
39} 
40

Source https://stackoverflow.com/questions/71892778

Community Discussions contain sources that include Stack Exchange Network

Tutorials and Learning Resources in Pub Sub

Tutorials and Learning Resources are not available at this moment for Pub Sub

Share this Page

share link

Get latest updates on Pub Sub

  • © 2022 Open Weaver Inc.