RMongo | R client to interface with MongoDB | Application Framework library

I am trying to connect mongodb in R using Rmongo library.But query function giving me error.

We have been struggling with this issue for a long time now. In short, our storm topology stops emitting messages from spout after some time in a random fashion. We have an automated script which re-deploys the topology at 06:00 UTC everyday after the master data refresh activity is complete.

In the last 2 weeks, our topology stopped emitting the messages for 3 times in late UTC hours (between 22:00 and 02:00). It only comes online when we restart it which is around 06:00 UTC.

I've searched for many answers & blogs but couldn't find out what's happening here. We have an un-anchored topology which is a choice we have made like 3-4 years ago. We started with 0.9.2 and now we are on 1.1.0.

I've checked all kind of logs and I'm 100% sure that the nextTuple() method for the controller is not getting called and there are no exceptions happening in the system which may cause this. I've also checked all kind of logs we accumulate and there is not even a single ERROR or WARN logs explaining the abrupt stoppage. The INFO logs are also not that helpful. There is nothing which can be connected to this issue in worker logs or supervisor logs or nimbus logs.

This is how our spout class looks: Controller.java

I am trying to obtain a database that comes from Mongo DB to R, so I can make anlaysis on it. The bridge between these two is a R package: Rmongo. As I have some policy rules, I cannot show you the dataset and my output, so I will try to explain as best as possible.

My two first commands, after installing the package, are these ones:

mg1 <- mongoDbConnect("test", "localhost", 27018)

dbShowCollections(mg1)

Which works, as it shows the collection, or the different variables. Then, I can use the commands made by the Rmongo package, meaning:

query = dbGetQuery(mg1, 'address_history','{}')

This normally returns a data frame with all the variables on each column. But, because it is a nested file, I only get the first three variables (out of around fifty) because they are at the top of the nest. For the rest, I get one column of the data frame with the json code (so of approximately 50 variables) that I cannot seem to turn in a data frame. If someone is familiar with that, please help me.

I already saw on Stack Overflow a way to do it manually thanks to gsub, and in general pattern with the code, but this code is dissimilar, and doing it manually will not make it work.

Furthermore, there is also another command via the Rmongo package:

query2 = dbGetQueryForKeys(mg1, 'address_history', '{}', '{address:1}')

where I can return the variable that I want. Unfortunately, because this is a nested file, it also cannot find the variables that are not in the top of the nest.

Is there another command or another package that I can use? I am open to any other opportunity to get this dataset (very large) into an R data frame, so I can make any inferences.

Thank you very much!

index.php:

In pymongo I'm doing a sort query like this:

I extracted some data from a mongo database using the RMongo library. I have been working with the data with no problem. However, I need to access a field that was saved, originally in the database, as JSON. Since rmongodb saves the data as data frame, I now have a large character vector of length 1:

I am using RMongo library for storing data in mongo from R. This is the script I am trying to run:

I am trying to export datapoints from mongodb. I was unable to directly connect it to rstudio unfortunately. So from the query outcome I created a text file and attempted to read it as text file in R.