data-hub | Data delivery & consumption platform for Pimcore | Continuous Deployment library

It is possible to issue such a query, as long as there is a date or dateTime being tracked and indexed.

For any MarkLogic database, there is a database option to enable "maintain last modified", which will set and maintain a document property prop:last-modified that can be indexed and range-queries applied.

Your example table was "data-hub-STAGING". If you are using a MarkLogic Data Hub, the datahubCreatedOn field is available with a dateTime range index.

https://docs.marklogic.com/datahub/5.6/flows/about-flows.html#about-flows__pre_ay4_frh_ypb

For every content object outputted by a Data Hub step, regardless of the step type, Data Hub will add the following document metadata keys and values to the document wrapped by the content object:

• datahubCreatedOn = the date and time at which the document is written

Of course, any other date/dateTime element, JSON field, or document property can also be indexed and then used to filter docs as well.

Then the appropriate range query can be applied to a search in JavaScript, XQuery, Optic, SQL, etc.

For example, in JavaScript searching by field datahubCreatedOn with cts.fieldRangeQuery():

This is long, because the solution is mostly to dig around and figure out how you want to work with this file. There are a few potential issues here:

• Like I mentioned in your first post, the file might be malformed as far as the structure for a GeoJSON file. Some methods of reading it throw a warning that the last line is incomplete; a trailing empty line is part of the expected structure in json files.
• The nodes and interconnection attributes are arrays, and I think R functions don't know what to do with those; sf functions know to read these as list-columns but that might not be available for geojsonio ones.
• There's a mix of geometry types.

That last point is the big one, because it prevents you from using sp::Spatial* classes—those are only for single geometry types (SpatialGrid, SpatialPolygons, etc.). This file has both points and lines mixed together, so you need something like sf that can hold mixed geometry types.

A few ways I tried reading this, some of which return lists that you could further dig into, some return sf objects:

You can use cts.uris() with a cts.jsonPropertyValueQuery() to obtain the list of document URIs that have a JSON property named name with the value "John", and then iterate over the list of URIs and call xdmp.documentDelete()

The name values are not being de-duplicated by the Set because they are text() nodes. You need to get the .valueOf() those text nodes and then the strings will be de-duplicated.

The purpose of cts.search is to run a search that returns entire documents. The cts.jsonPropertyValueQuery identifies a criterion for selecting a document. After you've run the search, you'll be able to pull out whatever you'd like from that document. I think this should do it:

With a JavaScript module, you can use a cts.jsonPropertyValueQuery() in order to target the documents that have the source property with the value source1, and then use cts.estimate() to get the count of docs:

You need quite a few pieces to get it fully working in offline mode:

• local maven repo in buildscripts and repositories
• a copy of gradlew that has been edited to use a local dir for gradle home, and has its own copy of gradle dist (which kind of beats its purpose, but not entirely)
• and to make plugins work properly, you also need a settings.gradle with pluginManagement.repositories pointing to local maven as well

This PR in grove-ml-gradle gives best overview of necessary changes to my opinion: https://github.com/marklogic-community/grove-ml-gradle/pull/2/files

HTH!

I think filter is currently only supported on text fields, I have tested title and email. Date filters are not working with same error above. Also to note, text search is finicky, works only when using wildcards example:

This is unlikely to be a MarkLogic or Gradle issue, more likely Kerberos cannot resolve your Proxy host to a valid address in the /etc/krb5.conf configuration files. You will need to check that your Kerberos configuration is set up to serve domains based on the Proxy server address.