geomesa | big geo-spatial data

When you run an export with a query hint for stats, GeoMesa will always run a query. If you want to use the cached statistics, use the stats-* commands instead. In code, you'd use the stats method which all GeoMesa data stores implement.

This will involve several steps. You probably want to back up your data before attempting this, in case something goes wrong. First, you want to use updateSchema to add the new index and remove the old one, and set the existing indices to "read only" mode. You can use the GeoMesa CLI scala-console command to run the following:

The query is slow because it's doing a full scan of all your data. If you look at the explain output, you'll see:

You can use the GeoMesa stats API for some of this. You can use the enumeration stat to get all unique values, and you can use the min/max stat to get the "last" value.

The stats API does not bring back the entire feature, so if you want that you would need to do a secondary lookup or simply iterate over results as you are doing.

However, since Cassandra does not have any server-side processing, using the stats API will not provide much advantage over your current client-side processing.

I would recommend you to install same version of geomesa-spark-jts_2.11 as given in this notebook.

To install geomesa-spark-jts_2.11:2.3.2 follow below steps:

Step1: Click on install library.

Step2: Select Maven, Search and install geomesa-spark-jts_2.11:2.3.2.

Step3: You can also download jar file and upload it to Library Source.

hadoop 3.1 not support this feature, need 3.2 update

For posterity, the setup that worked was:

• GeoMesa 3.1.1
• GeoServer 2.17.3
• Extract the geomesa-hbase-gs-plugin into GeoServer's WEB-INF/lib directory
• Run install-dependencies.sh (without modification) from the GeoMesa binary distribution to copy jars into GeoServer's WEB-INF/lib directory
• Copy the hbase-site.xml into GeoServer's WEB-INF/classes directory

You can ingest or export remotely just by running GeoMesa code on a remote box. This could mean installing the command-line tools, or using the GeoTools API in a processing framework of your choice. GeoServer is typically used for interactive (not bulk) querying.

There isn't any out-of-the-box solution for ingest/export via AWS lambdas, but you could create a docker image with the GeoMesa command-line tools and invoke that.

Also note that the command-line tools support ingest and export via map/reduce job, which allows you to run a distributed process using your local install.

Good question! It appears that there's a small error with this tutorial. The GeoMesaSparkKryoRegistrator is used for managing the serialization of SimpleFeatures in Spark.

This tutorial does not seem to use SimpleFeatures (at least as of August 2020). As such, this import is likely unnecessary. You ought to be able to progress by skipping that import and the registration of the GeoMesaSparkKryoRegistrator.

The imported module provides just the spatial types and functions necessary for achieving basic geometry support in Spark. To leverage GeoMesa's datastores in Spark, one would import a GeoMesa database-specific spark-runtime jar. Since those datastores use GeoTools SimpleFeatures, that jars would include the GeoMesaSparkKryoRegistrator, and its use would be similar to what is in that notebook and in the documentation on geomesa.org.