geotrellis | geographic data processing engine for high performance | Dataset library

Is there a way to extract/query latitude, longitude and elevation data from a tif file using RasterFrames (http://rasterframes.io/)?

Following the documentation, I did loadRF a tif file from the following site: https://visibleearth.nasa.gov/view.php?id=73934, however all I can see is generic information and don't know which RasterFunction to use in order to extract position and elevation or any other relevant information. I did try everything I can find in the API.

I did also try to extract temperature information using the following source as well: http://worldclim.org/version2

All I get is tile column with DoubleUserDefinedNoDataArrayTile and boundary (extend or crs).

RasterStack in R can extract this information according to this blog: https://www.benjaminbell.co.uk/2018/01/extracting-data-and-making-climate-maps.html

I need a more granular DataFrame such as lat,lon,temperature(or whatever data is embedded into the tif file).

Is this possible with RasterFrames or GeoTrellis?

I'm writing MapBox vector tiles using geotrellis vectorpipe.

see here for the basic flow: https://geotrellis.github.io/vectorpipe/usage.html

Typically GeoJson Features can have an id field, so that Features can be rolled up into FeatureCollections. I need to make use of this field, but vectorpipe doesn't (natively) have this capability.

This is the Feature type used, and you can see it only has space for 1) a Geometry and 2) a data object D (which ends up populating properties in the output). There is no spot for an id.

https://geotrellis.github.io/scaladocs/latest/index.html#geotrellis.vector.Feature

Upstream there's a method called writeFeatureJsonWithID() that does let you inject an id field into a Feature when writing GeoJson.

https://github.com/locationtech/geotrellis/blob/master/vector/src/main/scala/geotrellis/vector/io/json/FeatureFormats.scala#L41-L49

My question is this:

I have worked through the vectorpipe code (https://github.com/geotrellis/vectorpipe), and I can't figure out if/where the data ever exists as GeoJson in a way where I can override and inject the id, maybe using the writeFeatureJsonWithID() or something I write explicitly. A lot of conversions are implicit, but it also may never explicitly sit as json.

Any ideas for how to get an id field in the final GeoJson written to vector tiles?

EDIT

Right now I think the trick is going to be finding a way to override .unfeature() method here:

https://github.com/locationtech/geotrellis/blob/master/vectortile/src/main/scala/geotrellis/vectortile/Layer.scala

The problem is that the internal.vector_tile.Tile is private, so I can construct it without forking the project.

I am trying to process Sentinel 2 data from (example)

http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#tiles/10/S/EG/2016/10/12/0/

The jp2 files are not georeferenced, and I need to put all the jp2 files as bands in a geotif. I have googled aplenty and find no way to do this in Java or Scala.

I am pretty familiar with Geotools, i've done a lot of geotif processing with geotools, but I can't figure out how to

a. Make a geotif raster out of a jp2 file (given coords for the envelope), and

b. take those and make a multiband geotif out of them

I am decent with Scala, so I've looked at geotrellis, but don't see a solution with that either.

Does anyone know how to make geotifs out of JP2 files (given a polygon) and then make a multiband geotif?

thanks

I want to submit a Spark job on a remote YARN cluster using the spark-submit command. My client is a Windows machine and the cluster is composed of a master and 4 slaves. I copied the Hadoop config files from my cluster to the remote machine, namely core-site.xml and yarn-site.xml and set the HADOOP_CONF_DIR variable in spark-env.sh to point to them.

However, when I submit a job using the following command :

I'm using IntelliJ IDEA v2017.3.2 Community Edition and it seems to keep pulling the wrong repo versions for Scala plugins.

I've tried setting scalaVersion := "2.11.8" in build.sbt and idea.sbt, but it keeps trying to pull from 2.12 plugin repos.

My build.sbt is as follows:

I load a set of point from a CSV file to a RDD:

I'm using geotrellis to load geotiff rasters from Landsat 8 that are sitting on S3. However, they are stored on a per-band basis. I can use S3GeoTiff class to load individual bands, e.g.:

Given an RDD[(SpatialKey, Tile)] in GeoTrellis, how do I compute the aggregate KeyBounds[SpatialKey]?

I am working with an API written in Scala that uses SBT to build it into a JAR file. I deploy the JAR file to a virtual machine (VM) and I can execute it successfully - the API starts up and handles requests.

Now I am trying to use the IntelliJ (v 2016.3.4) remote-debug configuration to place breakpoints in the source Scala code. Here's what I have done:

1) Open my source code as a Scala SBT project in IntelliJ .

2) Opened VM port 5005

3) Executed JAR on VM:
java -noverify "-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005" -cp /home/vagrant/shared/build/geotrellis-chatta-demo-assembly-0.1-SNAPSHOT.jar geotrellis.chatta.Main

I get this output:
Listening for transport dt_socket at address: 5005

4) Added a remote-debug configuration in IntelliJ and started it. The IntelliJ console shows me this:
Connected to the target VM, address: 'localhost:5005', transport: 'socket'

However, none of my breakpoints work.

Am I missing something in the above steps? I'm very new to Scala, haven't worked with Java seriously in years, and am just starting with IntelliJ.

UPDATE: IntelliJ debugger is definitely working - if I pause it, the API stops responding to requests until it becomes un-paused. But no stops on breakpoints.