hbase- | https

The above issue has been already filed under the below issue.

https://issues.apache.org/jira/browse/HBASE-22784

Upgrading to 1.4.11 fixed the zknode grows exponetially

I was able to solve the issue by using s3a. EMRFS libs used in the emr are not public and cannot be used outside EMR. Hence I used S3AFileSystem to access hbase on S3 from my ecs cluster. Add hadoop-aws and aws-java-sdk-bundle maven dependencies corresponding to your hadoop version. And add the below property in my core-site.xml.

After struggling with Apache Atlas for a while, I found 3.0.0 Snapshot version very buggy! Therefore I have decided to build and install Apache Atlas 2.1.0 RC3.

Prerequisite:

Make sure you have installed java on your machine. In case it is not installed on your computer, you can install it using the following command in Linux:

sudo apt-get install openjdk-8-jre

Then JAVA_HOME should be set:

After 5 days of hustle, I learned what went wrong. Posting my solution here. Hope it can help some of the other developers too. Also would like to thank @VV_FS for the comments.

In my scenario, I used virtual machines which I burrowed from an external party. Therefore, there were certain firewalls and other security measures. In case if you follow a similar experimental setup, these steps might help you.

To set up HBase cluster, follow the following tutorials.

1. Set up Hadoop in distributed mode.

Notes when setting up HBase in fully distributed-mode:

• Make sure to open all the ports mentioned in the post. For example, use sudo ufw allow 9000 to open port 9000. Follow the command to open all the ports in relation to running Hadoop.

1. Set up Zookeeper in distributed mode.

Notes when setting up Zookeeper in fully distributed mode:

• Make sure to open all the ports mentioned in the post. For example, use sudo ufw allow 3888 to open port 3888. Follow the command to open all the ports in relation to running Zookeeper.
• DO NOT START ZOOKEEPER NODES AFTER INSTALLATION. ZOOKEEPER WILL BE MANAGED HBASE INTERNALLY. THEREFORE, DON'T START ZOOKEEPER AT THIS STAGE.

1. Set up HBase in distributed mode.

• When setting up values for hbase-site.xml, use port number 60000 for hbase.master tag, not 60010. (thanks @VV_FS to point this out in the earlier discussion).

• Make sure to open all the ports mentioned in the post. For example, use sudo ufw allow 60000 to open port 60000. Follow the command to open all the ports in relation to running Zookeeper.

[Important thoughts]: If encounters errors, always refer to HBase logs. In my case, hbase-mater-xxxxx.log and zookeeper-master--xxx.log helped me to track down exact errors.

Welcome to dependency hell! The jar file phoenix-5.0.0-HBase-2.0-client.jar contains the equivalent of 170 jar files (listed below as Maven groupId:artifactId:version). Specifically it contains Jersey 1.19, while you have already Jersey 2.15. Whenever you have multiple version of the same library on your classpath you will get linkage errors as the one in your question. One Jersey library (probably 2.15) tried to load a dependency, but the dependency came from another version (probably 1.19).

The solution is not to use fat jars like phoenix-5.0.0-HBase-2.0-client.jar, but use Maven or another system to deal with dependencies. They might still get it wrong (e.g. the com.sun.xml.bind and org.glassfish.jaxb's version of jaxb-core contain the same classes, but Maven does not know that), but the result is much better, than manual dependency management.

In your case, if you just need the org.apache.phoenix.jdbc.PhoenixDriver you need to include the phoenix-core artifact and let Maven do the rest.

You'll end up with many useless dependencies (e.g. you can immediately exclude the jetty-* dependencies, since those are the libraries of the Jetty Servlet container, as well as hbase-server if you are connecting to an external one), but hopefully there will be no conflicts.

I suspect NPE here happens because HBaseContext should be properly initialized before HBase-Spark connector can lookup in hbase:meta a table you're referencing, and create a datasource. I.e. follow the Customizing HBase configuration section from your link, something like:

For posterity, the setup that worked was:

• GeoMesa 3.1.1
• GeoServer 2.17.3
• Extract the geomesa-hbase-gs-plugin into GeoServer's WEB-INF/lib directory
• Run install-dependencies.sh (without modification) from the GeoMesa binary distribution to copy jars into GeoServer's WEB-INF/lib directory
• Copy the hbase-site.xml into GeoServer's WEB-INF/classes directory

You can ingest or export remotely just by running GeoMesa code on a remote box. This could mean installing the command-line tools, or using the GeoTools API in a processing framework of your choice. GeoServer is typically used for interactive (not bulk) querying.

There isn't any out-of-the-box solution for ingest/export via AWS lambdas, but you could create a docker image with the GeoMesa command-line tools and invoke that.

Also note that the command-line tools support ingest and export via map/reduce job, which allows you to run a distributed process using your local install.

If you have a CMD command in your Dockerfile, that replaces the CMD commands of any of its base images. The CMD specifies what process runs in the container, and there can only be one. The latest declared one wins.

If all you need to do is start the processes, you can use the parent image's CMD by simply omitting CMD in your Dockerfile. If you need to make changes, copy the CMD from the base image and make the necessary changes.