thrift-server | Thrift core library in TypeScript

In order to behave as a Java library, a rule needs to provide certain information. Evidently, thrift_java_libary does not. There is a blog post on how to implement this if you're a rule writer.

However, assuming you're just using the Thrift rules and don't want to modify them: are the Thrift rules generating a .jar file as output? If so, you could do:

5gb heaps (-Xmx5120M) are lower than the minimum recommended heap size of 8gb. While you can probably tune it to make it work the database is not designed with its defaults to run in that limited of a heap. Probably the fastest and most likely way to help is to set concurrent_reads and concurrent_writes to 8 or 4. Then concurrent_compactors and concurrent_validations (if more recent version) to 1.

If you set the -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath=/somewhere/writable then on the OOM it will dump the heap and you can open it up and see what is cause. If you get it you can post heap histogram (using jmap on the dump) and people may be able to give more direction - but it may require actually poking around with a tool like yourkit or MAT to know exact cause.

If thats a deadend you can use async profiler

The problem there is that it seems that you have to use one diferent Transport object for each thread. This is probably related to Thrift's implementation!

Reference here : http://grokbase.com/t/thrift/user/134s16ks4m/single-connection-and-multiple-threads

Spark Thrift Server is a Hive-compatible interface for Spark.

That means, it creates implementation of HiveServer2, you can connect with beeline, however almost all the computation will be computed with Spark, not Hive.

In the previous versions, query parser was from Hive. Currently Spark Thrift Server works with Spark query parser.

Apache Thrift is a framework to develop RPC - Remote Procedure Calls - so there are many implementations using Thrift. Also Cassandra used Thrift, now it's replaced with Cassandra native protocol.

So, Apache Thrift is a framework to develop RPCs, Spark Thrift Server is an implementation of Hive protol, but it uses Spark as a computation framework.

For more details, please see this link from @RussS

I was able to connect to Spark Thrift Server from JDBC client with following workaround.

Spark Thrift Server is running on port 10002, which is not publicly accessible as documented here in Azure HDInsight docs. Thus, here is alternative way to connect to Spark SQL from local JDBC client.

Background:

I connected to cluster head node via SSH.

Looking at org.apache.hive.service.cli.operation.LogDivertAppender.(LogDivertAppender.java:166) shows the following line:

Thanks for the hint from Mark Rotteveel. Now I understand better and let me post an answer to my own question.

Implementation of absolute() is optional

As specified by the Interface of ResultSet#absolute() (link), the implementation for absolute() is optional -- especially when the result set type is TYPE_FORWARD_ONLY.

Workaround

In my case, the result set comes from a Spark Thrift Server (STS) so I guess it is indeed forward-only. So the question became how to instruct my application to NOT making a call to absolute(), which is basically for cursor movement.

SmartGWT-specific answer

For SmartGWT, this is controlled by a property called sqlPaging, which we can specified for an OperationBinding. The right value to use seems to be dropAtServer (more reference here). So I set my SmartGWT DataSource XML file to something like this

It turned out the issue was that in a subproject, one of our teammates imported org.scalatest in the maven pom.xml and didn't set the scope to test... Aside from the crazy dependency conflicts this created, it was somehow overriding my version of scalatest. By setting the scope of scalatest to "test" in the subproject, this issue was fixed. i.e. Adding test fixed things; see the example below.

Actually, someone just helped me out. Here is the link of wildfly forum about solving this problem: https://developer.jboss.org/thread/274131