blog-examples | Sample applications for blog posts on https : //andrewlock.net | Blog library

I have some large data sets which I would like to compress before I send to my client. The compression works.

Utilizing this bit of code which turns my data into a nice, small base64String:

Example: string mytest = "This is some test text.";

Here is a simple program that:

1. Writes records into an Orc file
2. Then tries to read the file using predicate pushdown (searchArgument)

Questions:

1. Is this the right way to use predicate push down in Orc?
2. The read(..) method seems to return all the records, completely ignoring the searchArguments. Why is that?

Notes:

I have not been able to find any useful unit test that demonstrates how predicate pushdown works in Orc (Orc on GitHub). Nor am I able to find any clear documentation on this feature. Tried looking at Spark and Presto code, but I was not able to find anything useful.

The code below is a modified version of https://github.com/melanio/codecheese-blog-examples/tree/master/orc-examples/src/main/java/codecheese/blog/examples/orc