public-apis | A collective list of free APIs | REST library

A big reason these terms are confusing is because they are keywords in Java and they have meaning in a general, language-agnostic context. What the terms mean in the general context of programming is slightly different from what the specific keywords mean in Java.

In general, the term interface is synonymous with abstraction. When you read, "Program to an interface" it means program to an abstraction. In Java, the interface keyword indicates a specific type of stateless abstraction, which is contrasted with abstract class. In general, both of these are abstractions and thus both are programming interfaces.

In general, the term public indicates any part of code that is visible to and used by clients outside of the development team. In Java, the public keyword indicates a specific access level between classes and members. Since clients may view and use protected members as well, these may also be considered public in the general context of programming. Effective Java uses the term exported to combine public and protected members. Note that in Java 9+ modules can be used to control access, so even public classes may not be exported and thus may not be public in the general sense.

The term API is not a Java keyword, so at least we have no language-specific ambiguity to contend with. It is typically used to mean public API, which is synonymous with exported API. It is the sum of everything that you (the developer) allow your clients to view and use. Occasionally, the term internal API is used to indicate the opposite: classes and members that you do not allow your clients to view or use.

Your code could be much simpler and more resilient:

Meditate on this:

Which results in:

The issues with your code are:

• Using search('some selector')[0] is the same as at('some selector') only the second is cleaner resulting in less visual noise.

  There are other, more subtle, differences as far as what search returns in comparison to at, which is covered in the documentation. I highly recommend reading and experimenting with their examples as knowing which to use when can save you headaches.

• Relying on absolute XPath selectors: An absolute selector is very fragile. Any change to the HTML will have a high-likelihood of breaking. Instead, find useful nodes to check that are unique, and let the parser find them.

  Using a CSS selector 'article li a' jumps through all the nodes until it finds the "article" node, looks inside it for the child "li" and following "a". You can do the same thing with XPath, but it's visually noisy. I'm a big fan of keeping my code as easy to read and comprehend as possible.

  Similarly, at('article table') finds the first table under the "article" node, then search('tr') finds the embedded rows in only that table.

  Because you want to skip the table header [1..-1] slices the NodeSet and skips that first row.

• map makes it easier to build a structure:

assigns the fields to rows in one pass through that loop of rows.

values is assigned with the text of the NodeSet of every "td" node's text.

is taking the values from the cells and adding a second array containing the category and link from the row.

My example code is basically the same template every time I scrape a page with a table. There'll be minor differences, but it's a loop over the table extracting the cells and converting them to a hash. It's even possible, on a cleanly written table, to automatically grab the keys of the hash from cell text in the first row of the table.