R-conference | 中国R会议官方网站 China R Conference Homepage | Web Framework library

When setting up a concurrent pipeline, a good guideline to follow is to always first set up and instantiate the listeners that will execute concurrently (in your case, crawlers), and then start feeding them data through the pipeline (in your case, the urlChannel).

In your example, the only thing preventing a deadlock is the fact that you've instantiated a buffered channel with the same number of rows that your test file has (1000 rows). What the code does is it puts URLs inside the urlChannel. Since there are 1000 rows inside your file, the urlChannel can take all of them without blocking. If you put more URLs inside the file, the execution will block after filling up the urlChannel.

Here is the version of the code that should work:

It looks like knitr uses pandoc-citeproc for citation rendering, and that you can preserve casing by using another set of brackets around the title. See https://github.com/jgm/pandoc-citeproc/issues/269.

So instead of

The simplest way is to select a node that encloses both of the nodes you want for each row, then iterate over them, pulling out both of the nodes you want at once. purrr::map_df is handy for not only iterating, but even combining the results into a nice tibble:

Your xpath should be header/h3/a, not /header/h3/a. The leading slash would imply you want to start at the root of the tree again, not the current node.

I am new to R but something like the following where you define a function to retrieve the row info as a dataframe from a given url. Loop over how ever many pages you want calling the function and merging returned dfs into one big df. As nodeLists are not always the same length e.g. not every listing has a telephone number, you need to test for whether element is present in a loop over the rows. I use the method in the answer by alistaire (+ to him)

I am using css selectors rather than xpath. You can read about them here.

Given the # of possible pages I would look into using an http session. You get the efficiency of re-using a connection. I use them in other languages; from a quick google it seems R provides this, for example, with html_session.

I would welcome suggestions for improvement and any edits for correcting indentation. I'm learning as I go.

If you inspect the element you see that the class of the error is shiny-output-error, so you change the color with css, see below:

Css code would read: