node-worker | An implementation of the WebWorker API for node.js | Runtime Evironment library
kandi X-RAY | node-worker Summary
kandi X-RAY | node-worker Summary
An implementation of the WebWorker API for node.js
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Master process for a worker .
- Work with child process .
- debug helper
node-worker Key Features
node-worker Examples and Code Snippets
Community Discussions
Trending Discussions on node-worker
QUESTION
I recently read about Node's "worker_threads" module that allows parallel execution of Javascript code in multiple threads which is useful for CPU-intensive operations. (NOTE: these are not web workers made by Chrome in the browser)
I'm building a feature where I need to do a massive amount of Postgres INSERTs without blocking the browser.
The problem is: in my Javascript files where I instantiate the worker, I'm not allowed to import anything, including native Node modules or NPM libraries like Knex.js which is necessary to do database queries. I get an error that says: Cannot use import statement outside a module as soon as the file is executed.
I've tried putting the worker code in another file with an import statement at the top (same error). I've tried giving the Knex object to workerData but it cannot clone a non-native JS object.
I'm out of ideas- does anyone know how to interact with a database in a worker thread if we can't import any NPM libraries?!?!
...ANSWER
Answered 2020-Jul-16 at 00:48It is of course possible, but it's a very bad idea.
Database drivers are already asynchronous and non-blocking of the JavaScript thread. Moving your insert calls to a separate thread as you propose will not only get you no performance gains, it will actually decrease overall performance because of the overhead involved with interthread communication:
- Synchronization and message passing is not free
- JavaScript uses structured cloning when moving data between threads. This means all your
rowsToInsertmust be copied, which is (relatively) expensive.
Generally, the only time it's really appropriate to use JS threads is when your JavaScript code is performing CPU-intensive work. The node docs say as much right at the top:
Workers (threads) are useful for performing CPU-intensive JavaScript operations. They will not help much with I/O-intensive work. Node.js’s built-in asynchronous I/O operations are more efficient than Workers can be.
This means if you're doing a lot of parsing, math, or similar, it may be appropriate to do the work in a thread. However, simply shoveling data from one place to another (ie, I/O) is not a good candidate for a thread — after all, node's design is tuned to be efficient at this kind of work.
You don't say where your rowsToInsert come from, but if it's coming in from HTTP request(s), a thread is the wrong thing to use. However, if you're parsing eg a CSV or JSON file on the server, it may be worthwhile to do that in a thread, but it's important that the thread does all the work (so memory need not be moved between threads). The message you post to the worker should just be "process the file located at /foo/bar.csv", and then the worker thread does the rest.
The error you're getting is the same that you'd get without worker threads: you're trying to use import in a regular, non-module JS file. Either rename the worker file to *.mjs or use require('knex') instead.
node's ES module documentation goes into detail about how import differs from require.
QUESTION
My code is throwing an exception when I'm trying to get the test coverage of my project using nyc.
I'm also using the library node-worker-threads-pool to create a pool of worker threads.
I created a minimal example to emulate the behavior. With this example the behavior is the same, when I execute the test, everything works, but when I try to get the coverage I have an exception.
This is the file where I create the new thread
...ANSWER
Answered 2020-Jun-24 at 15:00So, after doing a little of research I understand that basically nyc creates variables with random names to count the coverage and insert those inside of all the functions and it uses cloture to access them.
The problem is that it's not possible to use cloture inside of a thread. It's imperative to use the workerData property to send the arguments to the Worker constructor. That's why I was having the ReferenceError exception.
Worker Threads
I didn't really found a solution. But I did find an alternative to nyc that also uses istambuljs which is c8. I no longer have those exceptions with c8.
QUESTION
I had been trying to separate some work that's done in my program in a different thread. One of the functions needs to return a stream to the main thread but I'm having the following exception:
...ANSWER
Answered 2020-Jun-22 at 14:15Short version: you can't.
IPC in node is handled through some black box, but what we know is that message objects are serialized before sending and deserialized once received: you can't serialize a Stream because it is based on underling level (a socket, a file descriptor, custom read and write functions, etc) which can't be serialized/deserialized.
So you are forced to exchange serializable data.
Taking a look at html-pdf I think an easy way to convert your program is to use pdf.toBuffer: rather than trying to send a Stream to main thread and reading it in main thread to obtain a Buffer, you should send a Buffer to main thread and than use it as is.
Hope this helps.
QUESTION
I'm trying to use Node's worker_threads using node-worker-threads-pool (a wrapper) package and its DynamicPool.
My issue is that executing a thread giving a function which has imports won't work because that function will be undefined.
ANSWER
Answered 2020-Jun-18 at 20:45It appears to be a limitation of the library, and has nothing to do with modules:
Notice: If
taskis a function, you can not use closure in it! If you do want to use external data in the function, you can use workerData to pass some cloneable data.
This sounds a lot like they are stringifying the function and creating a worker from that code only.
Use a worker file instead.
QUESTION
I have an application that looks like this one: https://github.com/heroku-examples/node-workers-example
In short, I have 2 processes:
- Server: it server US, handles requests and adds them to Redis
- Worker: it pulls requests from Redis and works on them
Should I use only one Docker image for both processes or should I have 2 docker images (one for the server and the second for the worker)? What is the best practice?
I personally think, it's better to have 2 images. In this case, can my project structure be like this one:
...ANSWER
Answered 2020-Mar-24 at 11:57Disclaimer: this is very opinionated response, but author is asking for an opinion.
You can do one or two but it all depends on how you wanna schedule it.
If you want to stay flexible in the amount of processes you want for each I would go with two docker images otherwise you'll need to each time spin a fixed amount of each or you'll need to tweak that setting via env variables or via other means...
Hence one for the frontend part and one for the background process.
As you are having two different images, I usually prefer to separate that in two distincts projects but that's a matter of taste. Even though because of how NodeJS manages dependencies (node_modules) it's easier to have 2 distincts folders when the dependencies are very different.
I would go with following:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install node-worker
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
