basic-ftp | FTP client for Node.js , supports FTPS | HTTP library

Inside the FtpManager.downloadFolder function, I see recursive calls to the same downloadFolder method with an await. Your Maximum call stack exceeded error could come from there, since your initial call will need to keep everything in memory while traversing all subdirectories.

Instead of awaiting everything recursively, you could setup a queue system, with an algorithm like this:

• Add the current folder to a queue
• While that queue is not empty:
  • Get the first folder in the queue (and remove it from it)
  • List all entries in it
  • Download all files
  • Add all subfolders to the queue

This allows you to download a lot of folders in a loop, instead of using recursion. Each loop iteration will run independently, meaning that the result of the root directory download won't depend on the deeeeeep file tree inside it.

There are plenty of queue manager modules for NodeJS, which allow you to have concurrency, timeouts, etc. One I've used in the past is simply named queue. It has a lot of useful features, but will require a little more work to implement in your project. Hence, for this answer, I used no external queue module, so that you can see the logic behind it. Feel free to search for queue, job, concurrency...

I wanted to implement that logic directly into your own code, but I don't use Typescript, so I thought I'd make a simple folder copy function, which uses the same logic.

Note: For simplicity, I've not added any error handling, this is just a proof of concept! You can find a demo project which uses this here on my Github.

Here is how I've done it:

The problem is that you are getting lost when creating an async function inside your handler. Since example() is async, it returns a Promise. But you don't await on it, so the way it has been coded, it's kind of a fire and forget thing. Also, your Lambda is being terminated before your callbacks are triggered, so even if it got to download you would not be able to see it.

I suggest you wrap your callbacks in Promises so you can easily await on them from your handler function.

I have managed to make it work: I have used https://dlptest.com/ftp-test/ for testing, so change it accordingly. Furthermore, see that I have uploaded the file myself. So if you want to replicate this example, just create a readme.txt on the root of your project and upload it. If you already have this readme.txt file on your FTP server, just delete the line where it uploads the file.

Here's a working example:

finalData is only defined within example which returns it, but you're not assigning this to anything. Combined with Luca Kiebel's comment, try adding

const finalData = await example();

then log that out.

Because finalData is defined within the function example, it is only available within that function and any function defined within that function.

You Don't Know JS explains this better than I can

Pretty sure your fs.createWriteStream() has to use an absolute path to /tmp in Lambdas. Your actual working directory is var/task not /.

Also, if you're using fs.createWriteStream() you'll need to wait for the finish event before reading from the file. Somethign like this...