Get Even More Visitors To Your Blog, Upgrade To A Business Listing >>

SSH2.SFTP node.js backup script freezes with large files

SSH2.SFTP node.js backup script freezes with large files


I'm trying to create a script to transfer files to my NAS for backup purposes. I'm using the SSH2 module ( to set up a SFTP connection. For smaller files all goes well, with larger files (>500MB) it often just freezes somewhere between 70% and 100% completion. The fastPut method, which i use for the transfers, suddenly just stops transfering, doesn't trigger any events anymore.

I've taken out the relevant piece of code:

var uploadUpdate=true;

sftp.fastPut(localPath+'/'+upload, receiver.path+'/'+newDir+'/'+upload, {step:function(totalTransferred, chunk, total){
    if(uploadUpdate){ //since this event will be triggered a lot I figured I should keep some time (5 sec) between the logs
        console.log("- Progress '"+upload+"' => "+(Math.floor(totalTransferred/total*10000)/100)+"%, "+totalTransferred+" of "+total+" bytes, chunk: "+chunk);
}}, function(err){
        if(!fileErrors[upload]) fileErrors[upload]=0;

            console.log("'"+upload+"' failed 5 times... skipping now...");
            console.log("Upload of '"+upload+"' failed, re-add to queue.");

Does anyone have any idea what might cause this problem?

Problem courtesy of: NullCrayfish


I've ended up splitting the files into smaller chunks and sending them, then whenever for a specific amount of time there had been no progress i restart the connection and start transfering where it died before.

Not very pretty but it gets the job done

Solution courtesy of: NullCrayfish


View additional discussion.

This post first appeared on Node.js Recipes, please read the originial post: here

Share the post

SSH2.SFTP node.js backup script freezes with large files


Subscribe to Node.js Recipes

Get updates delivered right to your inbox!

Thank you for your subscription