Node.js exec with wget
Problem
I'm trying to download a lot of files using nodejs and the exec
command, simplified like this:
var cmd = 'wget -O output.csv URL';
var child = exec(cmd, function(err) {
console.log('DONE');
});
However, the callback is triggered before the file was actually downloaded through wget
, leading to a file that contains garbage like '��0O�6D�1n�]v�����#�'. Shouldn't the callback be triggered once wget is done? When running the same command on the command line it takes rougly 5 seconds, since the file has several MB.
Btw: I'm not using the request
module since it's slower and I ran into emitter listener issues (EventEmitter memory leak detected. 11 listeners added
).
Thanks!
Solution
This will involve some debugging.
Can you please try running your script as:
var cmd = 'wget -O output.csv URL';
var child = exec(
cmd,
function (error, stdout, stderr) {
console.log('stdout: ' + stdout);
console.log('stderr: ' + stderr);
if (error !== null) {
console.log('exec error: ' + error);
}
}
);
It would be interesting to see what stdout and stderr say.
Right, you provided me your stderr
which said:
http://productdata.zanox.com/exportservice/v1/rest/22791753C32335607.csv?ticket=BC4B91472561713FD43BA766542E9240AFDD01B95B123E40B2C0375E3A68C142
This URL the command line gets is missing everything after the ampersand (&
character). This indicates a problem with escaping.
To get around this try replacing \&
with \\\&
.
Discussion
View additional discussion.