concise way to transform a node.js stream?
Problem
I'm trying to convert a newline-separated JSON file to CSV (kind of a dumb task, but it doesn't matter), and am curious how I can easily do this in node.js. What I have so far is,
var csv = require('csv'),
fs = require('fs'),
JSONStream = require('JSONStream');
var stream = fs.createReadStream('input.json', {encoding: 'utf8'})
.pipe(JSONStream.parse());
csv().from(stream).to('out.csv');
This works, but doesn't let me express ordering on the keys. For example, if I have an input file like this,
{"foo": 1, "bar": 2}
{"bar": 3, "foo": -1}
I'd like to add a tranformer function,
function(line_object) { return [line_object.foo, line_object.bar]; };
so that my output will be lines 1, 2
and -1, 3
, corresponding to foo
being column 1 and bar
being column 2, instead of the current behavior 1, 2
and 3, -1
.
However, all of the examples online seem rather verbose for adding transformers. They seem to create require('stream').Transform
objects, and then set the _transform
function on those objects. Question: Is there a cleaner / more concise way to do this? Thank you very much, and please forgive my ignorance as this is pretty much the first node.js
code I've written.
Solution
event-stream
might be useful:
var fs = require('fs');
var csv = require('csv');
var JSONStream = require('JSONStream');
var es = require('event-stream');
es.pipeline(
fs.createReadStream('input.json'),
JSONStream.parse(),
es.map(function(data, next) {
next(null, [ data.foo, data.bar ]);
}),
csv().to('out.csv')
);
Discussion
View additional discussion.