Get Even More Visitors To Your Blog, Upgrade To A Business Listing >>

concise way to transform a node.js stream?

concise way to transform a node.js stream?

Problem

I'm trying to convert a newline-separated JSON file to CSV (kind of a dumb task, but it doesn't matter), and am curious how I can easily do this in node.js. What I have so far is,

var csv = require('csv'), 
    fs = require('fs'),
    JSONStream = require('JSONStream');

var stream = fs.createReadStream('input.json', {encoding: 'utf8'})
    .pipe(JSONStream.parse());
csv().from(stream).to('out.csv');

This works, but doesn't let me express ordering on the keys. For example, if I have an input file like this,

{"foo": 1, "bar": 2}
{"bar": 3, "foo": -1}

I'd like to add a tranformer function,

function(line_object) { return [line_object.foo, line_object.bar]; };

so that my output will be lines 1, 2 and -1, 3, corresponding to foo being column 1 and bar being column 2, instead of the current behavior 1, 2 and 3, -1.

However, all of the examples online seem rather verbose for adding transformers. They seem to create require('stream').Transform objects, and then set the _transform function on those objects. Question: Is there a cleaner / more concise way to do this? Thank you very much, and please forgive my ignorance as this is pretty much the first node.js code I've written.

Problem courtesy of: gatoatigrado

Solution

event-stream might be useful:

var fs         = require('fs');
var csv        = require('csv');
var JSONStream = require('JSONStream');
var es         = require('event-stream');

es.pipeline(
  fs.createReadStream('input.json'),
  JSONStream.parse(),
  es.map(function(data, next) {
    next(null, [ data.foo, data.bar ]);
  }),
  csv().to('out.csv')
);
Solution courtesy of: robertklep

Discussion

View additional discussion.



This post first appeared on Node.js Recipes, please read the originial post: here

Share the post

concise way to transform a node.js stream?

×

Subscribe to Node.js Recipes

Get updates delivered right to your inbox!

Thank you for your subscription

×