Posted on Oct 20 • Originally published at deadsimplechat.com In this article we are going to learn how to handle CSV files on the backend with NodeJs and papaparse.We are going to do this with the help of real life examples, so that you have an understand of how to work with CSVsTo learn how to Parse CSV files with Papaparse in NodeJs we assume you have the following knowledge and prerequisitesand that's it. That is all you need to know to get along with this tutorialyou already have the node installed on your computer. Now create a new directory called csv-papaparse-node and set up a new project withnow open the project in your text editor. I am using VSCode, you can use anything you wantthen install the papaparse like soNow that we have installed the papaparse, we are going to parse a simple csv in the basic example belowIn this section we are going to parse a simple CSV that is already in the local folder inside the server.Note: In the sections after this we will look at how to create an expressjs server and send CSVs to the server parse them and send them back to the client.Now, we need the sample CSV that we are going to parse. In the root folder create a csv file with the following contentsNow let us take it step by step.*Step 1: * Create index.js fileIn the root folder create a new file and name it index.js we are going to write the Code to parse the csv in this fileStep 2: Importing the required Node moduleswe are going to need the fs module and the papaparse module so import them in your fileStep 2: Reading the fileNow read the file with the help of the fs moduleStep 3: parse the dataNow parse the data using the papaparse moduleStep 4: log the data to the consoleLastly log the data to the console so that you can verify that the file has been parsedThe complete index.js file looks like thisto run the code type the below command in your terminaland you can see the parsed data in your console likeWhile handling large CSV files many problems can arise like application running out of memory or application crashing that might lead to loss of workStreams are a NodeJs feature that allow you to read and write the files in chunks, thus minimizing memory usage and optimizing the processing of filesNow, using the above example to illustrate the point, open the index.js file and edit the papaparse code to use streams likeThe complete index.js code looks likeyou can run the code with the following codeThis updated code with streams can now handle even large csv files. You can try this by downloading a large csv file from the internetIn many real world scenarios we need to send the CSV to the server and need the parsed data back from the server.In this section we are going to achieve this. To do this we are going to build a express js server and use Postman to send and receive the data back from the server.Step 1: Installing required dependencies and importing themLet us install the express server and the multer package to handle multipart form datathen import the dependencies likeHere we have specified the destination folder as well where the files will be storedNow let us create a POST route where we will send the csv to be parsed, we are naming the csv csvFileNow we will add the parse logic to the POST endpoint to parse the incoming csv filewrite this code at the end of index.js to start the server.The complete code looks like this:to run the code simply open the terminal and type the below codeyou will see the : server running on port 3000 on your console.In this section we are going to test with postman whether the code is properly parsing the csv and sending the data to the client or not.Open the postman app and create a POST request to send to this urlNow, select the body tag and set the content type to multipart/form-data and then set the key to csvFile then set the value to the file that you are going to send to the server.this looks something likeNow press the send button this will send the csv file to our local server and the server will return the parsed data back to the postman appYou can also check the files being stored in the /uploads directory.We have covered all the basic here. Now, let us look at some of the advanced topics below.Error handling and data validation is an integral part of any software development.In handling csvs many things can go wrong and this could lead to server crashes and other thingsHere in this section we are going to learn a few basic error handling mechanisms and data validation that is a must for any softwareIt is important to check if the client is sending the file or not before parsing it or even attempting to read itAdd an error event listner inside a read stream to catch any errors which might arise while reading the fileIn this section we are looking if the file can be parsed or not. Sometime the file is corrupted or has some other error and cannot be parsedWe can check for this type of error using the below codethe completed code looks like thisthe complete code looks likeIn this section we are going to validate the data within the step functionIf our data fails to meet the criteriaThis is specific to the data that you are looking to parse. Like for example if we are looking for an email address in the csv data we can do the data validation like aboveData validation depends upon the data that you are looking to validate.you can easily pushed parsed data into an Array by using the JavaScript built in method of pushadd a new key value pair to the Object likeyou can return the parsedData to the client likeThus you can return the data back to the client.In this section we are going to learn how to work with files that are situated on another remote server. That server could be in the cloud or another server or anywhere it is accessible through an apiWe are going to use the Axios HTTP client to make requests to fetch the remote CSV fileWe are also going to stream the files in case some files are large and needed to be streamedyou can easily pipe the Axios res stream directly to PapaparseIn this section we are going to parse the data.Here what the complete code looks likeDeadSimpleChat is an Chat API providerIn this article we learned about how to parse csvs using NodeJs and PapaParse.I hope you liked the article and thank you for readingTemplates let you quickly answer FAQs or store snippets for re-use. Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink. Hide child comments as well Confirm For further actions, you may consider blocking this person and/or reporting abuse Alexandra - Sep 26 Vincent Will - Oct 18 Keyur Ramoliya - Sep 30 Nikita - Oct 9 Once suspended, alakkadshaw will not be able to comment or publish posts until their suspension is removed. Once unsuspended, alakkadshaw will be able to comment and publish posts again. Once unpublished, all posts by alakkadshaw will become hidden and only accessible to themselves. If alakkadshaw is not suspended, they can still re-publish their posts from their dashboard. Note: Once unpublished, this post will become invisible to the public and only accessible to alakkadshaw. They can still re-publish the post if they are not suspended. Thanks for keeping DEV Community safe. Here is what you can do to flag alakkadshaw: alakkadshaw consistently posts content that violates DEV Community's code of conduct because it is harassing, offensive or spammy. Unflagging alakkadshaw will restore default visibility to their posts. DEV Community — A constructive and inclusive social network for software developers. With you every step of your journey. Built on Forem — the open source software that powers DEV and other inclusive communities.Made with love and Ruby on Rails. DEV Community © 2016 - 2023. We're a place where coders share, stay up-to-date and grow their careers.