Read a file one line at a time in node.js?

javascript

file-handling

asynchronous

error-handling

byAnton Shumikhin·Aug 19, 2024

const readline = require('readline');
const fs = require('fs');

const lineReader = readline.createInterface({
    input: fs.createReadStream('file.txt')
});

lineReader.on('line', (line) => console.log(line));

This Node.js code chunk combines the power of the readline and fs modules to stream a text file and print each line as it's processed. Just swap out 'file.txt' with the path to your file.

Handling line breaks and file sizes correctly

Deal with inconsistent line breaks

When you're dealing with files that have mixed line endings or network streams, the crlfDelay option comes in handy, ensuring that every \r\n combination is considered a single newline.

const lineReader = readline.createInterface({
  input: fs.createReadStream('file.txt'),
  crlfDelay: Infinity // because time waits for no line
});

Using asynchronous handling for large files

Large files often choke memory if not handled efficiently. To prevent this horror, process each line as it's read - effectively utilizing the asynchronous nature of Node.js streams.

lineReader.on('line', async (line) => {
  await processLine(line); // processLine, not procrastinate
});

Reading to the very end

To ensure no line goes unread from large files - especially the last line that may not end with a newline character, make use of the close event.

lineReader.on('close', () => {
  console.log('Finished the whole marathon.');
});

Looping for the modern Node.js

For those lucky folks using Node.js v11.4 or later, the for await...of loop bestows an elegant line-by-line reading.

const rl = readline.createInterface({
  input: fs.createReadStream('file.txt'),
  crlfDelay: Infinity
});

for await (const line of rl) {
  console.log(`Line delivered: ${line}`);
}

Working around common hiccups

Choosing the correct text encoding

Always remember to specify the correct encoding, like 'utf-8', to avoid any mix-ups with special characters or text output.

const lineReader = readline.createInterface({
  input: fs.createReadStream('file.txt', { encoding: 'utf-8' }) // speaking the same language
});

Relying on native modules

Using readline generally wins out over third-party packages, but packages like Lazy and event-stream can assist with specific use cases when needed.

Smoothing out error handling

When setting up the readline interface, error handling is paramount. Always include it:

lineReader.on('error', (error) => {
  console.error(`Woops! Run into an error: ${error}`);
});

Considering synchronous reading

Asynchronous reading is King regarding performance, but for simpler scripts, or if you want to feel a bit old-fashioned, fs.readFileSync coupled with splitting the string can be used for line-by-line processing:

const allLines = fs.readFileSync('file.txt', { encoding: 'utf-8' }).split(/\r?\n/);
allLines.forEach((line) => {
  console.log(line); // takes a ForEach loop to tango
});