Nodejs Streams {Demystified}
Nodejs a very popular javascript runtime environment known for it’s asynchronous nature and server side application development. It has many core modules or say paradigms which are not well explored yet and are used in every application (sounds a bit confusing right?), One of such core modules is Stream.
Everything in nodejs is about efficient process handling (multiple requests) on a single thread. As a nodejs developer one just has to think about the application, everything else, such as efficiency, multiple connections etc. is taken care of by nodejs.
One such module which lets us handle data flow be it binary or objects, asynchronously is Stream.
Let’s first start by discussing about two data handling approaches offered by nodejs.
- Buffered Approach:
It can be seen in the above mock that, using buffered approach only when whole data is written to the buffer, it can read by the receiver, but this kind of approach violates our asynchronous paradigm. This is where streamed approach is different from buffered approach, data is read as soon as it arrives.
2. Streamed Approach:
So, above mock shows what exactly is streamed approach, unlike buffered approach where data can read only when whole of it has arrived, in streamed approach data arrives in chunks and is read as chunks, a chunk can even be of the whole data or a part of data.
A stream therefore is the interface that lets you handle the data asynchronously, Now this way asynchronous way of handling data has comes with great advantages regarding efficiency as well as code appearance.
Time Efficiency
There is one great behaviour of streams and that is piping, basically you can pipe two streams, where output of one stream is an input to the other. What happens is “data” (chunk) arrives at the “stream 1" which is piped to
“stream 2" which can further be piped to other streams, So data keeps on flowing between streams. Talking about Time Efficiency! Let’s look at this with an example:
You have data coming from a TCP socket and all that data you want to write it in a file, So here we have two streams, 1 for the TCP socket which is our input stream, and the other stream for the file, Stream 1 piped to Stream 2. Think of these streams as stages 1 and 2. So what will happen,
This is how we can parallelize multiple stages a data chunk might go through, This strategy is called as pipe lining. Node.js allows us to pipeline our tasks with the help of streams. Since, pipe lining allows to operate multiple stages at the same time (intrinsically), it is clear how we can achieve Time Efficiency here.
Spatial Efficiency
This advantage of streams comes from it’s name, Since data is not stored until whole of it arrives, This is how streams achieve Spatial Efficiency.
Let’s look at this over an example:
Suppose we want to read a file which is 20 MB in size and display the contents on the terminal but the buffer is only 16 MB, so what will happen is as soon as we start reading the file, our buffer gets leaked. Streams to the rescue!
To prevent this kind of situation, Create a readable stream (We’ll discuss this in coming sections) for the file, So as soon as some data is read from the file you can get it from the buffer, display it and empty the buffer, This way we can prevent our buffer from getting leaked. This is an example of Spatial Efficiency with streams.
Composability
This property is again some where related to the piping behaviour of streams, Because you can pipe one stream to other no matter how many streams you have, this results in composability. No matter what your stream is tied to, you can pipe one input (Readable Stream) to an output (Writable stream) and this can go on, There is one Duplex Stream which behaves like both Readable and Writable Streams, With the help of this stream you can pipe multiple streams.
Example:
Types of Streams
- Readable Streams: These streams lets you read data from the source.
- Writable Streams: These streams lets you write data to the destination.
- Duplex Streams: These streams are a hybrid of Readable and Writable streams.
Note: These are the 3 fundamental types of streams, There are other streams as well like Transform Stream etc. which is an extension of Duplex Stream.
In this article we’ll discuss about Readable and Writable Streams.
Implementing Writable Stream
Implementing Readable Stream
Note:
You can try above examples for more clarity on how streams work.
Conclusion:
Streams are of great use both in terms of clean code and efficiency. If one understands the data flow when using streams it is a lot easier to manage whole application in terms of data, because you don’t have to worry about making in-memory variables, define explicit read/write, push/pull methods in order to manage data flow.
For more details on stream, I would recommend `nodejs` official docs:
https://nodejs.org/api/stream.html
✉️ Subscribe to CodeBurst’s once-weekly Email Blast, 🐦 Follow CodeBurst on Twitter, view 🗺️ The 2018 Web Developer Roadmap, and 🕸️ Learn Full Stack Web Development.
