pull-stream

Experimental Minimal Pipeable Pull-stream

In classic-streams, streams push data to the next stream in the pipeline. In new-streams, data is pulled out of the source stream, into the destination.

pull-stream is an minimal take on pull streams.

Examples

What if implementing a stream was this simple:

Pipeable

var pipeable = require('pull-stream').pipeable

var createStream = pipeable(function (readable) {
  return function read (end, cb) {
    readable(end, cb)
  }
})

Readable & Reader vs. Readable & Writable

Instead of a readable stream, and a writable stream, there is a readable stream, and a reader stream.

the readable stream is just a function, that may be called many times, and will (asynchronously) callback.

if the user passes in end, then stop returning data.

var i = 100
var randomReadable = function (end, cb) {
  if(end) return cb(end)
  //only read 100 times
  if(i-- < 0) return cb(true)
  cb(null, Math.random())
}

A reader, is just a function that calls a readable. If you get an end stop reading.

var logger = function (readable) {
  readable(null, function next(end, data) {
    if(end === true) return
    if(end) throw err

    console.log(data)
    readable(end, next)
  })
}

These can be connected together by passing the readable to the reader

logger(randomReadable)

Thats cool, but to be useful, we need transformation streams, that do input and output.

Simple!

Duplex

var map = function (readable, map) {
  //return a readable function!
  return function (end, cb) {
    readable(end, function (end, data) {
      cb(end, data != null ? map(data) : null)
    })
  }
}

join them together!

function composition style "pipe"

logger(
  map(randomReadable, function (e) {
    return Math.round(e * 1000)
  }))

That is good -- but it's kinda weird, because we are used to left to right syntax for streams... ls | grep | wc -l

pipeability

So, we want to pass in the readable and reader function! It needs to be that order, so that it reads left to right.

A basic duplex function would look like this:

var i = 100
var multiply = function (readable) {
  return function (reader) {
    return reader(function (end, cb) {
      //insert your own code in here!
      readable(end, function (end, data) {
        cb(end,  Math.round(data * 1000))
      })
    })
  }
}

A stream that is only readable is simpler:

var randomReadable2 = function (reader) {
  return reader(function (end, cb) {
    cb(end, 'hello!')
  })
}

and a "sink" stream, that can only read, is the same as before!

var reader = function (readable) {
  readable(null, function (end, data) {
    if(end === true) return
    if(end) throw end
    readable(end, data)
  })
}

The reader stream is the same as before!

Left-to-Right pipe

Now PIPE THEM TOGETHER!

randomReader2 (multiply) (logger)

JUST LIKE THAT!

License

MIT