git ssb

2+

Dominic / pull-stream



Commit 11f0d082fcb805336ce40ead36918a2dc3cf4ee7

Browse Files

docs: refactor ./docs into per-module files

to be consistent with rest of pull modules for use in ecosystem docs aggregation.
Michael Williams committed on 6/12/2016, 2:57:37 AM
Parent: 81642b10bd4a0105dae4a0d01429420d3ddb2016

Files changed

README.mdchanged
sinks/collect.mdadded
sinks/concat.mdadded
sinks/drain.mdadded
sinks/index.mdadded
sinks/log.mdadded
sinks/on-end.mdadded
sinks/reduce.mdadded
sources/count.mdadded
sources/empty.mdadded
sources/error.mdadded
sources/index.mdadded
sources/infinite.mdadded
sources/keys.mdadded
sources/once.mdadded
sources/values.mdadded
throughs/async-map.mdadded
throughs/filter-not.mdadded
throughs/filter.mdadded
throughs/flatten.mdadded
throughs/index.mdadded
throughs/map.mdadded
throughs/non-unique.mdadded
throughs/take.mdadded
throughs/through.mdadded
throughs/unique.mdadded
docs/glossary.mddeleted
docs/sinks.mddeleted
docs/sources.mddeleted
docs/throughs.mddeleted
glossary.mdadded
pull.mdadded
README.mdView
@@ -44,11 +44,11 @@
4444 (aka "Source") and a `reader` stream (aka "Sink"). Through streams
4545 is a Sink that returns a Source.
4646
4747 See also:
48-* [Sources](https://github.com/dominictarr/pull-stream/blob/master/docs/sources.md)
49-* [Throughs](https://github.com/dominictarr/pull-stream/blob/master/docs/throughs.md)
50-* [Sinks](https://github.com/dominictarr/pull-stream/blob/master/docs/sinks.md)
48+* [Sources](./sources/index.md)
49+* [Throughs](./throughs/index.md)
50+* [Sinks](./sinks/index.md)
5151
5252 ### Source (aka, Readable)
5353
5454 The readable stream is just a `function read(end, cb)`,
@@ -79,10 +79,10 @@
7979
8080 A sink is just a `reader` function that calls a Source (read function),
8181 until it decideds to stop, or the readable ends. `cb(err || true)`
8282
83-All [Throughs](https://github.com/dominictarr/pull-stream/blob/master/docs/throughs.md)
84-and [Sinks](https://github.com/dominictarr/pull-stream/blob/master/docs/sinks.md)
83+All [Throughs](./throughs/index.md)
84+and [Sinks](./sinks/index.md)
8585 are reader streams.
8686
8787 ```js
8888 //read source and log it.
@@ -304,12 +304,12 @@
304304 - https://github.com/dominictarr/pull-stream-examples
305305 - https://github.com/pull-stream/pull-stream/blob/master/examples.md
306306
307307 Explore this repo further for more information about
308-[sources](https://github.com/pull-stream/pull-stream/tree/master/sources),
309-[throughs](https://github.com/pull-stream/pull-stream/tree/master/throughs),
310-[sinks](https://github.com/pull-stream/pull-stream/tree/master/sinks), and
311-[more](https://github.com/pull-stream/pull-stream/tree/master/docs).
308+[sources](./sources/index.md),
309+[throughs](./throughs/index.md),
310+[sinks](./sinks/index.md), and
311+[glossary](./glossary.md).
312312
313313
314314 ## License
315315
sinks/collect.mdView
@@ -1,0 +1,10 @@
1+# pull-stream/sinks/collect
2+
3+## usage
4+
5+### `collect = require('pull-stream/sinks/collect')`
6+
7+### `collect(cb)`
8+
9+Read the stream into an array, then callback.
10+
sinks/concat.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/sinks/concat
2+
3+## usage
4+
5+### `concat = require('pull-stream/sinks/concat')`
6+
7+### `concat(cb)`
8+
9+concat stream of strings into single string, then callback.
sinks/drain.mdView
@@ -1,0 +1,11 @@
1+# pull-stream/sinks/drain
2+
3+## usage
4+
5+### `drain = require('pull-stream/sinks/drain')`
6+
7+### `drain(op?, done?)`
8+
9+Drain the stream, calling `op` on each `data`.
10+call `done` when stream is finished.
11+If op returns `===false`, abort the stream.
sinks/index.mdView
@@ -1,0 +1,22 @@
1+# Sinks
2+
3+A Sink is a stream that is not readable.
4+You *must* have a sink at the end of a pipeline
5+for data to move towards.
6+
7+You can only use _one_ sink per pipeline.
8+
9+``` js
10+pull(source, through, sink)
11+```
12+
13+See also:
14+* [Sources](../sources/index.md)
15+* [Throughs](../throughs/index.md)
16+
17+## [drain](./drain.md)
18+## [reduce](./reduce.md)
19+## [concat](./collect.md)
20+## [collect](./collect.md)
21+## [onEnd](./on-end.md)
22+## [log](./log.md)
sinks/log.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/sinks/log
2+
3+## usage
4+
5+### `log = require('pull-stream/sinks/log')`
6+
7+### `log()`
8+
9+output the stream to `console.log`
sinks/on-end.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/sinks/onEnd
2+
3+## usage
4+
5+### `onEnd = require('pull-stream/sinks/onEnd')`
6+
7+### `onEnd(cb)`
8+
9+Drain the stream and then callback when done.
sinks/reduce.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/sinks/reduce
2+
3+## usage
4+
5+### `reduce = require('pull-stream/sinks/reduce')`
6+
7+### `reduce (reduce, initial, cb)`
8+
9+reduce stream into single value, then callback.
sources/count.mdView
@@ -1,0 +1,12 @@
1+# pull-stream/sources/count
2+
3+## usage
4+
5+### `count = require('pull-stream/sources/count')`
6+
7+### `count(max, onAbort)`
8+
9+create a stream that outputs `0 ... max`.
10+by default, `max = Infinity`, see
11+[take](../throughs/take.md)
12+
sources/empty.mdView
@@ -1,0 +1,20 @@
1+# pull-stream/sources/empty
2+
3+## usage
4+
5+### `empty = require('pull-stream/sources/empty')`
6+
7+### `empty()`
8+
9+A stream with no contents (it just ends immediately)
10+
11+``` js
12+pull(
13+ pull.empty(),
14+ pull.collect(function (err, ary) {
15+ console.log(arg)
16+ // ==> []
17+ })
18+}
19+```
20+
sources/error.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/sources/error
2+
3+## usage
4+
5+### `error = require('pull-stream/sources/error')`
6+
7+### `error(err)`
8+
9+a stream that errors immediately
sources/index.mdView
@@ -1,0 +1,23 @@
1+# Sources
2+
3+A source is a stream that is not writable.
4+You *must* have a source at the start of a pipeline
5+for data to move through.
6+
7+in general:
8+
9+``` js
10+pull(source, through, sink)
11+```
12+
13+See also:
14+* [Throughs](../throughs/index.md)
15+* [Sinks](../sinks/index.md)
16+
17+## [values](./values.md)
18+## [keys](./keys.md)
19+## [count](./count.md)
20+## [infinite](./infinite.md)
21+## [empty](./empty.md)
22+## [once](./once.md)
23+## [error](./error.md)
sources/infinite.mdView
@@ -1,0 +1,11 @@
1+# pull-stream/sources/infinite
2+
3+## usage
4+
5+### `infinite = require('pull-stream/sources/infinite')`
6+
7+### `infinite(generator, onAbort)`
8+
9+create an unending stream by repeatedly calling a generator
10+function (by default, `Math.random`)
11+see [take](../throughs/take.md)
sources/keys.mdView
@@ -1,0 +1,10 @@
1+# pull-stream/sources/keys
2+
3+## usage
4+
5+### `keys = require('pull-stream/sources/keys')`
6+
7+### `keys(array | object, onAbort)`
8+
9+stream the key names from an object (or array)
10+
sources/once.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/sources/once
2+
3+## usage
4+
5+### `once = require('pull-stream/sources/once')`
6+
7+### `once(value, onAbort)`
8+
9+a stream with a single value
sources/values.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/sources/values
2+
3+## usage
4+
5+### `values = require('pull-stream/sources/values')`
6+
7+### `values(array | object, onAbort)`
8+
9+create a SourceStream that reads the values from an array or object and then stops.
throughs/async-map.mdView
@@ -1,0 +1,10 @@
1+# pull-stream/throughs/async-map
2+
3+## usage
4+
5+### `asyncMap = require('pull-stream/throughs/async-map')`
6+
7+### `asyncMap(fn)`
8+
9+Like [`map`](./map.md) but the signature of `fn` must be
10+`function (data, cb) { cb(null, data) }`
throughs/filter-not.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/throughs/filterNot
2+
3+## usage
4+
5+### `filterNot = require('pull-stream/throughs/filter-not')`
6+
7+### `filterNot(test)`
8+
9+Like [`filter`](./filter.md), but remove items where the filter returns true.
throughs/filter.mdView
@@ -1,0 +1,11 @@
1+# pull-stream/throughs/filter
2+
3+## usage
4+
5+### `filter = require('pull-stream/throughs/filter')`
6+
7+### `filter(test)`
8+
9+Like `[].filter(function (data) {return true || false})`
10+only `data` where `test(data) == true` are let through
11+to the next stream.
throughs/flatten.mdView
@@ -1,0 +1,9 @@
1+# pull-stream/throughs/flatten
2+
3+## usage
4+
5+### `flatten = require('pull-stream/throughs/flatten')`
6+
7+### `flatten()`
8+
9+Turn a stream of arrays into a stream of their items, (undoes group).
throughs/index.mdView
@@ -1,0 +1,46 @@
1+# Throughs
2+
3+A Through is a stream that both reads and is read by
4+another stream.
5+
6+Through streams are optional.
7+
8+Put through streams in-between [sources](../sources/index.md) and [sinks](../sinks/index.md),
9+like this:
10+
11+```js
12+pull(source, through, sink)
13+```
14+
15+Also, if you don't have the source/sink yet,
16+you can pipe multiple through streams together
17+to get one through stream!
18+
19+```js
20+var throughABC = function () {
21+ return pull(
22+ throughA(),
23+ throughB(),
24+ throughC()
25+ )
26+}
27+```
28+
29+Which can then be treated like a normal through stream!
30+
31+```js
32+pull(source(), throughABC(), sink())
33+```
34+
35+See also:
36+* [Sources](../sources/index.md)
37+* [Sinks](../sinks/index.md)
38+
39+## [map](./map.md)
40+## [asyncMap](./async-map.md)
41+## [filter](./filter.md)
42+## [filterNot](./filter-not.md)
43+## [unique](./unique.md)
44+## [nonUnique](./non-unique.md)
45+## [take](./take.md)
46+## [flatten](./flatten.md)
throughs/map.mdView
@@ -1,0 +1,54 @@
1+# pull-stream/throughs/map
2+
3+> [].map for pull-streams
4+
5+## Background
6+
7+Pull-streams are arrays of data in time rather than space.
8+
9+As with a [`[].map`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map), we may want to map a function over a stream.
10+
11+## Example
12+
13+```js
14+var map = require('pull-stream/throughs/map')
15+```
16+
17+```js
18+pull(
19+ values([0, 1, 2, 3]),
20+ map(function (x) {
21+ return x * x
22+ }),
23+ log()
24+)
25+// 0
26+// 1
27+// 4
28+// 9
29+```
30+
31+## Usage
32+
33+### `map = require('pull-stream/throughs/map')`
34+
35+### `map((data) => data)`
36+
37+`map(fn)` returns a through stream that calls the given `fn` for each chunk of incoming data and outputs the return value, in the same order as before.
38+
39+## Install
40+
41+With [npm](https://npmjs.org/) installed, run
42+
43+```
44+$ npm install pull-stream
45+```
46+
47+## See Also
48+
49+- [`brycebaril/through2-map`](https://github.com/brycebaril/through2-map)
50+- [`Rx.Obsevable#map`](http://xgrommx.github.io/rx-book/content/observable/observable_instance_methods/map.html)
51+
52+## License
53+
54+[MIT](https://tldrlegal.com/license/mit-license)
throughs/non-unique.mdView
@@ -1,0 +1,10 @@
1+# pull-stream/throughs/nonUnique
2+
3+## usage
4+
5+### `nonUnique = require('pull-stream/throughs/non-unique')`
6+
7+### `nonUnique(prop)`
8+
9+Filter unique items -- get the duplicates.
10+The inverse of [`unique`](./unique.md)
throughs/take.mdView
@@ -1,0 +1,13 @@
1+# pull-stream/throughs/take
2+
3+## usage
4+
5+### `take = require('pull-stream/throughs/take')`
6+
7+### `take(test [, opts])`
8+
9+If test is a function, read data from the source stream and forward it downstream until test(data) returns false.
10+
11+If `opts.last` is set to true, the data for which the test failed will be included in what is forwarded.
12+
13+If test is an integer, take n item from the source.
throughs/through.mdView
@@ -1,0 +1,5 @@
1+# pull-stream/throughs/filter
2+
3+## usage
4+
5+### `filter = require('pull-stream/throughs/filter')`
throughs/unique.mdView
@@ -1,0 +1,11 @@
1+# pull-stream/throughs/unique
2+
3+## usage
4+
5+### `unique = require('pull-stream/throughs/unique')`
6+
7+### `unique(prop)`
8+
9+Filter items that have a repeated value for `prop()`,
10+by default, `prop = function (it) {return it }`, if prop is a string,
11+it will filter nodes which have repeated values for that property.
docs/glossary.mdView
@@ -1,47 +1,0 @@
1-# Glossary
2-
3-## read (end, cb)
4-
5-A function that retrives the next chunk.
6-All readable streams (sources, and throughs)
7-must return a `read` function.
8-
9-## reader (read,...)
10-
11-A function to create a reader. It takes a `read` function
12-as the first argument, and any other options after that.
13-
14-When passed to `pipeable` or `pipeableSource`,
15-a new function is created that adds `.pipe(dest)`
16-
17-## Lazy vs Eager
18-
19-Lazy means to avoid doing something until you know you have
20-to do it.
21-
22-Eager means to do something early, so you have it ready
23-immediately when you need it.
24-
25-## Source
26-
27-The first stream in the pipeline. The Source is not writable.
28-
29-## Sink
30-
31-The last Stream in the pipeline. The Sink is not readable.
32-
33-## Push vs Pull
34-
35-A pull-stream is a stream where the movement of data
36-is initiated by the sink, and a push-stream
37-is a stream where the movement of data is initiated
38-by the source.
39-
40-## Reader vs Writable
41-
42-In push streams, destination streams (Through and Sink),
43-are _writable_. They are written to by the source streams.
44-
45-In pull streams, destination streams _read_ from the source
46-streams. They are the active participant, so they are called
47-_readers_ rather than _writables_.
docs/sinks.mdView
@@ -1,38 +1,0 @@
1-# Sinks
2-
3-A Sink is a stream that is not readable.
4-You *must* have a sink at the end of a pipeline
5-for data to move towards.
6-
7-You can only use _one_ sink per pipeline.
8-
9-``` js
10-pull(source, through, sink)
11-```
12-
13-See also:
14-* [Sources](https://github.com/dominictarr/pull-stream/blob/master/docs/sources.md)
15-* [Throughs](https://github.com/dominictarr/pull-stream/blob/master/docs/throughs.md)
16-
17-## drain (op?, done?)
18-
19-Drain the stream, calling `op` on each `data`.
20-call `done` when stream is finished.
21-If op returns `===false`, abort the stream.
22-
23-## reduce (reduce, initial, cb)
24-
25-reduce stream into single value, then callback.
26-
27-## collect(cb)
28-
29-Read the stream into an array, then callback.
30-
31-## onEnd (cb)
32-
33-Drain the stream and then callback when done.
34-
35-## log
36-
37-output the stream to `console.log`
38-
docs/sources.mdView
@@ -1,48 +1,0 @@
1-# Sources
2-
3-A source is a stream that is not writable.
4-You *must* have a source at the start of a pipeline
5-for data to move through.
6-
7-in general:
8-
9-``` js
10-pull(source, through, sink)
11-```
12-
13-See also:
14-* [Throughs](https://github.com/dominictarr/pull-stream/blob/master/docs/throughs.md)
15-* [Sinks](https://github.com/dominictarr/pull-stream/blob/master/docs/sinks.md)
16-
17-## values (array | object)
18-
19-create a SourceStream that reads the values from an array or object and then stops.
20-
21-## keys (array | object)
22-
23-stream the key names from an object (or array)
24-
25-## count (max)
26-
27-create a stream that outputs `0 ... max`.
28-by default, `max = Infinity`, see
29-[take](https://github.com/dominictarr/pull-stream/blob/master/docs/throughs.md#take_test)
30-
31-## infinite (generator)
32-
33-create an unending stream by repeatedly calling a generator
34-function (by default, `Math.random`)
35-see
36-[take](https://github.com/dominictarr/pull-stream/blob/master/docs/throughs.md#take_test)
37-
38-## empty
39-
40-A stream with no contents (it just ends immediately)
41-
42-``` js
43-pull.empty().pipe(pull.collect(function (err, ary) {
44- console.log(arg)
45- // ==> []
46-})
47-```
48-
docs/throughs.mdView
@@ -1,79 +1,0 @@
1-# Throughs
2-
3-A Through is a stream that both reads and is read by
4-another stream.
5-
6-Through streams are optional.
7-
8-Put through streams in-between [sources](https://github.com/dominictarr/pull-stream/blob/master/docs/sources.md) and [sinks](https://github.com/dominictarr/pull-stream/blob/master/docs/sinks.md),
9-like this:
10-
11-```js
12-pull(source, through, sink)
13-```
14-
15-Also, if you don't have the source/sink yet,
16-you can pipe multiple through streams together
17-to get one through stream!
18-
19-```js
20-var throughABC = function () {
21- return throughA()
22- .pipe(throughB())
23- .pipe(throughC())
24-}
25-```
26-
27-Which can then be treated like a normal through stream!
28-
29-```js
30-source().pipe(throughABC()).pipe(sink())
31-```
32-
33-See also:
34-* [Sources](https://github.com/dominictarr/pull-stream/blob/master/docs/sources.md)
35-* [Sinks](https://github.com/dominictarr/pull-stream/blob/master/docs/sinks.md)
36-
37-## map (fun)
38-
39-Like `[].map(function (data) {return data})`
40-
41-## asyncMap (fun)
42-
43-Like `map` but the signature of `fun` must be
44-`function (data, cb) { cb(null, data) }`
45-
46-## filter (test)
47-
48-Like `[].filter(function (data) {return true || false})`
49-only `data` where `test(data) == true` are let through
50-to the next stream.
51-
52-
53-## filterNot (test)
54-
55-Like filter, but remove items where the filter returns true.
56-
57-## unique (prop)
58-
59-Filter items that have a repeated value for `prop()`,
60-by default, `prop = function (it) {return it }`, if prop is a string,
61-it will filter nodes which have repeated values for that property.
62-
63-## nonUnique (prop)
64-
65-Filter unique items -- get the duplicates.
66-The inverse of `unique`
67-
68-## take (test [, opts])
69-
70-If test is a function, read data from the source stream and forward it downstream until test(data) returns false.
71-
72-If `opts.last` is set to true, the data for which the test failed will be included in what is forwarded.
73-
74-If test is an integer, take n item from the source.
75-
76-## flatten ()
77-
78-Turn a stream of arrays into a stream of their items, (undoes group).
79-
glossary.mdView
@@ -1,0 +1,47 @@
1+# Glossary
2+
3+## read (end, cb)
4+
5+A function that retrives the next chunk.
6+All readable streams (sources, and throughs)
7+must return a `read` function.
8+
9+## reader (read,...)
10+
11+A function to create a reader. It takes a `read` function
12+as the first argument, and any other options after that.
13+
14+When passed to `pipeable` or `pipeableSource`,
15+a new function is created that adds `.pipe(dest)`
16+
17+## Lazy vs Eager
18+
19+Lazy means to avoid doing something until you know you have
20+to do it.
21+
22+Eager means to do something early, so you have it ready
23+immediately when you need it.
24+
25+## Source
26+
27+The first stream in the pipeline. The Source is not writable.
28+
29+## Sink
30+
31+The last Stream in the pipeline. The Sink is not readable.
32+
33+## Push vs Pull
34+
35+A pull-stream is a stream where the movement of data
36+is initiated by the sink, and a push-stream
37+is a stream where the movement of data is initiated
38+by the source.
39+
40+## Reader vs Writable
41+
42+In push streams, destination streams (Through and Sink),
43+are _writable_. They are written to by the source streams.
44+
45+In pull streams, destination streams _read_ from the source
46+streams. They are the active participant, so they are called
47+_readers_ rather than _writables_.
pull.mdView
@@ -1,0 +1,106 @@
1+# pull-stream/pull
2+
3+> pipe many pull streams into a pipeline
4+
5+## Background
6+
7+In pull-streams, you need a complete pipeline before data will flow.
8+
9+That means: a source, zero or more throughs, and a sink.
10+
11+But you can still create a _partial_ pipeline, which is a great for tiny pull-stream modules.
12+
13+## Usage
14+
15+```js
16+var pull = require('pull-stream/pull')
17+```
18+
19+Create a simple complete pipeline:
20+
21+```js
22+pull(source, sink) => undefined
23+```
24+
25+Create a source modified by a through:
26+
27+```js
28+pull(source, through) => source
29+```
30+
31+Create a sink, but modify it's input before it goes.
32+
33+```js
34+pull(through, sink) => sink
35+```
36+
37+Create a through, by chainging several throughs:
38+
39+```js
40+pull(through1, through2) => through
41+```
42+
43+These streams combine just like normal streams.
44+
45+```js
46+pull(
47+ pull(source, through),
48+ pull(through1, through2),
49+ pull(through, sink)
50+) => undefined
51+```
52+
53+The complete pipeline returns undefined, because it cannot be piped to anything else.
54+
55+Pipe duplex streams like this:
56+
57+```js
58+var a = duplex()
59+var b = duplex()
60+
61+pull(a.source, b.sink)
62+pull(b.source, a.sink)
63+
64+//which is the same as
65+
66+b.sink(a.source); a.sink(b.source)
67+
68+//but the easiest way is to allow pull to handle this
69+
70+pull(a, b, a)
71+
72+//"pull from a to b and then back to a"
73+```
74+
75+## API
76+
77+```js
78+var pull = require('pull-stream/pull')
79+```
80+
81+### `pull(...streams)`
82+
83+`pull` is a function that receives n-arity stream arguments and connects them into a pipeline.
84+
85+`pull` detects the type of stream by checking function arity, if the function takes only one argument it's either a sink or a through. Otherwise it's a source. A duplex stream is an object with the shape `{ source, sink }`.
86+
87+If the pipeline is complete (reduces into a source being passed into a sink), then `pull` returns `undefined`, as the data is flowing.
88+
89+If the pipeline is partial (reduces into either a source, a through, or a sink), then `pull` returns the partial pipeline, as it must be composed with other streams before the data will flow.
90+
91+## Install
92+
93+With [npm](https://npmjs.org/) installed, run
94+
95+```sh
96+$ npm install pull-stream
97+```
98+
99+## See Also
100+
101+- [`mafintosh/pump`](https://github.com/mafintosh/pump)
102+- [`mafintosh/pumpify`](https://github.com/mafintosh/pumpify)
103+
104+## License
105+
106+[MIT](https://tldrlegal.com/license/mit-license)

Built with git-ssb-web