Skip to content

Commit b37ccc7

Browse files
committed
docs: add stream wrapping example (#466)
* docs: add duplex wrapping example docs: add iterable types from @alanshaw's gist * docs(fix): add feedback fix Co-Authored-By: Vasco Santos <vasco.santos@moxy.studio> * docs: clean up based on feedback
1 parent 440fbf0 commit b37ccc7

File tree

1 file changed

+133
-1
lines changed

1 file changed

+133
-1
lines changed

doc/STREAMING_ITERABLES.md

+133-1
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,143 @@
77
- [Iterable Streams](#iterable-streams)
88
- [Table of Contents](#table-of-contents)
99
- [Usage Guide](#usage-guide)
10+
- [Transforming Bidirectional Data](#transforming-bidirectional-data)
11+
- [Iterable Stream Types](#iterable-stream-types)
12+
- [Source](#source)
13+
- [Sink](#sink)
14+
- [Transform](#transform)
15+
- [Duplex](#duplex)
1016
- [Iterable Modules](#iterable-modules)
1117

1218
## Usage Guide
1319

14-
**Coming Soon!**
20+
### Transforming Bidirectional Data
21+
22+
Sometimes you may need to wrap an existing duplex stream in order to perform incoming and outgoing [transforms](#transform) on data. This type of wrapping is commonly used in stream encryption/decryption. Using [it-pair][it-pair] and [it-pipe][it-pipe], we can do this rather easily, given an existing [duplex iterable](#duplex).
23+
24+
```js
25+
const duplexPair = require('it-pair/duplex')
26+
const pipe = require('it-pipe')
27+
28+
// Wrapper is what we will write and read from
29+
// This gives us two duplex iterables that are internally connected
30+
const [internal, external] = duplexPair()
31+
32+
// Now we can pipe our wrapper to the existing duplex iterable
33+
pipe(
34+
external, // The external half of the pair interacts with the existing duplex
35+
outgoingTransform, // A transform iterable to send data through (ie: encrypting)
36+
existingDuplex, // The original duplex iterable we are wrapping
37+
incomingTransform, // A transform iterable to read data through (ie: decrypting)
38+
external
39+
)
40+
41+
// We can now read and write from the other half of our pair
42+
pipe(
43+
['some data'],
44+
internal, // The internal half of the pair is what we will interact with to read/write data
45+
async (source) => {
46+
for await (const chunk of source) {
47+
console.log('Data: %s', chunk.toString())
48+
// > Data: some data
49+
}
50+
}
51+
)
52+
```
53+
54+
## Iterable Stream Types
55+
56+
These types are pulled from [@alanshaw's gist](https://gist.github.com/alanshaw/591dc7dd54e4f99338a347ef568d6ee9) on streaming iterables.
57+
58+
### Source
59+
60+
A "source" is something that can be consumed. It is an iterable object.
61+
62+
```js
63+
const ints = {
64+
[Symbol.asyncIterator] () {
65+
let i = 0
66+
return {
67+
async next () {
68+
return { done: false, value: i++ }
69+
}
70+
}
71+
}
72+
}
73+
74+
// or, more succinctly using a generator and for/await:
75+
76+
const ints = (async function * () {
77+
let i = 0
78+
while (true) yield i++
79+
})()
80+
```
81+
82+
### Sink
83+
84+
A "sink" is something that consumes (or drains) a source. It is a function that takes a source and iterates over it. It optionally returns a value.
85+
86+
```js
87+
const logger = async source => {
88+
const it = source[Symbol.asyncIterator]()
89+
while (true) {
90+
const { done, value } = await it.next()
91+
if (done) break
92+
console.log(value) // prints 0, 1, 2, 3...
93+
}
94+
}
95+
96+
// or, more succinctly using a generator and for/await:
97+
98+
const logger = async source => {
99+
for await (const chunk of source) {
100+
console.log(chunk) // prints 0, 1, 2, 3...
101+
}
102+
}
103+
```
104+
105+
### Transform
106+
107+
A "transform" is both a sink _and_ a source where the values it consumes and the values that can be consumed from it are connected in some way. It is a function that takes a source and returns a source.
108+
109+
```js
110+
const doubler = source => {
111+
return {
112+
[Symbol.asyncIterator] () {
113+
const it = source[Symbol.asyncIterator]()
114+
return {
115+
async next () {
116+
const { done, value } = await it.next()
117+
if (done) return { done }
118+
return { done, value: value * 2 }
119+
}
120+
return () {
121+
return it.return && it.return()
122+
}
123+
}
124+
}
125+
}
126+
}
127+
128+
// or, more succinctly using a generator and for/await:
129+
130+
const doubler = source => (async function * () {
131+
for await (const chunk of source) {
132+
yield chunk * 2
133+
}
134+
})()
135+
```
136+
137+
### Duplex
138+
139+
A "duplex" is similar to a transform but the values it consumes are not necessarily connected to the values that can be consumed from it. It is an object with two properties, `sink` and `source`.
140+
141+
```js
142+
const duplex = {
143+
sink: async source => {/* ... */},
144+
source: { [Symbol.asyncIterator] () {/* ... */} }
145+
}
146+
```
15147

16148
## Iterable Modules
17149

0 commit comments

Comments
 (0)