I'm trying to pipe streams to my response but it seems didn't work
Simple test case:
const Koa = require('koa');
const app = new Koa();
const Router = require('koa-better-router');
const router = Router().loadMethods();
const request = require('request');
app.use(async (ctx, next) => {
const start = new Date();
await next();
const ms = new Date() - start;
console.log(`${ctx.method} ${ctx.url} - ${ms}ms`);
});
router.get('/', async (ctx) => {
ctx.req.pipe(request(`https://www.google.com/?q=${ctx.query.q}`)).pipe(ctx.res);
});
app.use(router.middleware());
app.listen(3000, () => {
console.log('listening on port 3000');
});
An error occurred:
$ node index.js
listening on port 3000
GET / - 35ms
internal/streams/legacy.js:59
throw er; // Unhandled stream error in pipe.
^
Error: write after end
at ServerResponse.write (_http_outgoing.js:450:15)
at Request.ondata (internal/streams/legacy.js:16:26)
at emitOne (events.js:96:13)
at Request.emit (events.js:189:7)
at IncomingMessage.<anonymous> (/Users/knowlet/test/node_modules/request/request.js:1088:12)
at emitOne (events.js:96:13)
at IncomingMessage.emit (events.js:189:7)
at IncomingMessage.Readable.read (_stream_readable.js:381:10)
at flow (_stream_readable.js:761:34)
at resume_ (_stream_readable.js:743:3)
And req.pipe(request(...)).pipe(res) works in express.js, I'm wondering is there anything I do wrong?
$ node -v
v7.6.0
knowlet
All 14 comments
You'll want to let Koa handle the stream:
ctx.body = ctx.req.pipe(request(`https://www.google.com/?q=${ctx.query.q}`));
PlasmaPower
on 18 Mar 2017
It works, thanks for answering.
knowlet
on 18 Mar 2017
ctx.body = request(`https://www.google.com/?q=${ctx.query.q}`); will also work.
ChristianUlbrich
on 30 Mar 2017
I think OP might have wanted to stream the request and response (so POST data would be forwarded).
PlasmaPower
on 30 Mar 2017
@PlasmaPower
You'll want to let Koa handle the stream:
Why?
What for?
ctx.res is a standard HTTP response object so there shouldn't be any need for anything else
catamphetamine
on 7 Oct 2017
@catamphetamine Koa can be seen as a wrapper around the Node.JS HTTP APIs. Using those APIs directly bypasses Koa. Writing to Node.JS HTTP objects managed by Koa is likely to conflict with Koa, as it does in this case. If you don't want Koa to touch the response, you can use ctx.respond = false, which also makes OP's example work perfectly. However, that is not the recommended solution, and it may cause problems with other middleware.
PlasmaPower
on 7 Oct 2017
you can use
ctx.respond = false
Oh, you have that. Ok, that will do.
However, that is not the recommended solution, and it may cause problems with other middleware.
How about this
.use(async (ctx) =>
{
// https://medium.com/@aickin/whats-new-with-server-side-rendering-in-react-16-9b0d78585d67
ctx.res.write(header)
await pipe(stream, ctx.res, { end: false })
ctx.res.write(footer)
ctx.res.end()
})
// Pipes `from` stream to `to` stream.
// Returns a `Promise`.
function pipe(from, to, options)
{
return new Promise((resolve, reject) =>
{
from.pipe(to, options)
from.on('error', reject)
from.on('end', resolve)
})
}
That should work, as Koa checks ctx.res.writable. You could also use the combined-stream package. Create a combined stream, append the header and footer, and then set ctx.body to the combined stream.
PlasmaPower
on 7 Oct 2017
Ok, thanks
catamphetamine
on 7 Oct 2017
@PlasmaPower I've tried to play around with combined-stream but I'm struggling with it.
Following @catamphetamine's example, I can't get this to work:
const header = '<html><head></head><body>'
const footer = '</body></html>'
const combinedStream = CombinedStream.create()
combinedStream.write(header)
// https://medium.com/@aickin/whats-new-with-server-side-rendering-in-react-16-9b0d78585d67
const stream = renderToNodeStream()
combinedStream.append(stream)
combinedStream.pipe(ctx.body, { end : false }) // Triggers an error 'on' is not defined
combinedStream.write(footer)
I know I should let ctx.body handle the stream, but I'm not sure how in that case.
ctx.body = combinedStream.pipe(.. ?
jamesbechet
on 11 Oct 2017
@saxofficial Try multistream
import string_stream from 'string-to-stream'
import multi_stream from 'multistream'
ctx.status = 200
ctx.body = multi_stream
([
string_stream(before_content),
typeof content === 'string' ? string_stream(content) : content,
string_stream(after_content)
])
It works as part of react-isomorphic-render.
catamphetamine
on 11 Oct 2017
@catamphetamine thanks, that works 馃憤
jamesbechet
on 11 Oct 2017
@catamphetamine
let combinedStream = CombinedStream.create();
combinedStream.append(htmlTemplate[0]);
combinedStream.append(stream);
combinedStream.append(htmlTemplate[1]);
ctx.body = combinedStream;
it works~
don't use combinedStream.write directly. the API document said.
lzr900515
on 8 Jan 2019
For anyone using node-fetch. This seems to work:
const response = await fetch(...)
ctx.body = response.body
zachsa
on 1 Dec 2020
Related issues
ElegantScripting
路
5Comments
wlingke
路
3Comments
SteveCruise
路
3Comments
koalex
路
3Comments
usernameisalreadytaken2014
路
4Comments
Most helpful comment
You'll want to let Koa handle the stream: