Superagent: expose xml as res.body

[tj]

never thought I'd say this, but for amazon shit it's necessary

[PWani]

In an attempt to use superagent with an XML datasource (and have res.body reflect the XML content response), I was using xml2js as shown below. Although the parser seems to be working and myParse outputs the correct result, the callback on the request seems to be getting called before the parser 'end's. i.e. request.get outputs undefined (since parser is still at work) and then the parser outputs the correct result. Any ideas on what might be happening here? Thank you.

var parser = new xml2js.Parser();

function myParse (res, fn) {
  res.text = '';
  res.setEncoding('utf8');
  res.on('data', function(chunk){ 
      res.text += chunk; });
  res.on('end', function ()
    {
    parser.parseString(res.text, function (err, result) {
      if (err)
        fn(err);
      else
      {
            fn(null, JSON.stringify(result));
      }
      });

    });
}

request.get(url)
    .set('Content-Type', 'application/xml; charset=utf8') 
    .parse(myParse)
    .end(function(err, res){
          if (err) console.log("Could not access database.");
         else console.log("Output :", res.data);
                });

[PWani]

What I'm noticing may be related to: https://github.com/visionmedia/superagent/issues/244

[laurilarjo]

I got it working with this, so I guess it's already supported.

var request = superagent.get(url);
request.buffer();
request.type('xml');
request.end(function(res) {    
    console.log(res.text);
});

[hallas]

@visionmedia you still want this?

[toddgeist]

I need to work with some XML APIs :-(

I would like to help get this done. I just need some direction. My skills are intermediate at best, I need more practice. This looks like a good problem to solve. What can I do to help? Is there a plan for this?

Thanks

Todd

[toddgeist]

I just published a tiny module that converts xml in res.text to JSON in res.body

https://github.com/toddgeist/superagent-xml2jsparser

[gjohnson]

I'm -1 in core and maybe adding a little cleaner way to define new parsers. XML is a good example actually... some people will want/need a certain XML parser for various legit reasons (e.g libxml vs xml-parser vx xyz) and then on top of that some people will want the XML to be parsed into JSON and others will want the XML document and so on.

[gjohnson]

Perhaps something like:

var request = require('superagent');
var parse = require('xml-parser');

request.parsers.set('text/xml', parse);

request.get('/', function (res){
  res.body // << whatever xml-parser gives you (an object I think)
});

or

var request = require('superagent');
var libxml = require('libxmljs');

request.parsers.set('text/xml', libxml.parseXmlString);

request.get('/', function (res){
  res.body // << instanceof XmlDocument
});

[hallas]

why not as a plugin instead? lets do this right.

Would it be possible on the form request.use(parser) ? Maybe like var agent = request.agent(); agent.use(parser); and then only use that agent for the XML requests

[gjohnson]

Actually there is already a way... otherwise I think I am confusing the feature request here.

superagent.parse['application/xml'] = function(res, fn){
   fn(null, customParserHere(res.text));
};

https://github.com/visionmedia/superagent/blob/master/lib/node/index.js#L102-L104

[gjohnson]

Closing since we expose https://github.com/visionmedia/superagent/blob/master/lib/node/index.js#L102-L104

[stephy]

I was able to use the solution from @laurilarjo, so Thanks!
For those who want to use that solution as well, there's a little typo in his code. Don't forget the error comes first in the parameter list for the callback function on end.

var request = superagent.get(url);
request.buffer();
request.type('xml');
request.end(function(err, res) {    
    console.log(res.text);
});

or the 'shorter' version:

const request = superagent.get(url)
 .buffer()
 .type('xml')
 .end((err, res) => {    
    console.log(res.text);
});

[susumuasaga]

I was able to use solution from @laurilarjo, @PWani and @gjohnson, thx all!
The key point is enable buffering with .buffer(), the .type() refers to the request body and useless if you are only interested on the response body:
```var url = 'http://localhost:3000';
var superagent = require('superagent');
var xml2js = require('xml2js');
function myParse (res, cb) {
res.text = '';
res.on('data', chunk => res.text += chunk);
res.on('end', () => xml2js.parseString(res.text, cb));
}
superagent.parse['application/xml'] = myParse;
superagent.get(url).buffer().end((err, res) => console.dir(res.body, {depth: null});

[AdamT213]

@susumuasaga, your solution worked for me!