Node-ytdl-core: "Proxy with auth" example not working.

[mh4ck]

Hi, i tried the proxy with auth example and got a 400 Bad Request response.
Then i changed the path: parsed.href to path: parsed.path now i get the error To many redirects does anybody else got it to work?

Here is a test Proxy if smbd. want to test.
{"host":"217.29.63.224","port":"10798","user":"SgpHms","pass":"6QQu3N"}

[fent]

can you get the above working with the http module, without ytdl-core? I get an auth error with the following, is this an http proxy?

const http = require('http');
const { parse } = require('url');

let url = 'https://www.youtube.com/watch?v=iTg1bV2vks8';
let parsed = parse(url);
let username = 'SgpHms';
let password = '6QQu3N';
let auth = 'Basic ' + Buffer.from(username + ':' + password).toString('base64');

console.log('requesting...');
let useProxy = true;
http.get(useProxy ? {
  host: '217.29.63.224',
  port: 10798,
  path: parsed.path,
  headers: {
    Host: parsed.host,
    Authorization: auth,
  },
} : parsed, (res) => {
  console.log('got res', res.statusCode);
  res.setEncoding('utf8')
  res.on('data', (chunk) => {
    console.log('downloaded', chunk);
  });
});

I got most of the code from https://stackoverflow.com/a/3905553/157629

[RocoGreen]

Use Proxy-Authorization instead :

const http = require('http');
const { parse } = require('url');

let url = 'https://www.youtube.com/watch?v=iTg1bV2vks8';
let parsed = parse(url);
let username = 'SgpHms';
let password = '6QQu3N';
let auth = 'Basic ' + Buffer.from(username + ':' + password).toString('base64');

console.log('requesting...');
let useProxy = true;
http.get(useProxy ? {
  host: '217.29.63.224',
  port: 10798,
  path: parsed.path,
  headers: {
    Host: parsed.host,
    'Proxy-Authorization': auth,
  },
} : parsed, (res) => {
  console.log('got res', res.statusCode);
  res.setEncoding('utf8')
  res.on('data', (chunk) => {
    console.log('downloaded', chunk);
  });
});

EDIT : Also, this proxy return a 301 code so you can have the redirected url with res.headers.location

[fent]

if I remove protocol from the proxy_with_auth example, it works. with protocol set to always be "http:", I get a too many redirects error because youtube keeps trying to redirect to the https version of the site

[DELUUXE]

when removing the protocol: 'http' i am receiving the following error
os: windows 10

Error: write EPROTO 26768:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:openssl\ssl\record\ssl3_record.c:252:
at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:66:16)

[fent]

I'm getting that too with a proxy from proxyorbit. I also tried using http-proxy-agent and got a Status Code: 502 error.

[mh4ck]

I also tried a lot of things - but came up to switch over to "request" module. Found no other way to get it work.

[fent]

The proxy you have in your OP does work for me with the example, after removing protocol

[DELUUXE]

I'm getting that too with a proxy from proxyorbit. I also tried using http-proxy-agent and got a Status Code: 502 error.

seems like ill be trying to find a different proxy provider, sorry for bothering 😓

[fent]

ah, so I tried using https-proxy-agent instead, since ytdl-core makes https requests, and it worked.

const ytdl = require('..');
const Agent = require('https-proxy-agent');

const proxy = 'http://xxx.xxx.xxx.xxx:xxxx';
const agent = new Agent(proxy);
const stream = ytdl('https://www.youtube.com/watch?v=2UBFIhS1YBk', {
  requestOptions: { agent }
});

console.log('Starting Download');

stream.on('data', (chunk) => {
  console.log('downloaded', chunk.length);
});

stream.on('end', () => {
  console.log('Finished');
});

I can update the examples to show usage with https-proxy-agent instead of using requestOptions.transform since handling proxies can be complicated, and there are many types. so better to leave it up to another optional module to handle it.

[DonikaV]

{ Error: connect EHOSTUNREACH 0.0.254.209:80 - Local (MY_IP:61174)
    at internalConnect (net.js:881:16)
    at defaultTriggerAsyncIdScope (internal/async_hooks.js:294:19)
    at GetAddrInfoReqWrap.emitLookup [as callback] (net.js:1028:9)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:61:10)
  errno: 'EHOSTUNREACH',
  code: 'EHOSTUNREACH',
  syscall: 'connect',
  address: '0.0.254.209',
  port: 80 }

[mh4ck]

ah, so I tried using https-proxy-agent instead, since ytdl-core makes https requests, and it worked.

const ytdl = require('..');
const Agent = require('https-proxy-agent');

const proxy = 'http://xxx.xxx.xxx.xxx:xxxx';
const agent = new Agent(proxy);
const stream = ytdl('https://www.youtube.com/watch?v=2UBFIhS1YBk', {
  requestOptions: { agent }
});

console.log('Starting Download');

stream.on('data', (chunk) => {
  console.log('downloaded', chunk.length);
});

stream.on('end', () => {
  console.log('Finished');
});

I can update the examples to show usage with https-proxy-agent instead of using requestOptions.transform since handling proxies can be complicated, and there are many types. so better to leave it up to another optional module to handle it.

How to add Authorization to this example?

[DonikaV]

@mh4ck I tried this
https://github.com/TooTallNate/node-https-proxy-agent/issues/12#issuecomment-216098644 but not working for me

[mh4ck]

@mh4ck I tried this
TooTallNate/node-https-proxy-agent#12 (comment) but not working for me

And if you use "Authentication" instead of "Proxy-Authentication" ?

[DonikaV]

at internalConnect (net.js:881:16)
at defaultTriggerAsyncIdScope (internal/async_hooks.js:294:19)
at GetAddrInfoReqWrap.emitLookup [as callback] (net.js:1028:9)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:61:10)

errno: 'EHOSTUNREACH',
code: 'EHOSTUNREACH',
syscall: 'connect',
address: '0.0.254.209',
port: 80 }

[DonikaV]

const agent = new Agent({
    host,
    port,
    headers: {
        username,
        password,
    },
});

Error: Status code: 407
    at ClientRequest.httpLib.get (/Users/donika/Sites/mp3-ytb/node_modules/miniget/dist/index.js:128:27)
    at Object.onceWrapper (events.js:286:20)
    at ClientRequest.emit (events.js:198:13)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:556:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:109:17)
    at Socket.socketOnData (_http_client.js:442:20)
    at Socket.emit (events.js:198:13)
    at Socket.Readable.read (_stream_readable.js:491:10)
    at flow (_stream_readable.js:957:34)
    at resume_ (_stream_readable.js:938:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)

[DonikaV]

If i try to use some free proxy without auth

getInfo(videoId, {
          requestOptions: {
            transform: (parsed) => {
              return {
                host: '41.213.13.154',
                port: 4145,
                path: parsed.href,
                headers: { Host: parsed.host },
              };
            },
          },
        })

{ Error: connect ECONNREFUSED 41.213.13.154:4145
at Object._errnoException (util.js:1031:13)
at _exceptionWithHostPort (util.js:1052:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1195:14)
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '41.213.13.154',
port: 4145 }

[fent]

headers doesn't take username/password, look at the https-proxy-agent issue that was linked. and you probably still need https-proxy-agent for proxies without auth

[DonikaV]

Just found first proxy from google https://free-proxy-list.net/

const agent = new Agent("8.209.83.91:3128");
getInfo(videoId, {
          requestOptions: { agent },
        }).then(function successCallback(response) {
          res.json(response).end();
          convertAndDownload(userBitrate, response, res, socket, fileName);
      }, function errorCallback(response) {
          console.log(response);
          res.status(500).json({error: errMsg});
      });

Error: connect EHOSTUNREACH 0.0.12.56:80 - Local (192.168.88.247:63344)
at internalConnect (net.js:881:16)
at defaultTriggerAsyncIdScope (internal/async_hooks.js:294:19)
at GetAddrInfoReqWrap.emitLookup [as callback] (net.js:1028:9)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:61:10)
errno: 'EHOSTUNREACH',
code: 'EHOSTUNREACH',
syscall: 'connect',
address: '0.0.12.56',
port: 80 }

[fent]

I just tried a couple of random proxies from there, took a few minutes to start up but eventually it did start downloading.

const ytdl = require('..');
const Agent = require('https-proxy-agent');

const host = '177.54.130.149';
const port = '59044';
const proxy = `http://${host}:${port}`;
const agent = new Agent(proxy);
const stream = ytdl('https://www.youtube.com/watch?v=2UBFIhS1YBk', {
  requestOptions: { agent }
});

console.log('Starting Download');

stream.on('data', (chunk) => {
  console.log('downloaded', chunk.length);
});

stream.on('end', () => {
  console.log('Finished');
});

I noticed that the ones that don't support https give an error, even if I try using the request module just to make sure

const request = require('request');

const url = 'https://www.youtube.com/watch?v=2UBFIhS1YBk';
const host = '188.226.141.127';
const port = '3128';
const proxy = `http://${host}:${port}`;

request({ url, proxy }, (err, res) => {
  if (err) throw err;
  console.log('got res', res.statusCode);
});

[DonikaV]

@fent Yes, true, after couple of minutes it start download. But what i can do if i have 'premium' pfoxies from service? Where i need use login and password?

[fent]

did you try https://github.com/TooTallNate/node-https-proxy-agent/issues/12#issuecomment-216098644

[DonikaV]

Just tried again and it works! thanks!! you made my evening!!!!