Selenium: Download file in headless chrome

[jonhkr]

Meta -

OS: OSX
Selenium Version: 3.8.1
Browser: Chrome

Expected Behavior -

Permit files to be downloaded in headless mode.

Actual Behavior -

Files aren't downloaded.

Steps to reproduce -

Launch chrome in headless mode and try to download any file.

Looks like it is a feature to prevent sites from downloading files when running chrome in headless mode.
https://bugs.chromium.org/p/chromium/issues/detail?id=696481

To permit downloads it's necessary to send a command to chrome.
I'm using java and I couldn't find a way to make it work.

I even tried sending a post request to /session/:sessionId/chromium/send_command but that didn't work.

[p0deje]

Apparently, this new command needs to be added to all the bindings.

[jonhkr]

I managed to get it working by issuing the above request directly to the chromedriver but using a development build of chromium.

[jonhkr]

This is how I did it:

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

ChromeDriver driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");

Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepathString);
commandParams.put("params", params);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);

String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

[nstubi]

Hello,

Any plans to support setDownloadBehavior in order to download files using chrome in headless mode?

Any doc/examples in .Net would be much appreciated.

Regards,
Nicolas

[shreyanshp]

Even I am facing this same issue

[daljitcheema]

Has anyone got this working with Ruby/Rails? Also, can you get a development build of chrome on Heroku?

[pulkitsharma07]

@p0deje, @jonhkr
I am working on integrating this with the Ruby bindings, but for some reason I am not able to download files in headless mode.

Could you please verify this spec, it is failing for me: https://github.com/pulkitsharma07/selenium/commit/1142e0b5ff859ffea3b5b3fccad23433ad08089f#diff-b1ade8967288d0002579a914345a9f56R72

For me even https://github.com/SeleniumHQ/selenium/blob/master/javascript/node/selenium-webdriver/test/chrome/devtools_test.js#L51 is failing.

I am using Chrome 64 and chromedriver 2.35, not sure what could be causing this.

[luke-hill]

I've popped an optimisation in, also, whilst not having worked in this area of the specs. Is there a reason you verbosely create the driver, there are helpers such as create_driver! which will check to see if an instance exists and/or create one.

[pulkitsharma07]

No particular reason, I missed that. The branch is not yet ready, Will do the changes to use create_driver!.

But, still, the main issue is that I am not able to download files in headless mode.

[jbustillos]

Is there an example for .NET?

[p0deje]

@pulkitsharma07 I cannot make it work too, maybe something has changed in recent Chrome/ChromeDriver?

[pulkitsharma07]

So, in my headless tests Chrome is crashing on clicking the element to download the file, whereas it is not crashing with the same scenario in normal mode.

chromedriver logs:

  [1518706724.071][DEBUG]: DEVTOOLS COMMAND Input.dispatchMouseEvent (id=32) {
     "button": "left",
     "clickCount": 1,
     "modifiers": 0,
     "type": "mouseReleased",
     "x": 20,
     "y": 16
  }
  [1518706724.374][SEVERE]: Unable to receive message from renderer
  [1518706724.374][INFO]: Waiting for pending navigations...
  [1518706724.374][INFO]: Done waiting for pending navigations. Status: disconnected: not connected to DevTools
  [1518706724.374][DEBUG]: DevTools request: http://localhost:12345/json
  [1518706724.375][DEBUG]: DevTools request failed
  [1518706724.375][INFO]: RESPONSE ClickElement

[pulkitsharma07]

@p0deje I have filed https://bugs.chromium.org/p/chromedriver/issues/detail?id=2270. Please take a look.

[pulkitsharma07]

Upgrading to Chrome 65/chromedriver 2.36 fixed my crash issue with headless chrome.

[lmtierney]

After talking with the chromedriver team, the send_command will be moved to a w3c-compliant protocol extension endpoint (https://w3c.github.io/webdriver/webdriver-spec.html#protocol-extensions).

See the issue raised in the chromium bugtracker: https://bugs.chromium.org/p/chromedriver/issues/detail?id=2307

[p0deje]

Ruby landed in master using Driver#download_path=, so we might need to refactor it once Chromium changes to W3C compliant commands.

[thehme]

I just realized that my js selenium webdriver test, which was checking the downloads page in the Chrome browser, does not work when I run it in headless mode. From the conversation above, I gather this is not yet supported, correct?

[guigs]

It works for me on Ruby with current master.

[oriolgual]

Worked for me with Ruby and Capybara:

page.driver.browser.download_path = YOUR_DOWNLOAD_PATH
click_link "Download stuff"

[jackkinsella]

page.driver.browser.download_path

Technically this is available via the selenium-webdriver gem, as of v 3.13.0

Here's the implementation:

          params = {
            'cmd' => 'Page.setDownloadBehavior',
            'params' => {
              'behavior' => 'allow',
              'downloadPath' => path
            }
          }
          @bridge.send_command(params)

[Elangopalakrishnan]

@jackkinsella

How we can implement above solution in c# script coz i need to download pdf and validate in chrome headless browser mode..

Please guide me on this..

Thanks in advance,
Elango

[jbustillos]

Please provide a c# example using selenium.

El 28 ene. 2019 8:15 a. m., Epk [e-mail] escribió:

@jackkinsellahttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjackkinsella&data=02%7C01%7C%7C59daa4d67e094f9da93c08d6852b1dc9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636842817554656333&sdata=QsU6Z6qBNfaou9sb2AhwNtlOT8paFAqDrf2sL4XJ%2Bks%3D&reserved=0

How we can implement above solution in c# script coz i need to download pdf and validate in chrome headless browser mode..

Please guide me on this..

Thanks in advance,
Elango

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSeleniumHQ%2Fselenium%2Fissues%2F5159%23issuecomment-458147308&data=02%7C01%7C%7C59daa4d67e094f9da93c08d6852b1dc9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636842817554666338&sdata=VlgjyYp8I%2Fgls4Q65mseyO4mWeh9fE3UK4%2FoxLqd31M%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANIV2zat0Y6xKr1cPP-TPL84JAYMYWZ0ks5vHwYagaJpZM4Qzgxf&data=02%7C01%7C%7C59daa4d67e094f9da93c08d6852b1dc9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636842817554676343&sdata=XwBBLg%2BPxlvxL7iefd1VqnVstoUdwkfZcjDkeuRTx3A%3D&reserved=0.

[jackkinsella]

Sorry, don't know C#. I just provided the Ruby implementation as a reference to anyone who is facing the same issue and wanted to translate to their own environment.

[Elangopalakrishnan]

@jackkinsella @jbustillos I have tried in c# please have a look into,, https://github.com/SeleniumHQ/selenium/issues/6895 , Please let me know if you get any idea on this... :)

[poojaghosh47]

Hi All,

Thanks for the solution : It worked for me for HeadlessChrome Browser Even for Local windows as well as in Linux machine , And Also on jenkins server... :)
For below configuration 👍
Selenium java Version : 3.12.0, Chrome Browser version : 72.0.3626, ChromeDriver verison : 2.46 ,Linux Ubuntu 16.4 ,64bit )

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

ChromeDriver driver = new ChromeDriver(driverService, options);

Map commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");

Map params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepathString);
commandParams.put("params", params);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);

String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

[HaveSpacesuit]

Here is a .NET implementation to enable downloads using headless chrome:

ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.AddArgument("--headless");

ChromeDriver chromeDriver = new ChromeDriver(chromeOptions);

Dictionary<string, object> commandParameters = new Dictionary<string, object>
{
    ["behavior"] = "allow",
    ["downloadPath"] = @"C:\Downloads"
};

chromeDriver.ExecuteChromeCommand("Page.setDownloadBehavior", commandParameters);

[rishi1992]

This is how I did it:

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

ChromeDriver driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");

Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepathString);
commandParams.put("params", params);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);

String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

This fix is not working with my configuration. I'm still unable to download a file in headless mode.

[rishi1992]

public static WebDriver getChromeDriver() throws IOException {
//Initialize ChromeDriver

    ChromeOptions options = new ChromeOptions();
    Map<String, Object> prefs = new HashMap<>();
    //String downloadFilepath = System.getProperty("user.dir");
    prefs.put("safebrowsing.enabled", false);
    prefs.put("download.default_directory", System.getProperty("user.dir"));
    prefs.put("download.prompt_for_download", false);
    prefs.put("download.directory_upgrade", true);
    prefs.put("cmd", "Page.setDownloadBehavior");
    prefs.put("profile.default_content_settings.popups", 0);
    prefs.put("behavior", "allow");
    options.setExperimentalOption("prefs", prefs);
    options.addArguments("--disable-notifications");
    options.addArguments("--start-maximized");
    options.addArguments("disable-infobars");
    options.addArguments("--disable-gpu");
    options.addArguments("--headless");
    options.addArguments("window-size=1980,1080");
    options.addArguments("--allow-running-insecure-content");
    options.addArguments("--disable-extensions");
    options.addArguments("--no-sandbox");
    options.addArguments("--test-type");
    options.addArguments("--disable-web-security");
    //options.setBinary("C:\\Program Files (x86)\\Google\\Chrome");
    String chromedriverpath_1 = System.getProperty("user.dir");
    String chromedriverpath = chromedriverpath_1+"//chromedriver_2_44.exe";
    //System.out.println(chromedriverpath);
    System.setProperty("webdriver.chrome.driver", chromedriverpath);
    //System.setProperty("user.dir","//chromedriver_2.44.exe");



    ChromeDriverService driverService = ChromeDriverService.createDefaultService();
    ChromeDriver driver = new ChromeDriver(driverService, options);


    ObjectMapper objectMapper = new ObjectMapper();
    HttpClient httpClient = HttpClientBuilder.create().build();
    String command = null;
    try {
        command = objectMapper.writeValueAsString(prefs);
    } catch (JsonProcessingException e) {
        e.printStackTrace();
    }

    String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
    HttpPost request = new HttpPost(u);
    request.addHeader("content-type", "application/json");
    request.setEntity(new StringEntity(command));
    httpClient.execute(request);
    return driver;
}

[amishad02]

@rishi1992 did you found any solution can you please share.

[JDCain]

@amishad02 I use the following successfully

var driver = new ChromeDriver(driverService, options);

var param = new Dictionary<string, string> {{"behavior", "allow"}, {"downloadPath", DownloadPath}
};

var cmdParam = new Dictionary<string, object> {{"cmd", "Page.setDownloadBehavior"}, {"params", param}};

var url = driverService.ServiceUrl + "session/" + driver.SessionId + "/chromium/send_command";
var cli = new WebClient {Headers = {[HttpRequestHeader.ContentType] = "application/json"}};
_ = cli.UploadString(url, JsonConvert.SerializeObject(cmdParam));

[pouellet]

Anyone had success sending the /session/:sessionId/chromium/send_command command to a headless chrome process that was launched in a selenium grid? This solution seems to work well for a standalone selenium chrome, but I couldn't get it to work when proxied through the selenium grid.

[rishi1992]

This has worked for me,

// Set the ChromeDriver path
System.out.println("launching Chrome browser");
String chromedriverPath = System.getProperty("user.dir") + "\chromedriver_2_44.exe";
System.setProperty("webdriver.chrome.driver", chromedriverPath);
// Set the download folder path
String downloadPath = System.getProperty("user.dir");
//String downloadPath = "E:\seleniumdownload";
HashMap chromePrefs = new HashMap();
chromePrefs.put("safebrowsing.enabled", "true");
chromePrefs.put("downloadPath", downloadPath); //This is one of the main line to be written
chromePrefs.put("behavior", "allow"); //This is one of the main line to be written
// Initialize Chrome options
ChromeOptions options = new ChromeOptions();
options.addArguments("--disable-notifications");
options.addArguments("--start-maximized");
options.addArguments("disable-infobars");
options.addArguments("--disable-gpu");
options.addArguments("--headless");
options.addArguments("window-size=1980,1080");
options.addArguments("--allow-running-insecure-content");
options.addArguments("--disable-extensions");
options.addArguments("--no-sandbox");
options.addArguments("--ignore-certificate-errors");
// Initialize driver
ChromeDriverService driverService = ChromeDriverService.createDefaultService();
driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");
commandParams.put("params", chromePrefs);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);
String u = driverService.getUrl().toString() + "/session/" + ((RemoteWebDriver) driver).getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

[automationleg]

Any good solution/workaround in Python ?
I tried many ideas posted here and over internet, but nothing seems to work. I have a scenario when I just click on a download button and file get's downloaded(what is not happening in headless).

[cgoldberg]

to enable headless downloads in Python:

from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True 
driver = Chrome(options=options)
params = {'behavior': 'allow', 'downloadPath': '/path/for/download'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)

[automationleg]

Thanks cgoldberg - that works fine!

[deeptechs]

to enable headless downloads in Python:

from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True 
driver = Chrome(options=options)
params = {'behavior': 'allow', 'downloadPath': '/path/for/download'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)

@cgoldberg What is your env? Chrome and chromedriver versions, standalone selenium or Grid?

Thanks

[ander7agar]

to enable headless downloads in Python:

from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True 
driver = Chrome(options=options)
params = {'behavior': 'allow', 'downloadPath': '/path/for/download'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)

@cgoldberg I am using this similar procedure with selenium-webdriver for Node and it does not work for me. I suspect that it is because in my case a new tab is opened to download file (target "_blank"). This is executed through JavaScript code when I click on a button.

Does this happen to anyone else?

[ander7agar]

Is there any way to make this permanent? That is, I want it to be enabled by default when a new instance of the browser is opened.

[ckempster]

Is there any way to make this permanent? That is, I want it to be enabled by default when a new instance of the browser is opened.

Make use of Hooks

[ckempster]

Is there any way to make this permanent? That is, I want it to be enabled by default when a new instance of the browser is opened.

Make use of Hooks

An example I call from my hooks is 👍
private void createChromeBrowser() { WebDriverManager.chromedriver().setup(); String downloadPath = System.getProperty("user.dir"); HashMap<String, Object> chromePrefs = new HashMap<String, Object>(); chromePrefs.put("safebrowsing.enabled", "true"); chromePrefs.put("downloadPath", downloadPath); chromePrefs.put("behavior", "allow"); ChromeOptions options = new ChromeOptions(); options.addArguments("--headless"); options.addArguments("--window-size=1366,768"); options.addArguments("--disable-infobars"); options.addArguments("--incognito"); options.addArguments("--enable-javascript"); options.addArguments("--disable-websecurity"); options.addArguments("--start-maximised"); options.addArguments("--ignore-certificate-errors"); options.addArguments("--disable-popup-blocking"); options.addArguments("--fast-start"); options.addArguments("--javascript-harmony"); options.addArguments("--no-sandbox"); driver = new ChromeDriver(options); }

[jonsamwell]

The command does not seem to be working now in Chrome 74+ (using Proctractor 5.4.2 and node 10.16.0)

browser.driver.sendChromiumCommand('Page.setDownloadBehavior', {
      behavior: 'allow',
      downloadPath: fullDownloadDirectoryPath
    });

Fails with the error

UnsupportedOperationError: POST /session/ac771ec3019f7b4c7191e2c9ae2b2056/chromium/send_command

Any idea why it is failing or how to get headless Chrome downloads working again?

[lmtierney]

Because the endpoint for that command changed in chromedriver to /session/:sessionId/goog/cdp/execute

[jonsamwell]

@lmtierney many thanks for this. I am still getting a 500 internal server error now if I use that endpoint uri instead?

http://192.168.1.189:54193/wd/hub/session/d47c025d923e19c77aacab20f69809f6/goog/cdp/execute

[aditya2020]

This is how I did it:

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

ChromeDriver driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");

Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepathString);
commandParams.put("params", params);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);

String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

This fix is not working with my configuration. I'm still unable to download a file in headless mode.

I agree. My Chrome version is "78.0.3904.70". This doesnt work for me either. I have the same code

[NN88]

This worked for me:

Environment

ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin18]
Rails 4.2.10
chromedriver: 79.0.3945.36
webdrivers (4.0.1)
capybara (3.24.0)

Other resources

https://stackoverflow.com/questions/48810757/setting-default-download-directory-and-headless-chrome

Implementation

options = Selenium::WebDriver::Chrome::Options.new
  options.add_argument('--window-size=1500,1500')
  options.add_argument('--headless')
  options.add_argument('--no-sandbox')
  options.add_argument('--disable-gpu')
  options.add_argument('--disable-popup-blocking')

  options.add_preference(:download,
                          directory_upgrade: true,
                          prompt_for_download: false,
                          default_directory: ENV['downloads_folder']
                          )

  options.add_preference(:browser, set_download_behavior: { behavior: 'allow' })

Capybara.register_driver :headless_chrome do |app|
  driver = Capybara::Selenium::Driver.new(
                                app,
                                browser: :chrome,
                                options: options
                                )


  bridge = driver.browser.send(:bridge)
  path = '/session/:session_id/chromium/send_command'
  path[':session_id'] = bridge.session_id

  bridge.http.call(:post, path, cmd: 'Page.setDownloadBehavior',
                                params: {
                                  behavior: 'allow',
                                  downloadPath: ENV['downloads_folder'],
                              })

  driver
end

Capybara.javascript_driver = ENV['capybara_js_driver'].to_sym

[sanathfp]

I'm trying to get this working on javascript using nodejs. Is there a solution yet?