Sql-docs: Azure Blob Storage from SQL Server on Linux

Created on 16 May 2019 · 26Comments · Source: MicrosoftDocs/sql-docs

The first paragraph states The backup functionality is the same across platforms. For example, you can backup databases locally, to remote drives, or to Microsoft Azure Blob storage service. Is it really correct that Azure Blob Storage is supported on Linux?

I tried following documentation here https://docs.microsoft.com/en-us/sql/relational-databases/tutorial-sql-server-backup-and-restore-to-azure-blob-storage-service?view=sql-server-2017, but I receive error:

Msg 3292, Level 16, State 6, Line 2
A failure occurred while attempting to execute Backup or Restore with a URL device specified. Consult the operating system error log for details.
Msg 3013, Level 16, State 1, Line 2
BACKUP DATABASE is terminating abnormally.

Using cat /var/opt/mssql/log/errorlog shows (<dbname> replaced):

BACKUP failed to complete the command BACKUP DATABASE &lt;dbname&gt;. Check the backup application log for detailed messages.
2019-05-16 17:44:20.81 spid53      VDI: "C:\binn\BackupToUrl.exe" "b" "p" "680074007400700073003A002F002F0061007A006400610063007500730071006C006200610063006B00750070002E0062006C006F0062002E0063006F00720065002E00770069006E0064006F00770073002E006E00650074002F00700072006F0064002F0045006700670049006E007300700065006300740069006F006E002E00620061006B00" "61007A006400610063007500730071006C006200610063006B0075007000" "01000000D08C9DDF0115D1118C7A00C04FC297EB0100000071DD94E6A43C7E46898CEE110058026F000000001200000061007A007500720065006B00650079000000106600000001000020000000EFE9E128B1ACFB77EBFB5681D9650528430EBF879CE32318425655640A376801000000000E80000000020000200000009DC4790D94DF8E8294B2DE5FDE9BE6246B13CA5F39E57EEB13FF1676E11F804E5000000012C97B1BCEE0912FE948B3B1ADA62324A83B78150A2B9976BC5DE18D52FB4BBEC89B4F46062116E362CFD16286D68682B6018C94C6D06B063157854D5186CE3D6A45EF86386E26D8FAB351EBDA60744240000000ADC39B9047969C4B30BDC84CD2BA56F85AF15CADC3E5CCBF56890FC21D49AAA87FE84131667366A5C0F8BAD85D67418E2FB745B6F63D88D6456F8DE21D338DB4" "NOFORMAT" "4D005300530051004C00530045005200560045005200" "" "DB" "45006700670049006E007300700065006300740069006F006E00" "TRACE"
2019-05-16 17:44:20.82 spid53      BackupToUrl: couldn't load process Error Code:  80070002
2019-05-16 17:44:20.82 Backup      Error: 3041, Severity: 16, State: 1.

Please clarify the statement that Backup to URL (Azure blob storage) is really supported on all platforms.
Running SQL Server 2017 Web on Ubuntu.

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: aa12baf4-fafe-817e-e3f2-1fe94a77a83e
Version Independent ID: 6224becb-b6be-0bf4-5e7a-b75d7f6d07c4
Content: Backup and restore SQL Server databases on Linux - SQL Server
Content Source: docs/linux/sql-server-linux-backup-and-restore-database.md
Product: sql
Technology: linux
GitHub Login: @MikeRayMSFT
Microsoft Alias: mikeray

Pri2 assigned-to-author doc-bug linutech sqprod

Source

dahovey

Most helpful comment

@MikeRayMSFT This needs further clarification. My Linux backup is still looking for "C:\binn\BackupToUrl.exe". That's insane. Worse, your lack of clarity causes readers to try myriad permutations of what you could possible mean. Telling people to use block blobs, not page blobs in incredibly unhelpful since that's not even remotely how that works. At one point you say block blob account, at another you say block blob container (neither is how this works). If you mean ADLS2, say ADLS2, if you mean StorageV2, say StorageV2, if you mean BlobStorage, say BlobStorage. Furthermore, before publishing, verify what does NOT work, then clarify what NOT to do (e.g. do NOT cut the RED wire, but the PURPLE wire). All you're doing is bragging about what works on your machine. Not helpful. I invite you to try again.

davidbetz on 20 Jul 2020

👍2

All 26 comments

Hello @dahovey! Thank you for reach out to Microsoft on this Documentation forum. It seems that your question is more of a Support issue, but I'll have my colleague, Mike verify.

@MikeRayMSFT - please help verify that Azure Blob Storage is supported for backup in Linux. Thanks!

VanMSFT on 17 May 2019

Hi @dahovey - thank you for the submission. I think there maybe something we can address with docs. I will get back to you soon.

MikeRayMSFT on 20 May 2019

@MikeRayMSFT Any word on this? Thanks!

visitpay on 3 Jun 2019

I'm putting together an answer for the documentation, but here's what I've got so far:

Backup to page blob isn't supported as it uses older mechanism (using a separate binary, backuptourl.exe). Backup to block blob should work just fine since that method is integrated into engine itself. I'm working on an update for the documentation. We should have an update in ~ 1 week. In the mean time you can try using backup to block blob and it should just work.

Quickstart: SQL Server Backup and Restore to Azure Blob Storage Service

MikeRayMSFT on 5 Jun 2019

Creating a work item to track 1554601

please-close

MikeRayMSFT on 12 Jun 2019

I just updated to CU15 - 14.0.3162.1. Tried again after update, and issue still persists.

dahovey on 18 Jun 2019

I am having this same problem trying to restore from Azure into a linux docker container. Is there a fix coming soon and when?

Daemach on 17 Jul 2019

Same issue when trying to restore page blobs to ubuntu linux 16.04 with SQL Server Express 2017.

@MikeRayMSFT mentions

Backup to page blob isn't supported as it uses older mechanism (using a separate binary, backuptourl.exe)

Just curious if this will ever be supported and if there is a timeline for it.

jakedenyer on 29 Aug 2019

What is the status here. We are testing SQL Server 2017 on Ubuntu in Azure and we are not able to restore from azure storage. As soon as we select the storage container in SSMS an error is returned:
The remote server returned an error: (403) Forbidden.

nicolaibaralmueller on 10 Feb 2020

Reopening this issue because there's still confusion. Will track down straightway.

MikeRayMSFT on 2 Mar 2020

Thanks all for monitoring this thread - I recognize it's old, but I want to make sure the issues are resolved.

I went through the steps in the tutorial with success on a (Ubuntu 16.04 with SQL Server 2017 from Azure marketplace to a storage account in a different resource group.) My storage account was in the same region with the default configuration. I had the latest (SSMS v 18.4).
Restore was successful. Required the SAS key.

Storage container must be a block blob.

I'm going to keep this issue open for a week or so in case anyone wants to chime in.

We're working on an update to the article to clarify a few things, but at this moment it works as designed.

MikeRayMSFT on 3 Mar 2020

👎1

Are you saying that we can backup to and restore from Azure using SQLS for Linux? I gave up and started using S3 because I could never get it to work. Which tutorial are you referencing?

Daemach on 3 Mar 2020

👍1

@Daemach, yes, Backup to Azure using SQL Server for Linux works. The Storage account has to use block blob (not page plobs).

I'm referencing Quickstart: SQL backup and restore to Azure Blob storage service.

I should have called it a Quickstart and not a Tutorial.

MikeRayMSFT on 4 Mar 2020

I followed this quickstart to get it working on windows. But it would not
work on SQLS for Linux after trying for a week. Is this something that has
changed in the past 8 months since I commented on it? We ended up choosing
AWS over Azure for our entire infrastructure because of this issue. Our
backups are large enough that we needed to keep them on the same network.

On Wed, Mar 4, 2020 at 10:42 AM Mike Ray notifications@github.com wrote:

@Daemach https://github.com/Daemach, yes, Backup to Azure using SQL
Server for Linux works. The Storage account has to use block blob (not page
plobs).

I'm referencing Quickstart: SQL backup and restore to Azure Blob storage
service
https://docs.microsoft.com/sql/relational-databases/tutorial-sql-server-backup-and-restore-to-azure-blob-storage-service
.

I should have called it a Quickstart and not a Tutorial.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/sql-docs/issues/2150?email_source=notifications&email_token=AC7Y5YPLXQD7OESZYFIQF53RF2OJLA5CNFSM4HNO5342YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENZQOOI#issuecomment-594741049,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC7Y5YMO7HRRMFTVRJ4E5TDRF2OJLANCNFSM4HNO534Q
.

Daemach on 4 Mar 2020

@Daemach @dahovey have you considered https://docs.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux blobfuse

cawrites on 29 Mar 2020

@dahovey thank you for contacting Microsoft regarding your question. It has been quite some time since your original request and it appears there are a few suggestions to try if this is still an issue for you. Please don't hesitate to contact us again if you have a question regarding the documentation.

cawrites on 3 Apr 2020

please-close

cawrites on 3 Apr 2020

@MikeRayMSFT Hi, I tested this on these versions on docker:
2017-CU20-ubuntu-16.04
2019-CU4-ubuntu-16.04
and have the next errors:

VDI: "C:\binn\BackupToUrl.exe" "b" "p" "680074007400700073003A002F002F006800650074007A006E006500720063006C006F00750064006200690034006D0065002E0062006C006F0062002E0063006F00720065002E00770069006E0064006F00770073002E006E00650074002F00730071006C002D00310063002D00300031002F006500720070002D006300700070002E00620061006B00" "6800650074007A006E006500720063006C006F00750064006200690034006D006500" "0100000000000000000000000000000000000000010000000000000000000000000000000000000023FAC7FAC67455D8FB20FE9CD693074DEB92EC6CC7BF4E2479A083AB1644CB87FF48193D85D17A8079A779651C9E4EBE7EDE59E57EEF22928AC7E29929D684168603C81F1B8228432D615B6FD4218667" "FORMAT" "4D005300530051004C00530045005200560045005200" "" "DB" "7400650073007400" "NOTRACE"
BackupToUrl: couldn't load process Error Code:  80070002
2020-04-22 12:54:07.23 Backup      Error: 3041, Severity: 16, State: 1.
2020-04-22 12:54:07.23 Backup      BACKUP failed to complete the command BACKUP DATABASE test. Check the backup application log for detailed messages.

VDI: "C:\binn\BackupToUrl.exe" "b" "p" "680074007400700073003A002F002F006800650074007A006E006500720063006C006F00750064006200690034006D0065002E0062006C006F0062002E0063006F00720065002E00770069006E0064006F00770073002E006E00650074002F00730071006C002D00310063002D00300031002F006500720070002D006300700070002E00620061006B00" "6800650074007A006E006500720063006C006F00750064006200690034006D006500" "01000000D08C9DDF0115D1118C7A00C04FC297EB01000000EAC005F6570CE84BB0FD58AF68466574000000001200000061007A007500720065006B00650079000000106600000001000020000000030340941878238F79C494C75A98E4C72CE424AC7BD0B5122E277226260D2B77000000000E8000000002000020000000189DFAE10A26935C393ECF9B68168A9CE485937EDF0D1448680062A6BCC6602A500000005095E378BBF31F45B4B4ED9B43C07D4DE13FD0E44D4DF158276B81CCF9C61311BF33FBF9EC8EEAB8B68C4DB72578B36A808CA9939781F6411312C2015ED8112EFBF59CFA360758C697AB23D19835334B4000000001B568690AA99DE0F711A3363E8A2801F29EA6815D8B2CF4148E65557222B677827DF76CBC950E85E7BEA4BA3F0FA7D818DD17270CB4F5ADAEDE8EE10FE464B9" "FORMAT" "4D005300530051004C00530045005200560045005200" "" "DB" "7400650073007400" "NOTRACE"
BackupToUrl: couldn't load process Error Code: 80070002
2020-04-22 13:02:57.21 Backup Error: 3041, Severity: 16, State: 1.
2020-04-22 13:02:57.21 Backup BACKUP failed to complete the command BACKUP DATABASE test. Check the backup application log for detailed messages.
```

HryhoriiDidenko on 22 Apr 2020

👍1

davidbetz on 20 Jul 2020

👍2

Any update on this? I'm having the same errors, it seems it's calling the BackupToUrl process in a windows path when using a linux container.

yosbeleg89 on 1 Oct 2020

The lack of clarity is all I am seeing in this thread and the documentation.

Any backup using "WITH CREDENTIAL" uses Page Blob, not supported on Linux. When that is used, it looks for the backuptourl.exe.

Create and use a Shared Access Signature to use block blob.

I was setting up my backups like I had done on Windows and it was not working, looking for the .exe above.

Tested on Ubuntu & SQL 2019 CU6

skrewloose on 3 Oct 2020

I tested with SAS and it works (2017-CU21-ubuntu-16.04), it's kind of weird anyway that it is being called C:\binn\BackupToUrl.exe on Linux, instead it should give a more clear error.

yosbeleg89 on 5 Oct 2020

@skrewloose - thanks for posting a solution.

@yosbeleg89, @HryhoriiDidenko & @davidbetz, I'm reopning for additional review.

@tejasaks & @VanMSFT - could you please review?

MikeRayMSFT on 5 Oct 2020

SQL Server on Linux only supports backing up to Azure Blob storage using block blobs. Using a storage key for backup and restore will result in a page blog being used, which isn't supported. Use a Shared Access Signature instead. For information on block blogs versus page blogs, see Backup to block blob vs. page blob

VanMSFT on 5 Oct 2020

The documentation for SQL Server Linux does not make it clear that the
Linux version ONLY supports block blobs. This paragraph makes it sound
like it should support both. Please clarify so others don't experience the
same problem. This issue has been open for months so clearly more can be
done.

Backup to block blob vs. page blob

There are two types of blobs that can be stored in the Microsoft Azure Blob
storage service: block and page blobs. SQL Server backup can use either
blob type depending upon the Transact-SQL syntax used: If the storage key
is used in the credential, page blob will be used; if the Shared Access
Signature is used, block blob will be used.

On Mon, Oct 5, 2020 at 11:54 AM PRMerger19 notifications@github.com wrote:

Closed #2150 https://github.com/MicrosoftDocs/sql-docs/issues/2150.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/sql-docs/issues/2150#event-3841954630,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC7Y5YJWML2UXBDKA5RK4GLSJII5BANCNFSM4HNO534Q
.