Mssql-docker: Enabling Machine Learning Service In mssql-docker

[shankarshastri]

Wanted to know how to enable Machine Learning Service (In-Database)?

[forkazmoden]

Hi,

SQL Server Machine Learning Services runs on Linux operating systems starting in the preview release of SQL Server 2019. Please refer to the document from Install SQL Server 2019 Machine Learning Services (R, Python, Java) on Linux.

[shankarshastri]

@forkazmoden, does it mean it's not available before 2019 in Linux version and only available in 2017 windows? I actually wanted to try it on docker containers.

[twright-msft]

Yes, that's correct. And even then, it's not installed in the Windows containers we provide today and we havent tested it in containers yet. Best to install it natively on Windows (2017) or Windows/Linux (2019 preview) VM for now. We'll containerize it on Linux pretty soon.

[shankarshastri]

Thanks, @twright-msft. Hope it will happen soon.

[DeppLearning]

Any news on this since CTP 2.3? According to the release notes sp_execute_external_script should work now.

I tried running a R script on an SQL server sporting the CTP 2.3 ubuntu release but get this:
SQL Server was unable to communicate with the LaunchPad service for request id: E66F4BF9-3991-4C93-A151-FCB76F625687. Please verify the configuration of the service.

When following the instructions for configuring under linux, I always get to the point where I have to restart the sql server using systemctl, for example to enable external scripts.

How is one supposed to enable external scripts? I tried by enabling it in a fresh docker container via sqlcmd/SSMS, exporting the master db and adding it to a new docker image. I always end up at the "unable to communicate with the LaunchPad" error no matter what I try. Restarting Launchpad seems to require systemctl again

Super exited for sql server + R in docker!

[lonaysta]

Also excited to see Machine Learning Services coming to SQL Server 2019 for Linux Docker! Have been recently researching approaches for an upcoming development task, and this seems to be promising for my need. Eager to see news on it.

[lonaysta]

Seems this feature is not live as of CTP 2.4. Hope this will be available soon!

[ClientIco]

So I got this working by adding this line

RUN chown -R root:root /var/opt/mssql-extensibility

Before it would just throw out launchpad.cer: Access denied

I just wonder if I introduced a security issue by adding this in for myself.

[jgfdsa]

Hi @ClientIco

Your post put me in the right direction. I have been trying to install SQL Server 2019 + Machine learning services in Ubuntu 16 but with no luck. I also tried using the Docker as described here

I confirm that running chown -R root:root /var/opt/mssql-extensibility fix the issue: _SQL Server was unable to communicate with the LaunchPad service_

I dont think this introduces any security issue. In fact, is part of the procedure when running building the DockerFile
39...
`# set directory permissions and create default directories

RUN chown -R root:root /opt/mssql/bin/launchpadd

RUN chown -R root:root /opt/mssql/bin/setnetbr

RUN mkdir -p /var/opt/mssql-extensibility/data

RUN mkdir -p /var/opt/mssql-extensibility/log

RUN chown -R root:root /var/opt/mssql-extensibility

RUN chmod -R 777 /var/opt/mssql-extensibility`

47...

[vin-yu]

we just added these two lines to the dockerfile to get things working. @ClientIco

RUN chown -R root:root /var/opt/mssql-extensibility
RUN chmod -R 777 /var/opt/mssql-extensibility

at the moment launchpaddd and mssql run as root but in the future we will run as non-root

[lerinave]

@Clientlco @jgfdsa @vin-yu Could you please list some instructions on how to get it to work?
I have the latest docker image from mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04 running, but cannot get the following two commands work:
sudo apt-get install mssql-mlservices-mlm-py
sudo apt-get install mssql-mlservices-mlm-r
it prompts "Unable to locate package XXX" ...

[peace-blaster]

Came here with the same issue as @lerinave. Did these packages move somewhere or something?

[jlehew]

I get the unable to locate package error as well. Says it cannot load the microsoft-openmpi package

[ogsadmin]

There seems to be official documentation for this now (see https://docs.microsoft.com/de-de/sql/linux/sql-server-linux-setup-machine-learning?view=sql-server-ver15) - but I could not get them to work. Anybody else?

[cawrites]

@jlehew this is specifically for installing machine learning services in Docker.
In order for the command to work you must be in the correct directory.
Please be sure you are in the correct directory: /mssql-docker/linux/preview/examples/mssql-mlservices.

The DOCKERFILE will be located when it is executed from this path. If you have executed the 'git clone' command in a different directory please confirm.

This was previously reported as an error. Running the 'docker build' command in the wrong directory will generate an error.

[cawrites]

Documentation (https://docs.microsoft.com/sql/linux/sql-server-linux-setup-machine-learning-docker)

[jlehew]

On another thread heard the Sql Server team updated the Python/R install package after I posted here. Thought the error here might be related. Reinstalled about a week later around end of March 2020 following the documentation for Ubuntu and this time it worked. I installed Sql on aws EC2 Ubuntu instance not Docker. Thanks!

[ogsadmin]

For reference: I have reported the exact steps, my environment and the build error messages here.

[cawrites]

@jlehew thanks. the update I provided was only related to Docker. I am glad to hear you resolved your issue.

[cawrites]

@ogsadmin please refer to your specific issue for updates.

[ogsadmin]

Hmm, maybe I was not clear. My issue is not resolved - it actually looks like there are now two issues:

1. The documentation (https://docs.microsoft.com/sql/linux/sql-server-linux-setup-machine-learning-docker) is not as clear/exact it should be. This is tracked at https://github.com/MicrosoftDocs/sql-docs/issues/4742
2. The current version of the sample ML Dockerfile does not build (from https://github.com/microsoft/mssql-docker/tree/master/linux/preview/examples/mssql-mlservices). The dockerfile here has not been updated for 7 months...

I've already posted the steps to reproduce the broken build in the original documentation issue report (https://github.com/MicrosoftDocs/sql-docs/issues/4742) - do you want me to post them here again?

Thanks!

[cawrites]

@ogsadmin I am requesting that you keep a single line of communication not two. We are working on one issue and it is preferred that we don't duplicate the issue it two places.

[ogsadmin]

Agreed, sorry - I was a bit unsure whether it would make sense to continue here instead (as it now seems to be more build related than documentation related).

[cawrites]

@ogsadmin I am posting a summary comment for discovery purposes. Your environment does not meet the requirements. The documentation you referred to is for Docker in Linux. You are correct. This is a build issue, not documentation.

[ogsadmin]

@cawrites I am trying with Docker for Linux (host is Ubuntu 18.04) now. The result is a bit more promising (build succeeds), but I still cannot access the SQL server instance. Inspecting the container with docker logs reveals the following:

<snip>
2020-05-14 12:51:18,637 INFO spawned: 'sqlservr' with pid 25
2020-05-14 12:51:19,640 INFO success: sqlservr entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-05-14 12:51:20,456 DEBG 'sqlservr' stderr output:
Error: The evaluation period has expired.

2020-05-14 12:51:20,541 DEBG 'sqlservr' stderr output:
This program has encountered a fatal error and cannot continue running at Thu May 14 12:51:20 2020
The following diagnostic information is available:

         Reason: 0x00000003
         Status: 0x00000000
        Message: Unexpected call to legacy ABI.

<snip>

So basically SQL Server starts, but then throws Error: The evaluation period has expired. followed by a crash with Unexpected call to legacy ABI. .

So again: either there are steps missing in the documentation or there is something wrong with the provided sample files for building the container.

[cawrites]

@ogsadmin please make sure you are using a licensed copy of the product or the version identified as free for non-production environments.

[cawrites]

@ogsadmin I would like to request again that you only provide updates or questions to your open issue. You are asking questions specific to your environment and setup. Please post any further comments on the issue you have already noted.

[ogsadmin]

@cawrites sorry, I am confused. So we will continue at https://github.com/MicrosoftDocs/sql-docs/issues/4742?

Regarding your comment on licensing, I have been trying with MSSQL_PID=Express and MSSQL_PID=Developer, both failing in the exact same way. I've also set the ACCEPT_EULA as described. If there are other steps needed, please provide a hint - I could not find more in the docs...

[vin-yu]

This should be possible with https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-machine-learning-docker?view=sql-server-ver15

[ogsadmin]

Obviously nobody at Microsoft tested this recently. With the comments from @dbamaster regarding the Polybase sample (see https://github.com/microsoft/mssql-docker/issues/589), I finally was able to get this working.

It turned out, that the sample from the master branch refers to a time-bombed preview version of SQL server, which does not work anymore today.

If anybody stumbles over it: here is a working version using the SQL Server 2019 repositories: https://github.com/ogsadmin/mssql-docker/blob/master/linux/preview/examples/mssql-mlservices/Dockerfile

Best regards!