Are there any plans to provide this API?
My project is still rightfully so. Not everything can to use with ADO.NET like Excel ODBC
You see that even the Mono has support for it.
Att.
It's not part of .NET Standard 2.0.
CC: @saurabh500 @weshaggard @stephentoub
cc @YoungGah
@YoungGah says the code is a thin layer on top of native libraries. It should be in reference source code, so we just need it ported over.
says the code is a thin layer on top of native libraries. It should be in reference source code, so we just need it ported over
The reference source code probably only includes the managed layer. Is the native code in Windows? If so, a counterpart would be needed for Unix. Or is the native code portable? If so, we'd need to open source it and make it available as part of corefx.
@stephentoub ODBC driver is available on Windows and Linux and the API is available to be consumed by applications.
System.Data.Odbc was a managed layer calling into the ODBC APIs
ODBC driver is available on Windows and Linux and the API is available to be consumed by applications.
Excellent. And macOS as well?
No its not available on macOS.
EDIT: SqlServer ODBC driver is not available on macOS
@stephentoub & @saurabh500 Any ODBC drivers that run on Linux and MacOS should work with System.Data.Odbc layer once it is ported. There are many difference ODBC drivers that are provided by other companies.
@saurabh500
No its not available on macOS.
Do you mean that ODBC isn't available to macOS or that specific ODBC drivers aren't supported on macOS?
@terrajobst When I wrote that comment I meant that Sql ODBC driver is not available on macOS.
However there are ODBC managers for macOS which can be used with ODBC drivers. Hence this namespace implementation is feasible on Xplat.
@terrajobst When I wrote that comment I meant that Sql ODBC driver is not available on macOS.
Got it. Thanks!
Hence this namespace implementation is feasible on Xplat.
That's what I was concerned about. Sweet.
The MIT-licensed source code to port can be found here: https://github.com/Microsoft/referencesource/tree/4fe4349175f4c5091d972a7e56ea12012f1e7170/System.Data/System/Data/Odbc
FYI: as part of our effort in https://github.com/aspnet/EntityFramework/issues/7432 I'll be looking at porting this library.
On a side note, I was just looking at the DbConnectionHelper.cs
code and there is clearly some build step missing. I found how mono generates the XXXXConnection (e.g. OdbcConnection) files here: https://github.com/mono/mono/blob/master/mcs/class/System.Data/Makefile.
For the record, I believe this is missing from System.Data.SqlClient
, but maybe because they just didn't use this partial class' features. For Odbc it seems to be necessary.
On a side note, I was just looking at the DbConnectionHelper.cs code and there is clearly some build step missing. I found how mono generates the XXXXConnection (e.g. OdbcConnection) files here: https://github.com/mono/mono/blob/master/mcs/class/System.Data/Makefile.
The namespace and the Class name in DbConnectionHelper.cs were changed to the name of the provider i.e. OdbcConnection during build time in System.Data
You can copy the file over and change the name to OdbcConnection and the relevant namespace.
@saurabh500 figured it out from the Makefile :')
First passing test! Connecting, selecting and reading seem to be working fine in Windows using the native odbc32.dll
.
I'm wondering now what a good approach would be for making tests that don't depend on my own environment. Haven't yet looked at how SqlClient does it, but will do on Monday. Anyway, please let me know if you have any suggestions!
Just a quick update since I've been working on other stuff and couldn't get this merged yet:
- Support for Linux and Mac comes from unixODBC, and since it implements the ODBC standard which specifies the C interface, we don't even need to change the native function signatures.
C types can have different sizes on different platforms. Do we know that's not an issue here? This is one of the reasons we have the native shims for other libraries, so that we can have the managed APIs P/Invoke to native signatures that use types of known sizes regardless of platform, e.g. int32_t, int64_t, etc.
From what I see here, it looks like all parameters are standard-specific types, e.g. SQLUSMALLINT
which is a C# UInt16
or SQLSMALLINT
which is a C# Int16
.
unixODBC also has these types: http://www.unixodbc.org/doc/ProgrammerManual/Tutorial/gloss.html
From what I see here, it looks like all parameters are standard-specific types, e.g. SQLUSMALLINT which is a C# UInt16 or SQLSMALLINT which is a C# Int16.
The trouble is that the C standard doesn't actually define the byte size of these types, only the range of values they're required to be able to hold and the relationship between them. Thus, for example, in theory a type like int
could be 32-bit on some systems and 64-bit on others, and they'd all conform to the spec.
C# specifies the Int16 is a 16-bit integer. And ODBC specifies SQLSMALLINT
is a 16-bit integer. Am I missing something?
https://msdn.microsoft.com/en-us/library/system.int16(v=vs.110).aspx
On Sat, Feb 25, 2017, 08:34 Stephen Toub notifications@github.com wrote:
From what I see here, it looks like all parameters are standard-specific
types, e.g. SQLUSMALLINT which is a C# UInt16 or SQLSMALLINT which is a C#
Int16.
The trouble is that the C standard doesn't actually define the byte size of
these types, only the range of values they're required to be able to hold
and the relationship between them. Thus, for example, a type like int could
be 32-bit on some systems and 64-bit on others, and they'd all conform to
the spec.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/dotnet/corefx/issues/13035#issuecomment-282495062, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABeg9Fh6HKVmTBtdnl6PWbIphYoM46U1ks5rgFf7gaJpZM4KhHPs
.
If the odbc spec specifies the exact size, then it's fine. What I saw specified it in terms of C types, but maybe I misread it or was looking at the wrong thing. Where does it say that SQLSMALLINT is 16-bit? The link you shared above I thought just said it was a C short.
@stephentoub thank you for checking, I actually hadn't thought of this. Now I found this recent article, and it seems it typedef
s types such as unsigned short int
.
Do you think this'll be any problem for .NET Core? I'm mapping automatically using DllImport
. My integration tests have passed in every machine/OS I tried so far.
@tinchou any plans to do testing with oracle odbc windows/linux providers ?
Adding dependencies to the CI is not a simple process and I don't think an
Oracle Database will be added for testing. I don't manage that, so feel
free to make a request if you think it should be done and someone else
might give you a better answer.
On Thu, Apr 20, 2017, 17:38 Grahame Horner notifications@github.com wrote:
@tinchou https://github.com/tinchou any plans to do testing with oracle
odbc windows/linux providers ?—
You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub
https://github.com/dotnet/corefx/issues/13035#issuecomment-295900626,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABeg9JVkyFWUZQrmPAoDmD5VfMFKqMUMks5rx8I1gaJpZM4KhHPs
.
May it be on scope to address on .Net Core side some current shortcomings of .Net Odbc with SQL Server? Or will it have the exact same features (and limitations)? I am thinking of the troubles about support of some SQL Server 2008 types and even datetime
as illustrated here.
@fredericDelaporte this covers just porting as-is. If there are issues/limitations with the code please do open a new issue.
Closing as completed - available in myget, expect on nuget in mid November.
@danmosemsft link? I'm just looking to proof some connections to Snowflake
@maryamariyan can you please share a link to the latest odbc package on MyGet. Santi can show you if needed
Can you post an official, non-preview build on nuget?
Can you post an official, non-preview build on nuget?
@jonwagner that is the plan, right now we have a preview1 on nuget. We can't publish a non-preview package right now because we have to follow the release schedule that we have for 2.1, but once we release netcoreapp 2.1 there will be a non-preview package on nuget for this guy.
It's great to hear that it's not forgotten.
Is this part of the netcore2.1 release, or netcore2.1 plus X amount of time. As much detail as possible would be appreciated, but I understand if there are a lot of moving pieces.
We don't have the date yet but this will be part of the .NET Core 2.1 release.
@jonwagner if it helps, the ODBC library we ship then will most likely be essentially identical to the one already on nuget. If you don't mind the preview label, you can use it now.
hi, can someone help me? I keep getting this error
DllNotFoundException: Dependency unixODBC with minimum version 2.3.1 is required.
Unable to load DLL 'libodbc.so.2': The specified module or one of its dependencies could not be found.
I'm using version 2.3.5
my OS is linux debian
I can connect through the OS with isql -V DSNServer
but when I try to use this System.Data.Odbc
with New ObdcConnetion("DSN=DSNServer")
I'm stuck at this error.
Thank you.
@maryamariyan can you please help @RobsonKarls with the installation of libodbc.
Sure. I'm looking at it.
snippet from my Dockerfile:
COPY ./unixODBC-2.3.4.tar.gz /opt/
WORKDIR /opt
RUN tar xf unixODBC-2.3.4.tar.gz
WORKDIR /opt/unixODBC-2.3.4
RUN ./configure --disable-gui --enable-iconv --with-iconv-ucode-enc=UNICODE
RUN make
RUN make install
RUN echo '/usr/local/lib' >> /etc/ld.so.conf.d/x86_64-linux-gnu.conf
RUN ldconfig
RUN rm -rf /opt/unixODBC-2.3.4
Here is my Dockerfile
RUN curl http://www.unixodbc.org/unixODBC-2.3.5.tar.gz | tar xz
WORKDIR unixODBC-2.3.5
ENV CPPFLAGS="-DSIZEOF_LONG_INT=8"
RUN ./configure --prefix=/usr --libdir=/usr/lib64 --sysconfdir=/etc --enable-gui=no --enable-drivers=no --enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UTF16LE && make && make install
RUN mkdir -p /var/log/unixodbc
ENV LD_LIBRARY_PATH="/usr/lib64"
RUN echo "/usr/lib64" >> /etc/ld.so.conf.d/x86_64-linux-gnu.conf
RUN echo "/usr/lib64" >> /etc/ld.so.conf
RUN ldconfig
@RobsonKarls in the meantime please look at the notes here: https://github.com/dotnet/corefx/blob/531023d0c6129c87c256448682cd31f931c95ed2/src/System.Data.Odbc/src/DatabaseSetupInstructions.md
@RobsonKarls The dockerfiles pointed in the link above are moved to repo https://github.com/dotnet/dotnet-buildtools-prereqs-docker
the debian 8.2 dockerfile is here
I will update the notes accordingly
here where my docker image is from
FROM microsoft/aspnetcore:2.0.5 AS base
WORKDIR /app
EXPOSE 80
FROM microsoft/aspnetcore-build:2.0 AS build
RUN apt-get update
RUN apt-get -y install curl g++ make bzip2
RUN curl http://www.unixodbc.org/unixODBC-2.3.5.tar.gz | tar xz
WORKDIR unixODBC-2.3.5
ENV CPPFLAGS="-DSIZEOF_LONG_INT=8"
RUN ./configure --prefix=/usr --libdir=/usr/lib64 --sysconfdir=/etc --enable-gui=no --enable-drivers=no --enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UNICODE && make && make install
RUN mkdir -p /var/log/unixodbc
RUN echo "/usr/lib64" >> /etc/ld.so.conf.d/x86_64-linux-gnu.conf
RUN ldconfig
FROM microsoft/aspnetcore:2.0.5 AS base
WORKDIR /app
EXPOSE 80
FROM microsoft/aspnetcore-build:2.0 AS build
RUN apt-get update
RUN apt-get -y install curl g++ make bzip2
RUN curl http://www.unixodbc.org/unixODBC-2.3.4.tar.gz | tar xz
WORKDIR unixODBC-2.3.4
RUN ./configure && make && make install
RUN echo "/usr/local/lib" >> /etc/ld.so.conf.d/x86_64-linux-gnu.conf
RUN ldconfig
WORKDIR /usr/local/lib
RUN ls
RUN mkdir -p /var/log/unixodbc
RUN mkdir -p /opt/vertica-odbc-driver
RUN cd /opt/vertica-odbc-driver && curl https://my.vertica.com/client_drivers/9.0.x/9.0.1-4/vertica-client-9.0.1-4.x86_64.tar.gz | tar xz
RUN cp /opt/vertica-odbc-driver/opt/vertica/include/* /usr/include
RUN echo "[VMart]" >> /usr/local/etc/odbc.ini
RUN echo "Description = Vertica Database using ODBC Driver" >> /usr/local/etc/odbc.ini
RUN echo "Driver = VerticaDSN" >> /usr/local/etc/odbc.ini
RUN echo "Database = fc_wmart" >> /usr/local/etc/odbc.ini
RUN echo "Servername = mylocalhost" >> /usr/local/etc/odbc.ini
RUN echo "UID = dbadmin" >> /usr/local/etc/odbc.ini
RUN echo "PWD = password" >> /usr/local/etc/odbc.ini
RUN echo "[VerticaDSN]" >> /usr/local/etc/odbcinst.ini
RUN echo "Description = Vertica Database using ODBC Driver" >> /usr/local/etc/odbcinst.ini
RUN echo "Driver =/opt/vertica-odbc-driver/opt/vertica/lib64/libverticaodbc.so" >> /usr/local/etc/odbcinst.ini
RUN echo "Locale = en_US" >> /usr/local/etc/odbcinst.ini
RUN echo "[ODBC]" >> /usr/local/etc/odbcinst.ini
RUN echo "Threading = 1" >> /usr/local/etc/odbcinst.ini
COPY vertica.ini /etc/vertica.ini
ENV ODBCINI=/usr/local/etc/odbc.ini
ENV VERTICAINI=/etc/vertica.ini
RUN isql -v VMart #<----- work properly
WORKDIR /src
COPY FleetComplete.BI.ServicesAndWebApps.sln ./
COPY src/Services/Data/FleetComplete.BI.API/FleetComplete.BI.API.csproj src/Services/Data/FleetComplete.BI.API/
COPY NuGet.Config /root/.nuget/NuGet/
RUN dotnet restore
COPY . .
WORKDIR /src/src/Services/Data/FleetComplete.BI.API
RUN dotnet build -c Release -o /app
FROM build AS publish
RUN dotnet publish -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "FleetComplete.BI.API.dll"]
Well here is my entire dockerfile and I'm still getting this error, I think im gonna give up on ODBC and try another approach to connect to the DB, because I cant open a connection through System.Data.Odbc :(
Here is my complete Dockerfile. This works and is in production.
FROM microsoft/aspnetcore-build:2-jessie
RUN apt-get -y -qq update \
&& apt-get -y -qq install make gcc
COPY ./unixODBC-2.3.4.tar.gz /opt/
WORKDIR /opt
RUN tar xf unixODBC-2.3.4.tar.gz
WORKDIR /opt/unixODBC-2.3.4
RUN ./configure --disable-gui --enable-iconv --with-iconv-ucode-enc=UNICODE
RUN make
RUN make install
RUN echo '/usr/local/lib' >> /etc/ld.so.conf.d/x86_64-linux-gnu.conf
RUN ldconfig
RUN rm -rf /opt/unixODBC-2.3.4
COPY .odbc.ini.dev /usr/local/etc/odbc.ini
COPY .odbcinst.ini.dev /usr/local/etc/odbcinst.ini
COPY ./netezza /opt/netezza/
RUN /opt/netezza/unpack -f /usr/local/nz
RUN echo '/usr/local/nz/lib' >> /etc/ld.so.conf.d/x86_64-linux-gnu.conf
RUN echo '/usr/local/nz/lib64' >> /etc/ld.so.conf.d/x86_64-linux-gnu.conf
RUN ldconfig
RUN rm -rf /opt/netezza
RUN ln -s /usr/local/nz/bin64/nzodbcsql /usr/local/bin/nzodbcsql
COPY . /app
WORKDIR /app
RUN ["dotnet", "restore"]
RUN ["dotnet", "build"]
CMD ["dotnet", "run", "--server.urls", "http://*:80"]
@bigmacd can you share your connectionString format?
Thank you
Sure, although it is Netezza specific:
Driver={NetezzaSQL};server=
further to @bigmacd comments:
@RobsonKarls there are examples in System.Data.Odbc.Tests for connectionstrings format in unix and windows that you can look at: here
Did you get past your initial error?
See if you can configure this way:
sudo nano /usr/local/etc/odbcinst.ini
[ODBC Drivers]
SQLite3=Installed
[SQLite3]
Description=SQLite3
Driver=/usr/local/lib/libsqlite3odbc.so
Setup=/usr/local/lib/libsqlite3odbc.so
sudo nano /etc/odbcinst.ini
[SQLite3]
Description=SQLite3
Driver=/usr/local/lib/libsqlite3odbc.so
Setup=/usr/local/lib/libsqlite3odbc.so
Threading=4
@RobsonKarls Additional commands to help you troubleshoot:
ldconfig -p | grep sqlite
ldconfig -p | grep odbc
odbcinst -j
nano /usr/local/etc/odbcinst.ini
#see if you have this file and edit it as instructed abovels /usr/local/lib
# see if you have this directoryPlease let me know if you are still stuck.
Maryam the connection is working, I just got problem when I use the OdbcConnection.Open :(
I ran all this commands in Linux and they are working properly.
So I'm losing hope here,
Thank you all for the help :)
What problem are you getting when you call OdbcConnection.Open?
DllNotFoundException: Dependency unixODBC with minimum version 2.3.1 is required.
Unable to load DLL 'libodbc.so.2': The specified module or one of its dependencies could not be found.
I'm using version 2.3.5
@RobsonKarls What do you get when you run ldconfig -p | grep odbc
and ls /usr/local/lib
?
Probably irrelevant question: did you also configure /etc/odbcinst.ini
?
libodbc.la
libodbc.so
libodbc.so.2
libodbc.so.2.0.0
libodbccr.la
libodbccr.so
libodbccr.so.2
libodbccr.so.2.0.0
libodbcinst.la
libodbcinst.so
libodbcinst.so.2
libodbcinst.so.2.0.0
node_modules
python2.7
libodbcinst.so.2 (libc6,x86-64) => /usr/local/lib/libodbcinst.so.2
libodbcinst.so (libc6,x86-64) => /usr/local/lib/libodbcinst.so
libodbccr.so.2 (libc6,x86-64) => /usr/local/lib/libodbccr.so.2
libodbccr.so (libc6,x86-64) => /usr/local/lib/libodbccr.so
libodbc.so.2 (libc6,x86-64) => /usr/local/lib/libodbc.so.2
libodbc.so (libc6,x86-64) => /usr/local/lib/libodbc.so
Probably irrelevant question: did you also configure /etc/odbcinst.ini ?
I only obdcinst.ini in /usr/local/etc/
I believe here's what you have configured in /usr/local/etc/odbcinst.ini
[VerticaDSN]
Description = Vertica Database using ODBC Driver
Driver =/opt/vertica-odbc-driver/opt/vertica/lib64/libverticaodbc.so
Locale = en_US
[ODBC]
Threading = 1
Can you please try adding this to /usr/local/etc/odbcinst.ini
:
[ODBC Drivers]
VerticaDSN=Installed
Then to configure /etc/odbcinst.ini
as below:
[VerticaDSN]
Description=VerticaDSN
Driver=/opt/vertica-odbc-driver/opt/vertica/lib64/libverticaodbc.so
Setup=/opt/vertica-odbc-driver/opt/vertica/lib64/libverticaodbc.so
Threading=...
Let me know if this helps
Maryam this is my odbcinst -j
RUN odbcinst -j
---> Running in 43ac8c3bc11b
unixODBC 2.3.4
DRIVERS............: /usr/local/etc/odbcinst.ini
SYSTEM DATA SOURCES: /usr/local/etc/odbc.ini
FILE DATA SOURCES..: /usr/local/etc/ODBCDataSources
USER DATA SOURCES..: /usr/local/etc/odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8
RE: https://codefarm.me/2017/11/16/cloudera-odbc-driver-for-impala-on-debian-linux-with-.net-core-2.0/
To build a docker image with unixODBC driver manger, you can refer to https://github.com/qqbuby/aspnet-docker/blob/dev/runtime/2.0/clouderaimpalaodbc/Dockerfile
FROM microsoft/aspnetcore:2.0
LABEL maintainer="ROY <[email protected]>"
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
unixodbc \
krb5-user \
&& rm -rf /var/lib/apt/lists/*
RUN curl -sSL https://downloads.cloudera.com/connectors/impala_odbc_2.5.40.1025/Debian/clouderaimpalaodbc_2.5.40.1025-2_amd64.deb \
-o /tmp/clouderaimpalaodbc_2.5.40.1025-2_amd64.deb \
&& dpkg -i /tmp/clouderaimpalaodbc_2.5.40.1025-2_amd64.deb \
&& rm /tmp/clouderaimpalaodbc_2.5.40.1025-2_amd64.deb
COPY ./files/odbcinst.ini /etc/odbcinst.ini
COPY ./files/cloudera.impalaodbc.ini /opt/cloudera/impalaodbc/lib/64/cloudera.impalaodbc.ini
Could it be a LD_LIBRARY_PATH problem? We do pass environment variables into the docker container when we spin it up:
ASPNETCORE_URLS=http://*:80
LANG=en_us
ASPNETCORE_ENVIRONMENT=production
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/nz/lib:/usr/local/nz/lib64
ODBCINI=/root/odbc.ini
NZ_ODBC_INI_PATH=/root
Hey Guys I got it working, Thanks @maryamariyan @bigmacd and @qqbuby ,
@qqbuby this article that u shared is really a piece of work, it made so clear how ODBC works.
@RobsonKarls great, could you share what the problem was?
The problem was in my Dockerfile, I was installing the driver in the wrong image, my Dockerfile is posted in above posts.
I hope will get some help.
I am trying to connect to RedShift DB Using AWS Lambda from .Net Core 2.0 App.
I have included System.Data.Odbc in app
string connString = "Driver={Amazon Redshift (x64)};" +
String.Format("Server={0};Database={1};" +
"UID={2};PWD={3};Port={4};SSL=true;Sslmode=Require",
RedShiftServer, RedShiftDBName, RedShiftUsername,RedShiftPassword, RedShiftPort);
OdbcConnection conn = new OdbcConnection(connString);
conn.Open();
But I am not able to connect to RedShift DB (Unable to Open the connection) after deployment to Lambda Function.
I am getting Bellow Error.
"Dependency unixODBC with minimum version 2.3.1 is required. Unable to load DLL 'libodbc.so.2': The specified module or one of its dependencies could not be found.
how to solve it ?
ODBC on unix requires the installation of additional database-specific drivers on the server. I don’t know if you can do that on a lambda server.
If you are connecting to redshift, you might be able to use the postgres driver instead.
On Jun 22, 2018, at 8:55 AM, Digambar Malla <[email protected]notifications@github.com> wrote:
I hope will get some help.
I am trying to connect to RedShift DB Using AWS Lambda from .Net Core 2.0 App.
I am using Redshift 64 bit Driver getting it from here
https://docs.aws.amazon.com/redshift/latest/mgmt/install-odbc-driver-windows.html
string connString = "Driver={Amazon Redshift (x64)};" + String.Format("Server={0};Database={1};" + "UID={2};PWD={3};Port={4};SSL=true;Sslmode=Require", RedShiftServer, RedShiftDBName, RedShiftUsername,RedShiftPassword, RedShiftPort); OdbcConnection conn = new OdbcConnection(connString); conn.Open();
But I am not able to connect to RedShift DB (Unable to Open the connection) after deployment to Lambda Function.
I am getting Bellow Error.
"Dependency unixODBC with minimum version 2.3.1 is required. Unable to load DLL 'libodbc.so.2': The specified module or one of its dependencies could not be found.
how to solve it ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/dotnet/corefx/issues/13035#issuecomment-399432787, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABk3dCrx0UtRoAsFb82zy8j4oHiDlkkBks5t_OkmgaJpZM4KhHPs.
The Redshift drivers you mention are for windows. The use of these drivers and the error message are incongruent.
Regardlesss, as @jonwagner mentioned, you will need unixODBC in your environment and I don't know if Lambda has a runtime that supports that.
The proper architecture would be to have your lambda function make an api call that does the database heavy lifting.
Most helpful comment
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Data.Odbc.
Thanks @safern