Azure-docs: Can this be used along with Databricks Connect?

[empz]

It seems there are two ways to bring the DbUtils class. This one and using Databricks Connect. I'd like to use Databricks Connect when running locally on my IDE, but use this class when deploying the jar to the cluster. I've been trying to get this working with no luck.

Is this scenario supported? If so, could you please share an example on how to achieve it?

Thanks

---

Document details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 1c1af686-fd41-47e8-a6cb-70197f33bc67
• Version Independent ID: 6cc4bc4f-850d-9061-d66d-68620b4baa29
• Content: Databricks Utilities - Azure Databricks
• Content Source: databricks/dev-tools/databricks-utils.md
• Service: azure-databricks
• GitHub Login: @mamccrea
• Microsoft Alias: mamccrea

[himanshusinha-msft]

Thanks for the feedback and bringing this to our notice . At this time we are reviewing the feedback and will update the document as appropriate .

[himanshusinha-msft]

We have assigned the issue to the content author . They will evaluate & update the document as appropriate

[mamccrea]

Hi @eparizzi - Most of DButils aren't supported for Databricks Connect. The only parts that do work are fs and secrets. Please see https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect#limitations

[empz]

@mamccrea I'm aware of the limitations. Databricks Connect works just fine locally. I only need basic dbutils.fs usage. The problem is that the same code that works locally doesn't work when deploying a jar to a Databricks cluster.
The snippet shown here val dbutils = com.databricks.service.DBUtils simply doesn't work in Databricks. Job fails with error saying that it couldn't find that class.

There's a clear example on how to make this work in Python:

I just need the Scala version of it...

[mamccrea]

Hi @eparizzi - Thank you very much for your patience and for explaining the situation. We have a ticket open with the engineer to add a Scala example. Is this issue a duplicate of https://github.com/MicrosoftDocs/azure-docs/issues/43493?

[empz]

Hi @eparizzi - Thank you very much for your patience and for explaining the situation. We have a ticket open with the engineer to add a Scala example. Is this issue a duplicate of #43493?

Thanks. Yes, that's a duplicate.

[mamccrea]

I am tracking this issue with the engineering team and will respond here when the Scala example is available. Thanks again for your feedback. Please let me know if you have additional questions.

please-close