当前位置:首页 > articles > 正文内容

使用databrics dbutils api的问题

Luz4周前 (07-20)articles136

problems with use databrics dbutils-api

  • sutugin 问题:
    • 我们如何尝试使用它?
    • add dependency "com.databricks" %% "dbutils-api" % "0.0.1" to build.sbt
    • 用代码编写:
    • import com.databricks.dbutils_v1.DBUtilsHolder.dbutils class Job {     // business logic ..     val jobResult: String = ???     dbutils.notebook.exit(jobResult)}
    • 当我们将代码打包到jar中时,我们得到:
    • [error] missing or invalid dependency detected while loading class file 'NotebookUtils.class'.[error] Could not access term common in package com.databricks.backend,[error] because it (or its dependencies) are missing. Check your build definition for[error] missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)[error] A full rebuild may help if 'NotebookUtils.class' was compiled against an incompatible version of com.databricks.backend.[error] one error found[error] (compile:compileIncremental) Compilation failed
  • 回答:
    • morsik - vote: 1
      • The reason is even latest ver of
        dbutils-api
        for now (which is 0.0.3) relies on
        com.databricks.backend.common
        (which is neither not part of dbutils-api nor any other public databricks libs).
      • trait NotebookUtils ... {...  def getContext() : com.databricks.backend.common.rpc.CommandContext  def setContext(ctx : com.databricks.backend.common.rpc.CommandContext) : scala.Unit}
      • 同时:
      • jar tvf dbutils-api_2.11-0.0.3.jar | grep backend   439 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/Default.class  1337 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/FileInfo$$anonfun$hashCode$1.class  1265 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/FileInfo$$anonfun$hashCode$2.class  4109 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/FileInfo.class  1256 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/GrantInfo.class  1480 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/MountInfo.class   452 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/PackageGroup.class     0 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/     0 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/     0 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/
      • The most probable reason is
        com.databricks.backend.common
        marked with provided scope in dbutils definition file (but I can't be 100% sure because this lib is not available in open source).I would rather be intended to treat this as a bug in Databricks' libs.

相关文章

使用nodejs和mongodb进行前端搜索

Frontend search with nodejs and mongodbCaleb 问题: 我今天刚开始学习mongodb,我有一段代码可以用来查询我的数据库和搜索地址:“Park Lane 38”。这是代码,这是结果:...

单元测试名称重要吗?

Are unit-test names important?Xian 问题: 如果单元测试名称随着时间的推移而变得过时,如果您认为测试本身是最重要的事情,那么选择明智的测试名称是重要的吗?ie[Test]public void Sh...

应该如何使用kotlin将findViewById传递数据从活动页面写入片段页面

How should write the findViewById pass data to Fragment page from Activity page using kotlinMIN LEE 问题: //这是我的活动页面覆盖...

脚本1 linux中的crear usuarios和demás

Script 1 de crear usuarios y demás en linuxMaría Isabel De la Osa Rocha 问题: Cómo se haría el script del usuario porq...

C++:泛型&gt>如何为一个通用函数定义多个模板解题,该函数充当静态实例创建者

C++:Generics >> How to define several template deklarations for a generic function, which serves as an static inst...

三js如何向FBX模型添加多个没有蒙皮的Mixamo动画?

Three.js how to add multiple Mixamo animations without skin to an FBX model?Anye 问题: I'm trying to create a game whe...

发表评论

访客

◎欢迎参与讨论,请在这里发表您的看法和观点。