LivyScalaClient

Instance Constructors

new LivyScalaClient(livyJavaClient: LivyClient)

Creates a Scala client.
Creates a Scala client.
livyJavaClient
the Java client of Livy.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def addFile(uri: URI): Future[_]

Adds a file to the running remote context.
Adds a file to the running remote context.
Note that the URL should be reachable by the Spark driver process. If running the driver in cluster mode, it may reside on a different host, meaning "file:" URLs have to exist on that node (and not on the client machine).
If the provided URI has no scheme, it's considered to be relative to the default file system configured in the Livy server.
uri
The location of the file.
returns
A future that can be used to monitor the operation.
def addJar(uri: URI): Future[_]

Adds a jar file to the running remote context.
Adds a jar file to the running remote context.
Note that the URL should be reachable by the Spark driver process. If running the driver in cluster mode, it may reside on a different host, meaning "file:" URLs have to exist on that node (and not on the client machine).
If the provided URI has no scheme, it's considered to be relative to the default file system configured in the Livy server.
uri
The location of the jar file.
returns
A future that can be used to monitor the operation.
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def run[T](fn: (ScalaJobContext) ⇒ T): Future[T]

Asks the remote context to run a job immediately.
Asks the remote context to run a job immediately.
Normally, the remote context will queue jobs and execute them based on how many worker threads have been configured. This method will run the submitted job in the same thread processing the RPC message, so that queueing does not apply.
It's recommended that this method only be used to run code that finishes quickly. This avoids interfering with the normal operation of the context.
fn
The job to be executed. It is a function that takes in a ScalaJobContext and returns the result of the execution of the job with that context.
returns
A handle that can be used to monitor the job.
def stop(shutdownContext: Boolean): Unit

Stops the remote context.
Stops the remote context.
Any pending jobs will be cancelled, and the remote context will be torn down.
shutdownContext
Whether to shutdown the underlying Spark context. If false, the context will keep running and it's still possible to send commands to it, if the backend being used supports it.
def submit[T](fn: (ScalaJobContext) ⇒ T): ScalaJobHandle[T]

Submits a job for asynchronous execution.
Submits a job for asynchronous execution.
fn
The job to be executed. It is a function that takes in a ScalaJobContext and returns the result of the execution of the job with that context.
returns
A handle that can be used to monitor the job.
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
def uploadFile(file: File): Future[_]

Upload a file to be passed to the Spark application.
Upload a file to be passed to the Spark application.
file
The local file to be uploaded.
returns
A future that can be used to monitor this operation.
def uploadJar(jar: File): Future[_]

Upload a jar to be added to the Spark application classpath.
Upload a jar to be added to the Spark application classpath.
jar
The local file to be uploaded.
returns
A future that can be used to monitor this operation.
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package scalaapi

class LivyScalaClient extends AnyRef

Instance Constructors

new LivyScalaClient(livyJavaClient: LivyClient)

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

def addFile(uri: URI): Future[_]

def addJar(uri: URI): Future[_]

final def asInstanceOf[T0]: T0

def clone(): AnyRef

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def run[T](fn: (ScalaJobContext) ⇒ T): Future[T]

def stop(shutdownContext: Boolean): Unit

def submit[T](fn: (ScalaJobContext) ⇒ T): ScalaJobHandle[T]

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

def uploadFile(file: File): Future[_]

def uploadJar(jar: File): Future[_]

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from AnyRef

Inherited from Any

Ungrouped