Consuming Data from Databricks to Datasphere

Prerequisites:

Access: Access to both SAP Datasphere and Databricks.Credentials: Have Databricks authentication credentials ready.

Steps:

Login to Datasphere: In the home screen of the datasphere, Expand the navigation bar on the left and click “Configuration”.

Database Access Click database access and then click Database Analysis Users

Then click “Create” to create a database access user

Enter the Database Analysis username suffixEnable Space Schema AccessSet Database Analysis user expires in Never option.

Once created it will generate a Hostname, Port Number and Password and make sure that Enable Space schema access has been enabled, so by this you can be able to access Datasphere’s all spaces and its schemas

Login to DatabricksGo to Workspace, create Cluster

In the cluster, paste this code to connect to datasphere and Databricks

Python

import org.apache.spark.sql.SparkSession

# Initialize Spark Session

spark = SparkSession.builder()

    .appName(“SAP HANA Connection”)

    .config(“spark.jars”, “/usr/sap/hdbclient/ngdbc-2.4.62.jar”)

    .getOrCreate()

# JDBC URL

jdbcUrl = “jdbc:sap://zeus.hana.prod.eu-central-1.whatever.cloud.sap:443/?encrypt=true&validateCertificate=false&currentschema=MYSCHEMA”

# Connection Properties

connectionProperties = {

  “user” : “MYUSER”,

  “password” : “mypassword”,

  “driver” : “com.sap.db.jdbc.Driver”

}

# Query to Select all the Data from MYTABLE

val query = “SELECT * FROM MYTABLE”

# Read Data from JDBC Connection into DataFrame

val df = spark.read.jdbc(jdbcUrl, query, connectionProperties)

Once code is executed successfully, the connection is established between the Datasphere and Databricks. 

  Read More Technology Blogs by Members articles 

#abap