Error while copying from R to Spark


#1

Hi,
getting the permission error

“java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: C:/Users/rumsinha/AppData/Local/rstudio/spark/Cache/spark-1.6.2-bin-hadoop2.6/tmp/hive on HDFS should be writable. Current permissions are: rw-rw-rw-”

when running
copy_to(sc, iris, overwrite = TRUE)

I have ran

library(“sparklyr”)

spark_install(version = “1.6.2”)

sc <- spark_connect(master = “local”, spark_home = spark_home_dir(version=“1.6.2”))

Any help appreciated


#2

Change the permissions for this file -->C:/Users/rumsinha/AppData/Local/rstudio/spark/Cache/spark-1.6.2-bin-hadoop2.6/tmp/hive

[chmod u+x C:/…/hive]

Try the above command. Hope it works.