I want to enforce multi-tenancy in my spark cluster - my tenants would be allocated a certain amount of memory and CPU, and i want to make sure that when they submit their applications, they do not consume more than their quota.
I am also curious to know about the ownership of files and folders created in local file system/ NFS when a spark job is submitted in cluster mode?
How do I configure and use workpools with DSE Analytics?
How do i configure and use workpools with DSE Analytics ?
workpool settings can be configured in dse.yaml. You can also control access access to workpools for DSE users by granting permissions as described in link.
Who owns the files created in local file-system / NFS when a spark job is submitted in cluster mode?
in cluster mode, the OS user, who executes the driver or the executor code, owns the files and folders being created by the spark code - it will be cassandra user or slots users as per your configuration.
You can watch a demo on these two topics in this video.
6 People are following this question.
