Knowledge Repository
Labels
Docker
(1)
Java
(1)
Kafka
(2)
Kubernetes
(3)
Python
(1)
Spark
(15)
Streams
(2)
Monday, February 25, 2019
File encoding option for spark YARN container
1)
Configuration
to pass file ecoding to YARN containers,
--conf 'spark.executor.extraJavaOptions=-Dfile.encoding=UTF-8'
2) File encoding option to read the file using spark local mode
option("encoding", "UTF-8") .csv(inputPath)
No comments:
Post a Comment
Newer Post
Older Post
Home
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment