Skip to content

Error in writing a csv to sftp path using spark "Exception in thread "main" 2: No such file" #89

@alaukikharsh

Description

@alaukikharsh

getting error no such file while writing a df to sftp path using spark. how can i write a dummy df to sftp path instead? what am i doing wrong here?

val DF = spark.sql("""select 1,2,3 as a""")
DF.show(10)
DF.coalesce(1).write.
      format("com.springml.spark.sftp").
      option("host", "correct 1").
      option("username", "correct 1").
      option("pem", "correct 1").
      option("fileType", "csv").
      option("codec", "bzip2").
      save(/PRODUCTION/SAMPLE/a.csv)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions