Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Snakebite for HDFS commands #19

Open
DonDebonair opened this issue Jan 13, 2015 · 1 comment
Open

Use Snakebite for HDFS commands #19

DonDebonair opened this issue Jan 13, 2015 · 1 comment

Comments

@DonDebonair
Copy link
Member

This is so much better than doing it over SSH.

@DonDebonair
Copy link
Member Author

Apparently Snakebite doesn't support commands that require communication with the datanode(s) yet. It seems that there is no ETA for supporting this. A better option seems to be, to use pywebhdfs for working with HDFS. pywebhdfs isn't feature-complete either, but has support for writing to HDFS, so copyFromLocal-like functionality should be no problem. Another advantage is that webhdfs is a simpler protocol than the protobuf communication that happens in Snakebite, therefor it should be much easier to contribute to pywebhdfs for any missing features (chown/chmod come to mind).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant