Skip to content

Conversation

@alex-bo
Copy link

@alex-bo alex-bo commented Sep 29, 2020

Glob module for HDFS client. The source code was ported from Python 3.4 and tested.

@alex-bo
Copy link
Author

alex-bo commented Sep 29, 2020

I resolved all 2.7 compatibility issues. Would appreciate if somebody can point me at the right direction to deal with "Failed to establish a new connection: [Errno 111] Connection refused',))" failures. Looks like it has nothing to do with the source code but rather an environment problem.

Copy link
Owner

@mtth mtth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the slow response @alex-bo. Thank you for the PR, I think this feature would be useful to have. The connection error might be related to #168.

hdfs/glob.py Outdated
import fnmatch
import re

from hdfs import Client
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from .client import Client



def glob(client, hdfs_path):
"""Return a list of paths matching a pathname pattern.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good doc!

hdfs/glob.py Outdated
Comment on lines 20 to 26
Sample usages:

.. code-block:: python

glob(client, './foo/bar/*')
glob(client, './foo/bar/file[0-9].txt')
glob(client, './foo/bar/file?.txt')
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unindent sample usage? Same for iglob below.

glob(client, './foo/bar/file?.txt')

"""
return list(iglob(client, hdfs_path))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

glob provides little value over iglob: it saves just the list call. What do you think about only exposing iglob, possibly renaming it to glob?

hdfs/glob.py Outdated
yield posixpath.join(dirname, name)


def glob1(client, dirname, pattern):
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefix this (and the other functions below) with an underscore?

@alex-bo
Copy link
Author

alex-bo commented Dec 5, 2021

Hey @mtth any opinion on my latest updates and comments? Are you good with the changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants