Skip to content

Commit e6e7993

Browse files
ChaffelsonChaffelson
committed
Initial Release
Signed-off-by: Daniel Chaffelson <[email protected]>
1 parent c8dd1dc commit e6e7993

28 files changed

+2101
-0
lines changed

‎AUTHORS.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
============
2+
Contributors
3+
============
4+
5+
* Daniel Chaffelson <[email protected]>

‎CHANGELOG.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
=========
2+
Changelog
3+
=========
4+
5+
Version 0.1.0
6+
=============
7+
8+
- Initial Release

‎README.rst‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
====
2+
cdpy
3+
====
4+
5+
6+
A Simple Pythonic Client wrapper for Cloudera CDP CLI, designed for use with the Ansible framework
7+
8+
Installation
9+
============
10+
11+
To install directly from latest commits, in python requirements.txt ::
12+
13+
git+git://github.com/cloudera-labs/cdpy@main#egg=cdpy
14+
15+
For General usage, installed from cmdline ::
16+
17+
pip install cdpy
18+
19+
Usage
20+
=====
21+
Note that this readme covers usage of this wrapper library only, for details of CDPCLI and underlying commands please see the CDPCLI documentation.
22+
23+
Simple Usage
24+
------------
25+
26+
Basic function call with defaults where credentials are already available in user profile ::
27+
28+
from cdpy.cdpy import Cdpy
29+
Cdpy().iam.get_user()
30+
> {'userId': 'de740ef9-b40e-4497-8b3b-c137481c7633', 'crn': 'crn:altus:iam:us-west-1:558bc1d2-8867-4357-8524-311d51259233:user:de740ef9-b40e-4497-8b3b-c137481c7633', 'email': '[email protected]', 'firstName': 'Daniel', 'lastName': 'Chaffelson', 'creationDate': datetime.datetime(2019, 11, 4, 11, 54, 27, 581000, tzinfo=tzutc()), 'accountAdmin': False, 'identityProviderCrn': 'crn:altus:iam:us-west-1:558bc1d2-8867-4357-8524-311d51259233:samlProvider:cloudera-okta-production/a0afd6e3-ffc1-48bd-953a-60003d82f8ae', 'lastInteractiveLogin': datetime.datetime(2020, 12, 1, 11, 32, 38, 901000, tzinfo=tzutc()), 'workloadUsername': 'dchaffey'}
31+
32+
Typical function calls in CDP CLI are shadowed within the relevant Classes when wrapped by cdpy, or may be called directly using the call function shown further down.
33+
34+
Basic function call with target that does not exist (404) ::
35+
36+
from cdpy.cdpy import Cdpy
37+
Cdpy().iam.get_user('fakeusername')
38+
>> UserWarning:CDP User could not be retrieved
39+
> None
40+
41+
This Class function has been wrapped to automatically handle the NOT_FOUND error generated when requesting an unknown user, note the use of self.sdk.call within the Class ::
42+
43+
def get_user(self, name=None):
44+
return self.sdk.call(
45+
# Describe base function calls
46+
svc='iam', # Name of the client service
47+
func='get_user', # Name of the Function within the service to call
48+
ret_field='user', # Optional child field to return, often CDP CLI responses are wrapped like this
49+
squelch=[ # List of below Client Error Handlers
50+
# Describe any Client Error responses using the provided Squelch class
51+
Squelch(
52+
field='error_code', # CdpError Field to test
53+
value='NOT_FOUND', # String to check for in Field
54+
warning='CDP User could not be retrieved', # Warning to throw if encountered
55+
default=None # Value to return instead of Error
56+
)
57+
],
58+
# Include any keyword args that may be used in the function call, None/'' args will be ignored
59+
userId=name # As name is None by default, it will be ignored unless provided
60+
)
61+
62+
Basic function call with invalid value ::
63+
64+
from cdpy.cdpy import Cdpy
65+
Cdpy().iam.get_user('')
66+
>> UserWarning:Removing empty string arg userId from submission
67+
> {'userId': 'de740ef9-b40e-4497-8b3b-c137481c7633', 'crn': 'crn:altus:iam:us-west-1:558bc1d2-8867-4357-8524-311d51259233:user:de740ef9-b40e-4497-8b3b-c137481c7633', 'email': '[email protected]', 'firstName': 'Daniel', 'lastName': 'Chaffelson', 'creationDate': datetime.datetime(2019, 11, 4, 11, 54, 27, 581000, tzinfo=tzutc()), 'accountAdmin': False, 'identityProviderCrn': 'crn:altus:iam:us-west-1:558bc1d2-8867-4357-8524-311d51259233:samlProvider:cloudera-okta-production/a0afd6e3-ffc1-48bd-953a-60003d82f8ae', 'lastInteractiveLogin': datetime.datetime(2020, 12, 1, 11, 32, 38, 901000, tzinfo=tzutc()), 'workloadUsername': 'dchaffey'}
68+
69+
Here the invalid Parameter for for typical Ansible parameter `name` is mapped to `userId` within CDPCLI and is scrubbed from the submission ( may not submit zero length strings), a user warning is issued.
70+
The command is run by default with that Parameter removed, this triggering the 'list all' logic of the underlying call.
71+
72+
This behavior may be bypassed using the `scrub_inputs` switch and thus raise a native Python Error ::
73+
74+
Cdpy(scrub_inputs=False).iam.get_user('')
75+
Traceback (most recent call last):
76+
...
77+
78+
More Complex Usage
79+
------------------
80+
81+
Call Wrapper execution directly for arbitrary CDP Service and Function with arbitrary keyword param. This `call` function is the same that is exposed in the more abstract Classes in earlier examples, this is the more direct usage method allowing developers to work at varying levels within the layered abstractions with relative ease ::
82+
83+
from cdpy.common import CdpcliWrapper
84+
85+
CdpcliWrapper().call(svc='iam', func='set_workload_password_policy', maxPasswordLifetimeDays=lifetime)
86+
87+
Define function to call wrapped method with prebuild payload and use custom error handling logic.
88+
Note that the `env_config` arguments are unpacked and passed to the CDP API directly, allowing developers to work with additional parameters without waiting for the higher level abstractions to support them
89+
This example also demonstrates bypassing the Squelch error handler to implement custom error handling logic, and the ability to revert to the provided `throw_error` capabilities (which are superable) ::
90+
91+
from cdpy.common import CdpcliWrapper
92+
from cdpy.common import CdpError
93+
94+
wrap = CdpcliWrapper()
95+
env_config = dict(valueOne=value, valueTwo=value)
96+
97+
resp = wrap.call(
98+
svc='environments', func='create_aws_environment', ret_field='environment', ret_error=True,
99+
**env_config
100+
)
101+
if isinstance(resp, CdpError):
102+
if resp.error_code == 'INVALID_ARGUMENT':
103+
if 'constraintViolations' not in resp.violations:
104+
resp.update(message="Received violation warning:\n%s" % self.sdk.dumps(resp.violations))
105+
self.sdk.throw_warning(resp)
106+
self.sdk.throw_error(resp)
107+
return resp
108+
109+
Declare custom error handling function and instantiate with it. This abstraction is specifically to allow developers to replace native Python error handling with framework specific handling, such as the typical Ansible module `fail_json` seen here ::
110+
111+
from cdpy.common import CdpError
112+
113+
class CdpModule(object)
114+
def _cdp_module_throw_error(self, error: 'CdpError'):
115+
"""Wraps throwing Errors when used as Ansible module"""
116+
self.module.fail_json(msg=str(error.__dict__))
117+
118+
self.sdk = Cdpy(error_handler=self._cdp_module_throw_error)
119+
120+
Ideally for extensive development you would make use of the metaclass Cdpy, this is currently used as the basis for the Cloudera CDP Public Cloud Ansible Collection ::
121+
122+
from cdpy.cdpy import Cdpy
123+
client = Cdpy(debug=self.debug, tls_verify=self.tls, strict_errors=self.strict, error_handler=self._cdp_module_throw_error, warning_handler=self._cdp_module_throw_warning)
124+
client.sdk.call(...)
125+
client.sdk.iam.(...)
126+
client.sdk.TERMINATION_STATES
127+
etc.
128+
129+
130+
Development
131+
=====================
132+
133+
Contributing
134+
------------
135+
136+
Please create a feature branch from the current development Branch then submit a PR referencing an Issue for discussion.
137+
138+
Please note that we require signed commits inline with Developer Certificate of Origin best-practices for Open Source Collaboration.
139+
140+
PyScaffold Note
141+
===============
142+
143+
This project has been set up using PyScaffold 3.2.3. For details and usage
144+
information on PyScaffold see https://pyscaffold.org/.

‎docs/Makefile‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
# Makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line.
5+
SPHINXOPTS =
6+
SPHINXBUILD = sphinx-build
7+
PAPER =
8+
BUILDDIR = ../build/sphinx/
9+
AUTODOCDIR = api
10+
AUTODOCBUILD = sphinx-apidoc
11+
PROJECT = cdpy
12+
MODULEDIR = ../src/cdpy
13+
14+
# User-friendly check for sphinx-build
15+
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $?), 1)
16+
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
17+
endif
18+
19+
# Internal variables.
20+
PAPEROPT_a4 = -D latex_paper_size=a4
21+
PAPEROPT_letter = -D latex_paper_size=letter
22+
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
23+
# the i18n builder cannot share the environment and doctrees with the others
24+
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
25+
26+
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext doc-requirements
27+
28+
help:
29+
@echo "Please use \`make <target>' where <target> is one of"
30+
@echo " html to make standalone HTML files"
31+
@echo " dirhtml to make HTML files named index.html in directories"
32+
@echo " singlehtml to make a single large HTML file"
33+
@echo " pickle to make pickle files"
34+
@echo " json to make JSON files"
35+
@echo " htmlhelp to make HTML files and a HTML help project"
36+
@echo " qthelp to make HTML files and a qthelp project"
37+
@echo " devhelp to make HTML files and a Devhelp project"
38+
@echo " epub to make an epub"
39+
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
40+
@echo " latexpdf to make LaTeX files and run them through pdflatex"
41+
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
42+
@echo " text to make text files"
43+
@echo " man to make manual pages"
44+
@echo " texinfo to make Texinfo files"
45+
@echo " info to make Texinfo files and run them through makeinfo"
46+
@echo " gettext to make PO message catalogs"
47+
@echo " changes to make an overview of all changed/added/deprecated items"
48+
@echo " xml to make Docutils-native XML files"
49+
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
50+
@echo " linkcheck to check all external links for integrity"
51+
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
52+
53+
clean:
54+
rm -rf $(BUILDDIR)/* $(AUTODOCDIR)
55+
56+
$(AUTODOCDIR): $(MODULEDIR)
57+
mkdir -p $@
58+
$(AUTODOCBUILD) -f -o $@ $^
59+
60+
doc-requirements: $(AUTODOCDIR)
61+
62+
html: doc-requirements
63+
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
64+
@echo
65+
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
66+
67+
dirhtml: doc-requirements
68+
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
69+
@echo
70+
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
71+
72+
singlehtml: doc-requirements
73+
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
74+
@echo
75+
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
76+
77+
pickle: doc-requirements
78+
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
79+
@echo
80+
@echo "Build finished; now you can process the pickle files."
81+
82+
json: doc-requirements
83+
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
84+
@echo
85+
@echo "Build finished; now you can process the JSON files."
86+
87+
htmlhelp: doc-requirements
88+
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
89+
@echo
90+
@echo "Build finished; now you can run HTML Help Workshop with the" \
91+
".hhp project file in $(BUILDDIR)/htmlhelp."
92+
93+
qthelp: doc-requirements
94+
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
95+
@echo
96+
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
97+
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
98+
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/$(PROJECT).qhcp"
99+
@echo "To view the help file:"
100+
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/$(PROJECT).qhc"
101+
102+
devhelp: doc-requirements
103+
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
104+
@echo
105+
@echo "Build finished."
106+
@echo "To view the help file:"
107+
@echo "# mkdir -p $HOME/.local/share/devhelp/$(PROJECT)"
108+
@echo "# ln -s $(BUILDDIR)/devhelp $HOME/.local/share/devhelp/$(PROJEC)"
109+
@echo "# devhelp"
110+
111+
epub: doc-requirements
112+
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
113+
@echo
114+
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
115+
116+
patch-latex:
117+
find _build/latex -iname "*.tex" | xargs -- \
118+
sed -i'' 's~includegraphics{~includegraphics\[keepaspectratio,max size={\\textwidth}{\\textheight}\]{~g'
119+
120+
latex: doc-requirements
121+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
122+
$(MAKE) patch-latex
123+
@echo
124+
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
125+
@echo "Run \`make' in that directory to run these through (pdf)latex" \
126+
"(use \`make latexpdf' here to do that automatically)."
127+
128+
latexpdf: doc-requirements
129+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
130+
$(MAKE) patch-latex
131+
@echo "Running LaTeX files through pdflatex..."
132+
$(MAKE) -C $(BUILDDIR)/latex all-pdf
133+
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
134+
135+
latexpdfja: doc-requirements
136+
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
137+
@echo "Running LaTeX files through platex and dvipdfmx..."
138+
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
139+
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
140+
141+
text: doc-requirements
142+
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
143+
@echo
144+
@echo "Build finished. The text files are in $(BUILDDIR)/text."
145+
146+
man: doc-requirements
147+
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
148+
@echo
149+
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
150+
151+
texinfo: doc-requirements
152+
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
153+
@echo
154+
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
155+
@echo "Run \`make' in that directory to run these through makeinfo" \
156+
"(use \`make info' here to do that automatically)."
157+
158+
info: doc-requirements
159+
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
160+
@echo "Running Texinfo files through makeinfo..."
161+
make -C $(BUILDDIR)/texinfo info
162+
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
163+
164+
gettext: doc-requirements
165+
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
166+
@echo
167+
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
168+
169+
changes: doc-requirements
170+
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
171+
@echo
172+
@echo "The overview file is in $(BUILDDIR)/changes."
173+
174+
linkcheck: doc-requirements
175+
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
176+
@echo
177+
@echo "Link check complete; look for any errors in the above output " \
178+
"or in $(BUILDDIR)/linkcheck/output.txt."
179+
180+
doctest: doc-requirements
181+
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
182+
@echo "Testing of doctests in the sources finished, look at the " \
183+
"results in $(BUILDDIR)/doctest/output.txt."
184+
185+
xml: doc-requirements
186+
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
187+
@echo
188+
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
189+
190+
pseudoxml: doc-requirements
191+
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
192+
@echo
193+
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

‎docs/_static/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Empty directory

‎docs/authors.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.. _authors:
2+
.. include:: ../AUTHORS.rst

‎docs/changelog.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.. _changes:
2+
.. include:: ../CHANGELOG.rst

0 commit comments

Comments
 (0)