Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions docs/advanced/additional-requests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ a generic HTTP Request: :class:`~.HttpRequest`. Here's an example:
"Page": page_num,
"ProductID": product_id,
}
).encode("utf-8"),
),
)

print(request.url) # https://www.api.example.com/product-pagination/
Expand All @@ -73,9 +73,12 @@ There are a few things to take note here:

* ``body`` is represented by the :class:`~.HttpRequestBody` class which is
simply a subclass of the ``bytes`` class. Using the ``body`` param of
:class:`~.HttpRequest` needs to have an input argument in ``bytes``. In our
code example, we've converted it from ``str`` to ``bytes`` using the ``encode()``
string method.
:class:`~.HttpRequest` supports converting it from ``str`` to ``bytes``
using **utf-8** as the default encoding.

* Note that the :meth:`~.HttpRequestBody.from_anystr` alternative constructor
for :class:`~.HttpRequestBody` can also be used to support instantiation of
``str`` to ``bytes`` using any type of encoding (`default is` **utf-8**).

Most of the time though, what you'll be defining would be ``GET`` requests. Thus,
it's perfectly fine to define them as:
Expand Down Expand Up @@ -211,7 +214,7 @@ Thus, additional requests inside the Page Object are typically needed for it:
"Page": 2,
"ProductID": item["product_id"],
}
).encode("utf-8"),
),
)
item["related_product_ids"].extend(self.parse_related_product_ids(response))
return item
Expand Down Expand Up @@ -293,7 +296,7 @@ list of :class:`~.HttpRequest` to be executed in batch using the
"Page": page_num,
"ProductID": item["product_id"],
}
).encode("utf-8"),
),
)

@staticmethod
Expand Down
10 changes: 5 additions & 5 deletions tests/test_requests.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ def test_http_request_body_bytes_api():
assert b"ent" in http_body


def test_http_request_body_str_api():
with pytest.raises(TypeError):
HttpRequestBody("string content")
def test_http_request_body_from_anystr():
http_body = HttpRequestBody.from_anystr("originally string")
assert http_body == b"originally string"


def test_http_request_bytes_body():
Expand All @@ -49,8 +49,8 @@ def test_http_request_bytes_body():


def test_http_request_body_conversion_str():
with pytest.raises(TypeError):
HttpRequest("http://example.com", body="content")
request = HttpRequest("http://example.com", body="content")
assert request.body == b"content"


def test_http_request_body_validation_None():
Expand Down
14 changes: 12 additions & 2 deletions web_poet/requests.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,17 @@ class RequestBackendError(Exception):
class HttpRequestBody(bytes):
"""A container for holding the raw HTTP request body in bytes format."""

pass
@classmethod
def from_anystr(
cls: Type[T_body], value: AnyStr, encoding: str = "utf-8"
) -> T_body:
"""An alternative constructor to handle both ``str`` and ``bytes`` types
of input argument.
"""

if isinstance(value, str):
return cls(value, encoding=encoding)
return cls(value)


class HttpRequestHeaders(CIMultiDict):
Expand Down Expand Up @@ -93,7 +103,7 @@ class HttpRequest:
factory=HttpRequestHeaders, converter=HttpRequestHeaders
)
body: HttpRequestBody = attrs.field(
factory=HttpRequestBody, converter=HttpRequestBody
factory=HttpRequestBody, converter=HttpRequestBody.from_anystr # type: ignore
)


Expand Down