Support for PUT requests

Changes

• Add support for PUT and validate method in schema case insensitive @danieldotnl (#456)
• Update dependency ruff to v0.8.3 @renovate (#458)
• Update dependency pytest-homeassistant-custom-component to v0.13.192 @renovate (#457)
• Update dependency pytest-homeassistant-custom-component to v0.13.191 @renovate (#455)
• Update dependency ruff to v0.8.2 @renovate (#449)
• Update dependency pytest-homeassistant-custom-component to v0.13.183 @renovate (#451)
• Update dependency colorlog to v6.9.0 @renovate (#446)

v8.0.3 Fixing issues with form_submit payload and submit_once

This release contains 2 small (but for some people important) changes:

• It was not possible to provide a request payload in form_submit as it was always overwritten by the input_values. Now the payload is used when no input_values are provided.
• Fix an issues with submit_once (cannot unpack non-iterable NoneType object)

Changes

• Fix issue with unpacking result when submit_once @danieldotnl (#448)
• Use payload in form-submit when no input_values are given @danieldotnl (#447)
• Update actions/checkout action to v4.2.2 @renovate (#444)
• Update actions/setup-python action to v5.3.0 @renovate (#445)
• Update dependency pytest-homeassistant-custom-component to v0.13.182 @renovate (#442)
• Update dependency ruff to v0.7.3 @renovate (#441)
• Update dependency pytest-homeassistant-custom-component to v0.13.173 @renovate (#440)
• Install gitlens in dev container @danieldotnl (#439)
• Update dependency ruff to v0.6.9 @renovate (#434)
• Update dependency pytest-homeassistant-custom-component to v0.13.172 @renovate (#435)
• Update actions/checkout action to v4.2.1 @renovate (#436)

v8.0.2 ☀️ Startup performance & restoring values

⚠️ This release contains some important changes! ⚠️

🎉 I'm super excited about this new release! It's containing some changes that are not breaking but do make Multiscrape work slightly different than before. 🚀 Read on!

⚡ Performance Boosts!

This release brings huge improvements to startup performance. While it’s a big leap forward, you might not feel the impact as much because of the great improvements in Home Assistant over the past few months. But trust me, it’s fast! ⚡
On my production system:

🔄 Restoring Previous Values

Both sensor states and attribute values are now restored after a reboot, meaning you’ll be right back where you left off! No more waiting for new scrapes to kick in to get your data back after restarting. 🔄 (It is also restored when you set 'last' in 'on-error')

⏸️ No More Scrapes with scan_interval: 0

Here’s a subtle but important change: if you set scan_interval to 0, Multiscrape will no longer scrape on startup. Previously, it would still initiate a scrape on startup, but now it stays idle just like you’d expect. 🛑

As always, thanks for using Multiscrape! Let me know how it’s working for you. Happy scraping! 🕷️💻

Other notes

• I want to dedicate this release to @Madelena, Product Manager @ Nabu Casa, as she (unknowingly) was one of the main triggers for this release. Seeing Multiscrape pop up here was painful 😅: Release party 2024.4
• Happy with Multiscrape? Consider to buy me a coffee or sponsor me on Github!
• Check out my other custom integration: MeasureIt!

Changes

• Fix scrape on startup when interval = 0 @danieldotnl (#425)
• Update dependency ruff to v0.6.7 @renovate (#424)
• Update dependency pytest-homeassistant-custom-component to v0.13.163 @renovate (#423)
• Restore entity's last state on startup @danieldotnl (#417)
• Update dependency ruff to v0.6.5 @renovate (#422)
• Update dependency ruff to v0.6.4 @renovate (#419)
• Update dependency pytest-homeassistant-custom-component to v0.13.162 @renovate (#420)
• Update dependency pytest-homeassistant-custom-component to v0.13.161 @renovate (#415)
• Update README.md @danieldotnl (#431)
• Update README.md with support options @danieldotnl (#430)
• Update dependency pytest-homeassistant-custom-component to v0.13.171 @renovate (#429)
• Update dependency pytest-homeassistant-custom-component to v0.13.166 @renovate (#426)
• Update dependency ruff to v0.6.8 @renovate (#428)
• Update actions/checkout action to v4.2.0 @renovate (#427)

v7.2.0 🚀 Scraping HTML

New Feature: Raw HTML Scraping with Multiscrape 🌐

I'm excited to announce that it is now possible to scrape raw HTML in Multiscrape! This feature has been a recurring request over the years, and I'm happy I could finally implement it. 🎉
It could for example be used to for displaying rich content on a markdown card.

A new configuration option for selectors has been added called extract. It is optional and can have these values:
- Text (default): Extracts plain text, as you are used to. 📝
- Content: Returns the content of the selected tag. 📜
- Tag: Returns both the content and the tag itself. 🏷️

With this feature, your sensors (or attributes) can now have a state/value like:

<p>This is an <b>example</b> of what can be scraped with the <i>extract</i> feature.</p>

Thank you for your continued support, and happy scraping! 🥳

Changes

• New feature to scrape and extract raw html @danieldotnl (#416)
• Update README.md @danieldotnl (#413)
• Update dependency pytest-homeassistant-custom-component to v0.13.158 @renovate (#414)
• Update dependency ruff to v0.6.3 - autoclosed @renovate (#410)
• Update dependency pytest-homeassistant-custom-component to v0.13.157 @renovate (#412)
• Update actions/setup-python action to v5.2.0 @renovate (#411)
• Update dependency pytest-homeassistant-custom-component to v0.13.156 @renovate (#408)

v7.1.2 ⭐ Cookies and form variables

I'm excited to share some awesome new features and improvements in this update! 🚀 Here’s what’s new:

✨ New Feature: Form Variables

A big shoutout to @jeremicmilan for his incredible dedication to this feature! 👏 I’ve added Form Variables to Multiscrape, allowing you to scrape the (token of a) page returned after logging in on some sites (specifically PHP). This token can then be sent in a header for authentication or other purposes. For all the details, make sure to check out the README! 📚

🍪 New Feature: Cookies support

You asked, I delivered! The long-awaited support for cookies is finally here! 🎉 Now, all cookies returned in HTTP sessions are automatically transferred to the next request. Plus, I’ve added logging so you can easily see which cookies are set. Sweet, right? 🍪

🤖 Automated Tests!!

I’m taking stability to the next level with the newly set up automated testing infrastructure! The first 2 automated tests have been added to Multiscrape, ensuring even more reliability in the future. Continuous improvements are on the way! 🛠️

As always, a huge thank you to the amazing community for your continued support and feedback. Happy scraping! 🕷️💻

pre-v7.1.1 🌈 Fix issue with hass variable in HA 2024.8

Changes

• Fix hass variable @danieldotnl (#394)
• Update dependency pytest-homeassistant-custom-component to v0.13.150 @renovate (#393)
• Update dependency ruff to v0.5.6 @renovate (#390)

v7.0.3 Fix issue with hass variable in HA 2024.8

As of HA release 2024.8 an issue (#391) occurs when using a value_template showing the following error:
Unable to scrape data: hass variable not set on template

This release will fix this.

pre-v7.1.0 🌈 Form variables

New features: Form variables

A big thank you to @jeremicmilan for his dedication on this one! We now have form variables which can be used to scrape the (token of a) page that's returned after logging in on some sites (specifically PHP). The token can then be send in a header. See the README for more details.

Changes

• Form variables @jeremicmilan (#374)
• Update dependency pytest-homeassistant-custom-component to v0.13.144 @renovate (#384)
• Update dependency ruff to v0.5.0 @renovate (#382)
• Update dependency pytest-homeassistant-custom-component to v0.13.138 @renovate (#383)
• Update dependency pytest-homeassistant-custom-component to v0.13.137 @renovate (#381)
• Update dependency pytest-homeassistant-custom-component to v0.13.136 @renovate (#379)
• Update dependency ruff to v0.4.10 @renovate (#378)
• Update dependency pytest-homeassistant-custom-component to v0.13.135 @renovate (#375)
• Update dependency ruff to v0.4.9 @renovate (#377)
• Update actions/checkout action to v4.1.7 @renovate (#376)
• Update dependency ruff to v0.4.8 @renovate (#371)
• Update dependency pytest-homeassistant-custom-component to v0.13.133 @renovate (#372)
• Update dependency pytest-homeassistant-custom-component to v0.13.124 @renovate (#370)
• Update actions/checkout action to v4.1.6 @renovate (#369)
• Update dependency pytest-homeassistant-custom-component to v0.13.123 @renovate (#358)
• Update dependency ruff to v0.4.4 @renovate (#367)
• Update actions/checkout action to v4.1.5 @renovate (#366)
• Update actions/checkout action to v4.1.4 @renovate (#362)
• Update dependency ruff to v0.4.3 @renovate (#357)
• Update dependency ruff to v0.3.6 @renovate (#356)
• Update dependency pytest-homeassistant-custom-component to v0.13.113 @renovate (#354)

v7.0.2 Separate http settings for form submit and scrape requests

Update: Breaking change

In this release, the http request settings for submitting a form are different from those for scraping. Before this release some of the setting were shared. If you need specific settings for form_submit (like headers or verify_ssl), you can specify those under the form_submit part of your configuration.

Changes

• Separate http settings for form submit and scrape request @danieldotnl (#353)
• Update dev environment @danieldotnl (#352)
• Update lint.yml to python 3.12 @danieldotnl (#349)

v7.0.1 Fix for missing trigger services

Changes

• Create a trigger service for each configuration. @danieldotnl (#351)
• Upgrade dev container to python 3.12 + prep pytest @danieldotnl (#350)
• Update dependency homeassistant to v2024.4.0 @renovate (#347)
• Update dependency ruff to v0.3.5 @renovate (#344)