Growatt integration missing but basic entities

[BigFam194]

The problem

If adding an optional smart meter to the Growatt solar installation you feed additional data from the smart meter to the Growatt inverter. This extra information is exposed in Growatt app and the Growatt portal but is not visible as new additional entities in the integration. In addition to the basic solar production data and the inverter itself you also get (net) energy consumption of your house, export and import from grid data, these data entities are not surfaced by the integration.
After installing the smart meter, the Shine Phone app automatically displays the new additional information but no additional entities show up in home assistant. Tried to restart ha and also to remove and add the Growatt integration with same results.

What version of Home Assistant Core has the issue?

2022.11

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Growatt

Link to integration documentation on our website

https://www.home-assistant.io/integrations/growatt_server/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant]

Hey there @indykoning, @muppet3000, @JasperPlant, mind taking a look at this issue as it has been labeled with an integration (growatt_server) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of growatt_server can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Change the title of the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign growatt_server Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

growatt_server documentation
growatt_server source
_{^{(message by IssueLinks)}}

[bartpaeleman]

Sounds exactly the same as the problem I have reported with the sensors that I lost after the upgrade where I had all of my battery information coming from. @BigFam194 do you happen to talk about a datalogger like this one that I too got added when my batteries were installed?

[BigFam194]

Bart, yes it's a smart meter (data logger) of the same make but another model. The entities that were there before are still there but no additional coming from the smart meter.

[muppet3000]

Hi Both,

Thanks for logging this, I'm guessing what has happened is that when Growatt were making all the changes last week they changed something in the API that means your batteries that used to be presented through one method in the API are not longer presented in the same way.
Due to the way that this integration works we're unable to see what the people at Growatt are planning and therefore have to react.
Ultimately, if these values are still available in the ShinePhone app, it means we can get them back it just may take a bit of effort.

If you're willing to share your credentials with me (my contact info is on my Github page) then I can try and log into your systems myself and work out what additional API calls need to be made to get these sensors back.
I can't promise it'll be quick though as this is something I only work on in my spare time and there are lots of other issues that have been tagged against this integration that I've been promising to work on for some time.

I will update this ticket later with a link to a forum post that I'm making with a prioritised list of all of the tickets.

In the meantime @bartpaeleman - You posted some really useful 'before' and 'after' screenshots on this ticket: #80950 earlier, would you mind copy/pasting them over to this ticket so I can track it all in one place?

Edit: Can you please also tell me what type of system you have i.e. the bit I've circled below from the Growatt Web UI:

[muppet3000]

Hi All,

This is a massive cross-post on all the Growatt tickets so apologies for it's generic format. I've collated all of the open issues and prioritised them here: https://community.home-assistant.io/t/growatt-integration-prioritised-list-of-features-for-implementation-fixing/483850

I will be working through them in order and updating that post as well as the affected tickets. Using that method I can be sure to correctly track everything associated with the integration.

Watch this space!

[Saentist]

There is no Second and Third Phase values in three phase inverters

Inverter have two string connectors
integration show 3 strings ;)
Growatt app show 8 ;) ;)

[bartpaeleman]

Upon request from @muppet3000 posting the information before/after the new release got installed here, with no batteries shown in the end. I lost the following sensors which as you can see in this screenshot did work before:

• sensor.thuis_batterij_lifetime_total_energy_input_1
• sensor.thuis_batterij_lifetime_total_energy_input_2
• sensor.thuis_batterij_lifetime_energy_output

There are a lot more sensors but the above were used on the energy dashboard:

BEFORE THE UPGRADE ENERGY VIEW:

AFTER THE UPGRADE ENERGY VIEW:

I do think that my data comes from a datalogger device that was installed when I had (Growatt) batteries added to my configuration but it was working before so no clue what I can do to get that back?

[bartpaeleman]

Can you please also tell me what type of system you have i.e. the bit I've circled below from the Growatt Web UI:

This is what I see:

UPDATE: @muppet3000, temporary credentials sent to you for access to my system.

[bartpaeleman]

@muppet3000 did you receive my email?

[charlie063]

same issue here, after last HA update, no more data... if, here you can find the API documentation

https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

[muppet3000]

@muppet3000 did you receive my email?

What email? I've only seen notifications in this thread from you?

[muppet3000]

same issue here, after last HA update, no more data... if, here you can find the API documentation

https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

No more data just for the batteries?

[charlie063]

same issue here, after last HA update, no more data... if, here you can find the API documentation
https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

No more data just for the batteries?

no data at all... where I have two inverters, on single phase, on three phase, if you can share me who does have the batteries and BMS on stock? -european supplier preferred-

[Saentist]

same issue here, after last HA update, no more data... if, here you can find the API documentation

https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

Outdated API from 2016-12-20

@charlie063
GWL - Elerix

[charlie063]

same issue here, after last HA update, no more data... if, here you can find the API documentation
https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

Outdated API from 2016-12-20

@charlie063 GWL - Elerix

do you have a new API manual?

[muppet3000]

Not sure what you're asking about stock for, but when I get around to looking at this issue (which is nearing the top of my list) I'll reach out for credentials from you guys so I can take a look and work out why things have stopped working for you.

Please keep a track of things on this forum post: https://community.home-assistant.io/t/growatt-integration-prioritised-list-of-features-for-implementation-fixing/483850 where I'm updating what I'm working on.

[muppet3000]

same issue here, after last HA update, no more data... if, here you can find the API documentation
https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

Outdated API from 2016-12-20
@charlie063 GWL - Elerix

do you have a new API manual?

There is no new API manual, we reverse engineer it from the Android Application, there is no official support from Growatt via an API

[charlie063]

same issue here, after last HA update, no more data... if, here you can find the API documentation
https://growatt.pl/wp-content/uploads/2020/01/Growatt-Server-API-Guide.pdf

Outdated API from 2016-12-20

@charlie063 GWL - Elerix

got it working... mamma, I made the default error... rather username I used to registered email address. now I threw myself a fishbone

[bartpaeleman]

@muppet3000 did you receive my email?

What email? I've only seen notifications in this thread from you?

email with credentials to log into my system

[muppet3000]

@muppet3000 did you receive my email?

What email? I've only seen notifications in this thread from you?

email with credentials to log into my system

Got it this time, responded to you directly.

[muppet3000]

Hi Everyone, this is now the next item on my list, it's going to take me a little while to get my head around though.

What would be REALLY helpful is if someone can send me a screengrab from the shinephone app and highlight the specific values on there that are now missing from home-assistant.

What I think has happened is that when they (growatt) made all of the changes the other week they changed where this information came from.

@bartpaeleman - Thanks for the screengrab above with all of the missing values and what they used to be shown as, that's REALLY helpful, it shows me that it was previously identifying with Growatt as a 'tlx' system, but something may have changed and I need to re-map some values.

Could someone please send me a screengrab of all of the entities that are working and all that aren't please, then I can work out what's missing.

[Saentist]

@muppet3000



I. no string connected
II. Inverter don't have third string

missing:
L2 and L3
#81470 (comment)

UE series Inverter /6000UE/

it will be good to see
Mode and Status of Inverter
and
Signal and Status of Logger

[bartpaeleman]

@muppet3000 I had all my entities on a single test page too:

[muppet3000]

Thanks @Saentist - Can you also please send me a screengrab from the ShinePhone app on your mobile phone and circle the values on there so I can match them up?

NOTE - My time is going to be focussed on a a proper fix to #81951 currently, but as soon as I've got on top of that I'll be back looking at this again!

[Saentist]

[muppet3000]

Thanks - just what I needed, I think they must have just changed the api call.
Can you tell me the names from the screengrabs above that relate to the values that are now missing in the Home Assistant integration? That way when I next look at this I can map the values and work out why they're missing.

[Saentist]

Vac[R,S,T] (V)
Correspond to L1 L2 L3
L2 and L3 missing
In ShinePhone cannot see some of element with can be seen in web portal.
Will upload when there is Sun above horizon.

[danant96]

Hi Both,

Thanks for logging this, I'm guessing what has happened is that when Growatt were making all the changes last week they changed something in the API that means your batteries that used to be presented through one method in the API are not longer presented in the same way. Due to the way that this integration works we're unable to see what the people at Growatt are planning and therefore have to react. Ultimately, if these values are still available in the ShinePhone app, it means we can get them back it just may take a bit of effort.

If you're willing to share your credentials with me (my contact info is on my Github page) then I can try and log into your systems myself and work out what additional API calls need to be made to get these sensors back. I can't promise it'll be quick though as this is something I only work on in my spare time and there are lots of other issues that have been tagged against this integration that I've been promising to work on for some time.

I will update this ticket later with a link to a forum post that I'm making with a prioritised list of all of the tickets.

In the meantime @bartpaeleman - You posted some really useful 'before' and 'after' screenshots on this ticket: #80950 earlier, would you mind copy/pasting them over to this ticket so I can track it all in one place?

Edit: Can you please also tell me what type of system you have i.e. the bit I've circled below from the Growatt Web UI:

In relation to the first post, I too have an Eastron smart meter and in the web interface, the values for import, export, load consumption are displayed in the Single Exportlimit device type. Integration shows values from the inverter but not from the datalogger associated with the Single Exportlimit. Could that be the issue with some of the missing values?

[bartpaeleman]

Thank you @muppet3000 for your dedication, professional approach and contribution. Really look forward to see the results.

[muppet3000]

Well lets see if it's fixed/working properly in the next release before we go celebrating! I'll keep an eye on it and report back if I notice anything odd.
Whenever you start using it please let me know if there's anything odd.

[BigFam194]

Thank you muppet3000 for all your work, dedication and passion to share!

This is probably redundant information and possibly already fixed by now but might be useful, especially the screengrabs which include data sources for some values.

Is it possible to add entities for "extended" values provided by peripherals to the inverter , e.g. a smart meter connected via Modbus RTU? These values become available in the Growatt Server portal and in iOS app (probably also Android App). Values are not native to the inverter itself but exposed in the portal as mentioned and the user experience in the app is that of a seemless "Solar plant". In particular I'm missing the "Import" (and export) values from the smart meter. Basic usage scenario is I want to visualize Powerflow in Home Assistant using 1/ Import from from grid 2/ Solar output (PPV) 3/ Load (sum of 1 and 2 and automatically calculated by the Powerflow custom lovelace card). The current integration only seems to provide solar output value.

As a side note, I would imagine a battery pack would also end up under the Device Type drop down and add more data sources (I currently only have an inverter and smart meter so can't verify)

[muppet3000]

@BigFam194 - Thanks for the feedback. When I was investigating using @bartpaeleman 's credentials (he has a battery) I found out the additional endpoints that are available on the app for this sort of stuff, I've added as many as were available with the existing upstream library implementation.
I plan to add the rest as part of my work in the New Year when I add support for the MAX/MID inverter type as all those changes will be going into the upstream library at the same time.
I plan to keep this ticket open and then mark it as resolved once all the information from the additional API calls are added, you'll just have to be a little patient until I get around to it.

In the meantime the latest release/dev build will add some new values that weren't there before.

[muppet3000]

For those of you that were waiting on the latest release to test this, it was released last night (2022.12.7), I recommend switching to that and testing it.
You may need to removed and re-add the integration in order for it to pick up the new values.
Please can someone let me know that it's solving the bulk of this problem and also solving the overnight reset which causes the energy dashboard to show massive values.

[BigFam194]

Upgraded HA to 2022.12.7. Did not have to remove and re-add integration. Happy to report that the battery entities are now visible and the total number of entities went from 27 to 54. Unfortunately I personally don’t have batteries so they are all showing “0” as expected. I look forward to a future release where even more entities as per earlier thread are added. Very thankful for you dedication and work. Will buy you coffee :-) Skickat från Outlook för iOS<https://aka.ms/o0ukef>

…

________________________________ Från: Chris Straffon ***@***.***> Skickat: Sunday, December 18, 2022 10:11:08 AM Till: home-assistant/core ***@***.***> Kopia: BigFam ***@***.***>; Mention ***@***.***> Ämne: Re: [home-assistant/core] Growatt integration missing but basic entities (Issue #81470) For those of you that were waiting on the latest release to test this, it was released last night (2022.12.7), I recommend switching to that and testing it. You may need to removed and re-add the integration in order for it to pick up the new values. Please can someone let me know that it's solving the bulk of this problem and also solving the overnight reset which causes the energy dashboard to show massive values. — Reply to this email directly, view it on GitHub<#81470 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AO3GNOTCMKYRIRGS4MCDL73WN3IKZANCNFSM6AAAAAARWBYGVQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[BigFam194]

Upgraded HA to 2022.12.7. Did not have to remove and re-add integration. Happy to report that the battery entities are now visible and the total number of entities went from 27 to 54. Unfortunately I personally don’t have batteries so they are all showing “0” as expected. I look forward to a future release where even more entities as per earlier thread are added.
Very thankful for you dedication and work. Will buy you coffee :-)

[bartpaeleman]

Awesome job Chris @muppet3000 ! I seem to have all entities (and more) back again and am now relinking my existing automations and nodered flows to see if they all deliver as before. It sure looks VERY promising and I wanted once again to extend my gratitude for your awesome work!

PS: Keeping your account to access my system alive, it may heklp you develop or troubleshoot again in the future.

[charlie063]

@muppet3000 details of the data seems correct, where somehow... the graphs, I #^&@$ them up

I really have no idea what I am doing faulty, as the data is ...

[muppet3000]

Upgraded HA to 2022.12.7. Did not have to remove and re-add integration. Happy to report that the battery entities are now visible and the total number of entities went from 27 to 54. Unfortunately I personally don’t have batteries so they are all showing “0” as expected. I look forward to a future release where even more entities as per earlier thread are added. Very thankful for you dedication and work. Will buy you coffee :-)

Thanks for confirming, and offering to buy me a coffee, I actually added a 'buy-me-a-beer' repo on my github page the other day because I saw someone else with one and thought it looked cool. I don't even mind if it never gets used, I was just playing with it :)

Glad to hear it's working as expected, on the dashboard you can just change the display to hide the ones that aren't relevant to your system. The API just returns everything so there's no easy way to determine what is relevant for every system.

[muppet3000]

Awesome job Chris @muppet3000 ! I seem to have all entities (and more) back again and am now relinking my existing automations and nodered flows to see if they all deliver as before. It sure looks VERY promising and I wanted once again to extend my gratitude for your awesome work!

PS: Keeping your account to access my system alive, it may heklp you develop or troubleshoot again in the future.

Thanks @bartpaeleman - Great to hear it's up and running again, thanks for leaving my guest account open, I plan to use it when I add the additional values so they'll be really handy.

[muppet3000]

@muppet3000 details of the data seems correct, where somehow... the graphs, I #^&@$ them up

I really have no idea what I am doing faulty, as the data is ...

Are you sure you've updated to the latest version of Home Assistant (version 2022.12.7) that's the one that has my fix in it and will ensure the huge jumps like that don't happen.

If you are sure that you're using the latest version then if you share the credentials with me for your system then I'll happily try and debug it for you by running it locally.

[charlie063]

@muppet3000 details of the data seems correct, where somehow... the graphs, I #^&@$ them up
I really have no idea what I am doing faulty, as the data is ...

Are you sure you've updated to the latest version of Home Assistant (version 2022.12.7) that's the one that has my fix in it and will ensure the huge jumps like that don't happen.

If you are sure that you're using the latest version then if you share the credentials with me for your system then I'll happily try and debug it for you by running it locally.

in your email :)

[Saentist]

Why's important to see more data.
Electrical company good quality to users.

[sidequest-tomas]

great input from everyone! I also see the entities now, but for example "batteries discharged today" show the numbers for exported to grid today, not discharge numbers. See compare screenshots :

[muppet3000]

also see the entities now, but for example "batteries discharged today" show the numbers for exported to grid today, not discharge numbers. See compare screen

Thanks for pointing this out. I'm currently dealing with the chaos from the fact Growatt changed their user-agent identification method again!
Once I'm on top of things and back in the swing post-Christmas I'll take a look at what's gone wrong here.

[charlie063]

also see the entities now, but for example "batteries discharged today" show the numbers for exported to grid today, not discharge numbers. See compare screen

Thanks for pointing this out. I'm currently dealing with the chaos from the fact Growatt changed their user-agent identification method again! Once I'm on top of things and back in the swing post-Christmas I'll take a look at what's gone wrong here.

therefor, every user can have their own logon with with access to their own APIkey maximum poll is 1 / 5 sec. if you read the API documentation, it is in there my dear friend. related to the changes of the user-agent? there is something about the IP-adres of the user, I I look into what we investigated?

[frenck]

I've hidden your comment @muppet3000. The Home Assistant project does not recommend or support the use of custom integrations.

../Frenck

[muppet3000]

A great suggestion has been made on the above (see hidden comment) forum link to continue support of the Core integration by taking a monthly roll-up of the changes in the HACS version of the integration and submitting them via the formal review process into Core. This will ensure that the integration continues to meet the standards expected of a Core integration.

Therefore, those of you that wish to continue using just the Core version of the integration will still be able to it will just receive updates/fixes at a slightly slower rate.

I will continue to monitor & respond to bugs and issues raised in Core but they will be first fixed externally before making it into a Core PR.

I hope this provides the best of both worlds for everyone involved.

[frenck]

Please note that that model does have downsides; for example, we will be changing the core version to keep up with core changes and will not contribute those fixes and changes to custom integrations.

Additionally, custom integrations have to take into account multiple versions of Home Assistant in the same codebase (e.g., take care of compatibility differences), while core integrations do not need such logic.

The code base, therefore, will there diverge and not be the same. Your statement is therefore, something that generally can't be achieved.

@muppet3000 Will you please stop crossposting the same message on all issues related to growatt_server? Consider that a warning, please. That is not wished for.

../Frenck

[issue-triage-workflows]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.