Commit Graph

116 Commits

Author SHA1 Message Date
Stefan Allius
e0777dca8e Add support for MS-3000 inverter (#299)
* split register map into multiple maps

* add base support reg mapping 0x01900000

* fix shadowed builtin

* detect reg mapping for sensor automatically

* add device and test regs for MS-3000

* add more register mappings

* fix unit tests

* add more MS-3000 registers

* build modell string for TSUN MS-3000

* add MS3000 unit test

* remove obsolete method __set_config_parms

* fix start addr of modbus scans

- in server mode the start addr must be reduced
  by mb_step

* add tests for sensor_list of ms-3000 inverters

* MS-3000: add integer test register

* DCU-1000: add Out Status register

* add integer test and batterie out register

* fix Sonar Qube finding

* DCU-1000: add temp sensors
2025-03-16 18:49:01 +01:00
Stefan Allius
955657fd87 add first costumer apparmor definition (#296)
* add first costumer apparmor definition

* add initial apparmor support
2025-03-16 13:11:03 +01:00
Stefan Allius
88cb01f613 add Modbus polling mode for DCU1000 (#305)
* add Modbus scanning mode

* fix modbus polling for DCU 1000

* add modbus register for DCU 1000

* calculate meta values from modbus regs

* update changelog

* reduce code duplication

* refactor modbus_scan

* add additional unit tests
2025-03-11 19:47:37 +01:00
Stefan Allius
10b4a84701 allow R47serial numbers for GEN3 inverters (#302) 2025-03-02 19:05:36 +01:00
Stefan Allius
3f3ed1b14f add watchdog for Add-ons (#291) 2025-02-27 16:11:32 +01:00
Stefan Allius
036dd6d1dc S allius/issue281 (#282)
* accept DCU serial number starting with '410'

* determine sensor-list by serial number

* adapt unit test for DCU support

* send first batterie measurements to home assistant

* add test case for sensor-list==3036

* add more registers for batteries

* improve error logging (Monitoring SN)

* update the add-on repro only for one stage

* add configuration for energie storages

* add License and Readme file to the add-on

* addon: add date and time to dev and debug docker container tag

* disable duplicate code check for config.py

* cleanup unit test, remove trailing whitespaces

* update changelog

* fix example config for batteries

* cleanup config.jinja template

* fix comments

* improve help texts
2025-02-24 22:39:34 +01:00
Stefan Allius
ec3af69e62 S allius/issue288 (#289)
* remove apostrophes from fmt strings

- thanks to @onkelenno for the suggestion

* improve the logger initializing

- don't overwrite the logging.ini settings if the
env variable LOG_LVL isn't well defined
- Thanks to @onkelenno for the idea to improve

* set default argument for LOG_LVL to INFO in docker files

* adapt unit test
2025-02-23 14:17:57 +01:00
Stefan Allius
5f0a35d55b Update AddOn base docker image to version 17.1.3 and python3 to 3.12.9-r0 2025-02-11 20:45:01 +01:00
Stefan Allius
4df36e2672 revert AddOn base docker image to version 17.1.0 2025-02-11 20:20:48 +01:00
Stefan Allius
42fe33bacf add initial DCU support 2025-02-11 00:08:57 +01:00
Stefan Allius
01ad8eff6b Merge branch 'main' of https://github.com/s-allius/tsun-gen3-proxy into dev-0.13 2025-01-16 19:37:22 +01:00
Stefan Allius
53c76e72a2 Dev 0.12 (#275)
* bump version to 0.12.1

* add initial version for release candidates

* add rc version

* version 0.12.1

* addon: bump base image version to v17.1.0

* 270 ha addon add syntax check to config parameters (#274)

* fixed requirement status of client mode host

---------

Co-authored-by: Michael Metz <michael.metz@siemens.com>

---------

Co-authored-by: metzi <147942647+mime24@users.noreply.github.com>
Co-authored-by: Michael Metz <michael.metz@siemens.com>
2025-01-16 19:31:30 +01:00
Stefan Allius
3234e87b55 S allius/issue180 (#265)
* move default_config.toml into src/cnf/.

* improve file handling

* remove obsolete rules
2024-12-24 00:13:32 +01:00
Stefan Allius
3bf245300d Dev 0.12 (#257)
* add ha_addons repository to cscode workspace

* Issue220 ha addon dokumentation update (#232)

* initial DOCS.md for Addon

* links to Mosquitto and Adguard

* replaced _ by . for PV-Strings

* mentioned add-on installation method in README.md

* fix most of the markdown linter warnings

* add missing alt texts

* added nice add repository to my Home Assistant badges

---------

Co-authored-by: Michael Metz <michael.metz@siemens.com>
Co-authored-by: Stefan Allius <stefan.allius@t-online.de>

* S allius/issue216 (#235)


* improve docker run

- establish multistage Dockerfile
- build a python wheel for all needed packages
- remove unneeded tools like apk for runtime

* pin versions, fix hadolint warnings

* merge from dev-0.12

---------

Co-authored-by: Michael Metz <michael.metz@siemens.com>

* Issue220 ha addon dokumentation update (#245)

* revised config disclaimer

* add newline at end of file to fix linter warning

---------

Co-authored-by: Michael Metz <michael.metz@siemens.com>

* 238 ha addon repository check (#244)

* move Makefile and bake file into parent folder

* build config.yaml from template

* use Makefile instead of build shell script

* ignore temporary or created files

* add rules for building the add-on repository

* add rel version of add-on

* add  jinja2-cli

* ignore inverter replays which a older than 1 day (#246)

* S allius/issue7 (#248)

* report alarm and fault bitfield to ha

* define the alarm and fault names

* configure log path and max number of daily log files (#243)

* configure log path and max number of daily log files

* don't use a subfolder for configs

* use make instead of a build script

* mount /homeassistant/tsun-proxy

* Add venv to base image

* give write access to mounted folder

* intial checkin, ignore SC1091

* set advanced and stage value in config.yaml

* fix typo

* added watchdog and removed Port 8127 from mapping

* fixed typo and use new add-on repro

- change the install button to install from
 https://github.com/s-allius/ha-addons

* add addon-rel target

* disable watchdog due to exceptions in the ha supervisor

* update changelog

---------

Co-authored-by: Michael Metz <michael.metz@siemens.com>

* Update README.md (#251)

install `https://github.com/s-allius/ha-addons` as repro for our add-on

* add german language file (#253)

* fix return type get_extra_info in FakeWriter

* move global startup code into main methdod

* pin version of base image

* avoid forwarding to a private (lokal) IP addr (#256)

* avoid forwarding to a private (lokal) IP addr

* test DNS resolver issues

* increase test coverage

* update changelog

* fix client_mode configuration block (#252)

* fix client_mode block

* add client mode

* fix tests with client_mode values

* log client_mode configuration

* add forward flag for client_mode

* improve startup logging

* added client_mode example

* adjusted translation files

* AT commands added

* typo

* missing "PLUS"

* link to config details

* improve log msg for config problems

* improve log msg on config errors

* improve log msg for config problems

* copy CHANGELOG.md into add-on repro

---------

Co-authored-by: Michael Metz <michael.metz@siemens.com>

* rename "ConfigErr" to match naming convention

* disable test coverage for __main__

* update changelog version 0.12

---------

Co-authored-by: metzi <147942647+mime24@users.noreply.github.com>
Co-authored-by: Michael Metz <michael.metz@siemens.com>
2024-12-22 22:25:50 +01:00
Stefan Allius
3c81d446dd update changelog 2024-12-08 18:57:40 +01:00
Stefan Allius
be3b4d6df0 S allius/issue206 (#213)
* update changelog

* add addon-dev target

* initial version

* use prebuild docker image

* initial version for multi arch images

* fix missing label latest

* create log and config folder first.

* clean up and translate to english

* set labels with docker bake

* add addon-debug and addon-dev targets

* pass version number to proxy at runtime

* add two more callbacks

* get addon version from app

* deploy rc addon container to ghcr

* move ha_addon test into subdir

* fix crash on container restart

- mkdir -p returns no error even if the director
  exists

* prepation for unit testing

- move script into a method

* added further config to schema

* typo fixed

* added monitor_sn + PV-strings 3-6 to create toml

* added options.json for testing

* prepare pytest and coverage for addons

* fix missing values in resulting config.toml
- define mqtt default values
- convert filter configuration

* first running unittest for addons

* add ha_addons

* increase test coverage

* test empty options.json file for HA AddOn

* fix pytest call in terminal

* improve test coverage

* remove uneeded options.json

* move config.py into subdir cnf

---------

Co-authored-by: Michael Metz <michael.metz@siemens.com>
2024-12-03 22:22:50 +01:00
Stefan Allius
668c631018 S allius/issue222 (#223)
* github action: use ubuntu 24.04 and sonar-scanner-action 4
2024-12-02 23:41:58 +01:00
Stefan Allius
28cf875533 migrate paho.mqtt CallbackAPIVersion to VERSION2 (#225) 2024-12-02 22:49:56 +01:00
Stefan Allius
3bada76516 S allius/pytest (#211)
* - fix pytest setup that can be startet from the rootdir
  - support python venv environment
  - add pytest.ini
  - move common settings from .vscode/settings.json into pytest.ini
  - add missing requirements
  - fix import paths for pytests

* - support python venv environment

* initial version

* - add missing requirements python-dotenv

* fix import paths for pytests

* fix pytest warnings

* initial version

* report 5 slowest test durations

* add more vscode settings for python
2024-11-24 22:07:43 +01:00
Stefan Allius
d4fd396dcf Merge branch 'main' of https://github.com/s-allius/tsun-gen3-proxy into dev-0.12 2024-11-20 22:09:53 +01:00
dependabot[bot]
976eaed9ea Bump aiohttp from 3.10.5 to 3.10.11 in /app (#209)
* Bump aiohttp from 3.10.5 to 3.10.11 in /app

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.10.5 to 3.10.11.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.10.5...v3.10.11)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* bump sonarcloud-github-action to v3.1.0

* prepare version 0.11.1

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Stefan Allius
2024-11-20 21:08:47 +01:00
Stefan Allius
211a958080 add PROD_COMPL_TYPE to trace 2024-11-20 20:08:20 +01:00
Stefan Allius
78a35b5513 report alarm and fault bitfield to ha (#204)
* report alarm and fault bitfield to home assistant

* initial verson of message builder for SolarmanV5

- for SolarmaV5 we build he param field for the
  device and inverter message from the internal
  database
- added param description to the info table
  for constant values, which are not parsed and
  stored in internal database

* define constants for often used format strings

* update changelog
2024-11-02 15:09:10 +01:00
Stefan Allius
a6ffcc0949 update version 0.11 2024-10-13 18:24:00 +02:00
Stefan Allius
719c6f703a S allius/issue196 (#198)
* fix healthcheck

- on infrastructure with IPv6 support localhost
  might be resolved to an IPv6 adress. Since the
  proxy only support IPv4 for now, we replace
  localhost by 127.0.0.1, to fix this
2024-10-13 17:13:07 +02:00
Stefan Allius
62ea2a9e6f Refactoring async stream (#194)
* GEN3: Invalid Contact Info Msg
Fixes #191

* introduce ifc with FIFOs

* add object factory

* use AsyncIfc class with FIFO

* declare more methods as classmethods

* - refactoring

- remove _forward_buffer
- make async_write private

* remove _forward_buffer

* refactoring

* avoid mqtt handling for invalid serial numbers

* add two more callbacks

* FIX update_header_cb handling

* split AsyncStream in two classes

* split ConnectionG3(P) in server and client class

* update class diagramm

* refactor server creation

* remove duplicated imports

* reduce code duplication

* move StremPtr instances into Inverter class

* resolution of connection classes

- remove ConnectionG3Client
- remove ConnectionG3Server
- remove ConnectionG3PClient
- remove ConnectionG3PServer

* fix server connections

* fix client loop closing

* don't overwrite self.remote in constructor

* update class diagramm

* fixes

- fixes null pointer accesses
- initalize AsyncStreamClient with proper
  StreamPtr instance

* add close callback

* refactor close handling

* remove connection classes

* move more code into InverterBase class

* remove test_inverter_base.py

* add abstract inverter interface class

* initial commit

* fix sonar qube warnings

* rename class Inverter into Proxy

* fix typo

* move class InverterIfc into a separate file

* add more testcases

* use ProtocolIfc class

* add unit tests for AsyncStream class

* icrease test coverage

* reduce cognitive complexity

* increase test coverage

* increase tes coverage

* simplify heartbeat handler

* remove obsolete tx_get method

* add more unittests

* update changelog

* remove __del__ method for proper gc runs

* check releasing of ModbusConn instances

* call garbage collector to release unreachable objs

* decrease ref counter after the with block
2024-10-13 16:07:01 +02:00
Stefan Allius
614acbf32d update changelog 2024-09-15 01:18:36 +02:00
Stefan Allius
be4c6ac77f S allius/issue182 (#183)
* GEN3: After inverter firmware update the 'Unknown Msg Type' increases continuously
Fixes #182

* add support for Controller serial no and MAC

* test hardening

* GEN3: add support for new messages of version 3 firmwares

* bump libraries to latest versions

- bump aiomqtt to version 2.3.0
- bump aiohttp to version 3.10.5

* improve test coverage

* reduce cognective complexity
2024-09-07 11:45:16 +02:00
Stefan Allius
215dcd98e6 GEN3: don't crash on overwritten msg in the receive buffer 2024-09-03 17:22:34 +02:00
Stefan Allius
33d385db10 optimise versionstring handling (#159)
- Reading the version string from the image updates
  it even if the image is re-pulled without re-deployment
2024-08-10 22:53:25 +02:00
Stefan Allius
383be10e87 Hotfix v0.10.1: fix displaying the version string at startup and in HA (#155)
* Version 0.10.0 no longer displays the version string (#154)

Fixes #153
2024-08-10 14:18:25 +02:00
Stefan Allius
b364fb3f8e Dev 0.10 (#151)
* S allius/issue117 (#118)

* add shutdown flag

* add more register definitions

* add start commando for client side connections

* add first support for port 8899

* fix shutdown

* add client_mode configuration

* read client_mode config to setup inverter connections

* add client_mode connections over port 8899

* add preview build

* Update README.md

describe the new client-mode over port 8899 for GEN3PLUS

* MODBUS: the last digit of the inverter version is a hexadecimal number (#121)

* S allius/issue117 (#122)

* add shutdown flag

* add more register definitions

* add start commando for client side connections

* add first support for port 8899

* fix shutdown

* add client_mode configuration

* read client_mode config to setup inverter connections

* add client_mode connections over port 8899

* add preview build

* add documentation for client_mode

* catch os error and log thme with DEBUG level

* update changelog

* make the maximum output coefficient configurable (#124)

* S allius/issue120 (#126)

* add config option to disable the modbus polling

* read more modbus regs in polling mode

* extend connection timeouts if polling mode is disabled

* update changelog

* S allius/issue125 (#127)

* fix linter warning

* move sequence diagramm to wiki

* catch asyncio.CancelledError

* S allius/issue128 (#130)

* set Register.NO_INPUTS fix to 4 for GEN3PLUS

* don't set Register.NO_INPUTS per MODBUS

* fix unit tests

* register OUTPUT_COEFFICIENT at HA

* update changelog

* - Home Assistant: improve inverter status value texts

* - GEN3: add inverter status

* on closing send outstanding MQTT data to the broker

* force MQTT publish on every conn open and close

* reset inverter state on close

- workaround which reset the inverter status to
  offline when the inverter has a very low
  output power on connection close

* improve client modified
- reduce the polling cadence to 30s
- set controller statistics for HA

* client mode set controller IP for HA

* S allius/issue131 (#132)

* Make __publish_outstanding_mqtt public

* update proxy counter

- on client mode connection establishment or
  disconnecting update tje counection counter

* Update README.md (#133)

* reset inverter state on close

- workaround which reset the inverter status to
  offline when the inverter has a very low
  output power on connection close

* S allius/issue134 (#135)

* add polling invertval and method ha_remove()

* add client_mode arg to constructors

- add PollingInvervall

* hide some topics in client mode

- we hide topics in HA by sending an empty register
  MQTT topic during HA auto configuration

* add client_mode value

* update class diagram

* fix modbus close handler

- fix empty call and cleanup que
- add unit test

* don't sent an initial 1710 msg in client mode

* change HA icon for inverter status

* increase test coverage

* accelerate timer tests

* bump aiomqtt and schema to latest release (#137)

* MQTT timestamps and protocol improvements (#140)

* add TS_INPUT, TS_GRID and TS_TOTAL

* prepare MQTT timestamps

- add _set_mqtt_timestamp method
- fix hexdump printing

* push dev and debug images to docker.io

* add unix epoche timestamp for MQTT pakets

* set timezone for unit tests

* set name für setting timezone step

* trigger new action

* GEN3 and GEN3PLUS: handle multiple message

- read: iterate over the receive buffer
- forward: append messages to the forward buffer
- _update_header: iterate over the forward buffer

* GEN3: optimize timeout handling

- longer timeout in state init and reveived
- got to state pending only from state up

* update changelog

* cleanup

* print coloured logs

* Create sonarcloud.yml (#143)

* Update sonarcloud.yml

* Update sonarcloud.yml

* Update sonarcloud.yml

* Update sonarcloud.yml

* Update sonarcloud.yml

* build multi arch images with sboms (#146)

* don't send MODBUS request when state is not up (#147)

* adapt timings

* don't send MODBUS request when state is note up

* adapt unit test

* make test code more clean (#148)

* Make test code more clean (#149)

* cleanup

* Code coverage for SonarCloud (#150)


* cleanup code and unit tests

* add test coverage for SonarCloud

* configure SonarCloud

* update changelog

* Do no build on *.yml changes

* prepare release 0.10.0

* disable MODBUS_POLLING for GEN§PLUS in example config

* bump aiohttp to version 3.10.2

* code cleanup

* Fetch all history for all tags and branches
2024-08-09 23:16:47 +02:00
Stefan Allius
a42ba8a8c6 Dev 0.9 (#115)
* make timestamp handling stateless

* adapt tests for stateless timestamp handling

* initial version

* add more type annotations

* add more type annotations

* fix Generator annotation for ha_proxy_confs

* fix names of issue branches

* add more type annotations

* don't use depricated varn anymore

* don't mark all test as async

* fix imports

* fix solarman unit tests

- fake Mqtt class

* print image build time during proxy start

* update changelog

* fix pytest collect warning

* cleanup msg_get_time handler

* addapt unit test

* label debug images with debug

* dump droped packages

* fix warnings

* add systemtest with invalid start byte

* update changelog

* update changelog

* add exposed ports and healthcheck

* add wget for healthcheck

* add aiohttp

* use config validation for healthcheck

* add http server for healthcheck

* calculate msg prossesing time

* add healthy check methods

* fix typo

* log ConfigErr with DEBUG level

* Update async_stream.py

- check if processing time is < 5 sec

* add a close handler to release internal resources

* call modbus close hanlder on a close call

* add exception handling for forward handler

* update changelog

* isolate Modbus fix

* cleanup

* update changelog

* add heaithy handler

* log unrelease references

* add healtcheck

* complete exposed port list

* add wget for healtcheck

* add aiohttp

* use Enum class for State

* calc processing time for healthcheck

* add HTTP server for healthcheck

* cleanup

* Update CHANGELOG.md

* updat changelog

* add docstrings to state enum

* set new state State.received

* add healthy method

* log healthcheck infos with DEBUG level

* update changelog

* S allius/issue100 (#101)

* detect dead connections

- disconnect connection on Msg receive timeout
- improve connection trace (add connection id)

* update changelog

* fix merge conflict

* fix unittests

* S allius/issue108 (#109)

* add more data types

* adapt unittests

* improve test coverage

* fix linter warning

* update changelog

* S allius/issue102 (#110)

* hotfix: don't send two MODBUS commands together

* fix unit tests

* remove read loop

* optional sleep between msg read and sending rsp

* wait after read 0.5s before sending a response

* add pending state

* fix state definitions

* determine the connection timeout by the conn state

* avoid sending MODBUS cmds in the inverter's reporting phase

* update changelog

* S allius/issue111 (#112)

Synchronize regular MODBUS commands with the status of the inverter to prevent the inverter from crashing due to unexpected packets.

* inital checkin

* remove crontab entry for regular MODBUS cmds

* add timer for regular MODBUS polling

* fix Stop method call for already stopped timer

* optimize MB_START_TIMEOUT value

* cleanup

* update changelog

* fix buildx warnings

* fix timer cleanup

* fix Config.class_init()

- return error string or None
- release Schema structure after building thr config

* add quit flag to docker push

* fix timout calculation

* rename python to debugpy

* add asyncio log

* cleanup shutdown
- stop webserver on shutdown
- enable asyncio debug mode for debug versions

* update changelog

* update changelog

* fix exception in MODBUS timeout callback

* update changelog
2024-07-01 23:41:56 +02:00
Stefan Allius
f3e69ff217 Dev 0.8 (#107)
* S allius/issue102 (#103)

* hotfix: don't send two MODBUS commands together

* Update README.md

Exchange logger fw version with the real inverter fw version in the compatibility table

* Update python-app.yml

run also on pushes to issue branches
fix name for issues branches

* S allius/issue104 (#105)

* Update README.md

Exchange logger fw version with the real inverter fw version in the compatibility table

* Update python-app.yml

run also on pushes to issue branches
fix name for issues branches

* fix forwarding of MODBUS responses

* fix unit tests

* update changelog

* update changelog
2024-06-21 18:41:54 +02:00
Stefan Allius
a3c054d2b1 Dev 0.8 (#106)
* S allius/issue102 (#103)

* hotfix: don't send two MODBUS commands together

* Update README.md

Exchange logger fw version with the real inverter fw version in the compatibility table

* Update python-app.yml

run also on pushes to issue branches
fix name for issues branches

* S allius/issue104 (#105)

* Update README.md

Exchange logger fw version with the real inverter fw version in the compatibility table

* Update python-app.yml

run also on pushes to issue branches
fix name for issues branches

* fix forwarding of MODBUS responses

* fix unit tests

* update changelog
2024-06-21 18:12:48 +02:00
Stefan Allius
a62864218d update for version 0.8.0 2024-06-07 19:48:41 +02:00
Stefan Allius
49e2dfbd86 optimize docker-compose.yaml file 2024-06-04 20:27:15 +02:00
Stefan Allius
8204cae2b1 improve logging output 2024-06-03 19:52:37 +02:00
Stefan Allius
ec4261ae84 Merge branch 'dev-0.8.0' into s-allius/issue77 2024-05-31 20:17:03 +02:00
Stefan Allius
be57d11214 update changelog 2024-05-31 20:13:45 +02:00
Stefan Allius
407c1ceb2b control access via AT commands 2024-05-30 19:40:25 +02:00
Stefan Allius
3980ac013b catch all OSError errors in the read loop 2024-05-28 21:55:42 +02:00
Stefan Allius
3cc5f3ec53 - add Modbus fifo and timeout handler 2024-05-19 13:45:52 +02:00
Stefan Allius
841877305d timeout handler removed again, as it has no positive effect 2024-05-15 23:15:20 +02:00
Stefan Allius
26f108cc51 build version string in the same format as TSUN 2024-05-10 20:50:37 +02:00
Stefan Allius
537d81fa19 add graceful shutdown 2024-05-09 16:49:59 +02:00
Stefan Allius
2301511242 update documentation 2024-05-07 22:11:55 +02:00
Stefan Allius
5822f5de50 update changelog 2024-05-05 20:18:19 +02:00
Stefan Allius
4e8fd8e2a2 update changelog 2024-04-28 18:34:51 +02:00
Stefan Allius
aa0d432149 Update CHANGELOG.md
add version 0.7.0
2024-04-21 22:54:01 +02:00