need help with testing harness for writing DIDs

[alexdetrano]

I've written some code to allow a user to write to a DID from the command line. It works well when tested against a real ECU. However, I'm having issues with the testing harness. My write_did branch is here: https://github.com/alexdetrano/caringcaribou/tree/write_did

In class MockEcuIso14229, I've added the attribute WRITE_DATA, which represents some arbitrary data buffer that the user can write to using Write Data by Identifier. In the handle_write_data_by_identifier message handler, I added a new if statement for a new DID that will allow us to write to WRITE_DATA:

    def __init__(self, arb_id_request, arb_id_response, bus=None):
        MockEcu.__init__(self, bus)
        # <snip>
        self.WRITE_DATA = [0x00, 0x01, 0x02]

and further below in the message_handler

   # use a unique DID here so don't interfere with reading tests
        elif identifier == self.REQUEST_IDENTIFIER_VALID_WRITE:
            # Request for positive response
            # Standard specifies the response payload to be an echo of the data identifier from the request
            payload = data[identifier_start_position:identifier_start_position + identifier_length]
            response_data = self.create_positive_response(service_id, payload)

            # overwrite the underlying memory
            print(f'overwriting self.WRITE_DATA {self.WRITE_DATA} with {request_data}')
            self.WRITE_DATA = request_data
            print(f'new self.WRITE_DATA={self.WRITE_DATA}')

You can see above when the incoming DID is REQUEST_IDENTIFIER_VALID_WRITE, we return a positive response to the UDS request and update our WRITE_DATA. I've added extra print statements during debugging and indeed they show the value updating, i.e. when writing [3,4,5] to WRITE_DATA, we see that value when printing the value from inside handle_read_data_by_identifier.

However, back in the test_module_uds.py file, we add the test case test_write_did_success_verify and invoke the test like. Success would mean we were able to write to WRITE_DATA and failure means we could not. Note, we don't rely on the UDS Response, we are checking if the data buffer was updated. The reason for this is because I've come across ECUs that respond positively to DID writes but the underlying data does not change.

responses = uds.write_dids(arb_id_request=self.ARB_ID_REQUEST,
                                   arb_id_response=self.ARB_ID_RESPONSE,
                                   timeout=timeout,
                                   data = data,
                                   min_did=min_did,
                                   max_did=max_did,
                                   verify=verify,
                                   print_results=print_results)

        # make sure we get a positive response
        # response looks like (identifier, [0x6E, DID_UPPER, DID_LOWER])
        print(f'after={self.ecu.WRITE_DATA}')
        # <snip>
        self.assertEqual(self.ecu.WRITE_DATA, data)

We check that the underlying data actually changed and the assertion fails every time. Indeed, the self.ecu.WRITE_DATA from the test_write_did_success point-of-view indicates the WRITE_DATA buffer still has the original data [0,1,2]. I don't understand why the WRITE_DID attribute is not updating. As far as I can tell, the mock ECU has not been torn down since the test is still on going, but it seems like there is some type of disconnect with the class instance we update in the message handler and the class instance that's accessible from the test harness. This is where I need some help - i'm not that familiar with unit testing and I think something funky maybe be going on.

The test results are below (with some extra logging for debugging enabled)

test_write_did_success_skipverify (test_module_uds.UdsModuleTestCase.test_write_did_success_skipverify) ... Testing DIDs in range 0xdead-0xdead

overwriting self.WRITE_DATA [0, 1, 2] with [6, 7, 8]
new self.WRITE_DATA=[6, 7, 8]
Received positive response after writing 060708 to DID 0xdead

Done!

FAIL
test_write_did_success_verify (test_module_uds.UdsModuleTestCase.test_write_did_success_verify) ... before=[0, 1, 2]
Testing DIDs in range 0xdead-0xdead

overwriting self.WRITE_DATA [0, 1, 2] with [3, 4, 5]
new self.WRITE_DATA=[3, 4, 5]
Sucessfully wrote 030405 to DID 0xdead
Done!

after=[0, 1, 2]
FAIL
test_parse_candump_line (test_send.SendFileParserTestCase.test_parse_candump_line) ... ok
test_parse_pythoncan_line_v_20 (test_send.SendFileParserTestCase.test_parse_pythoncan_line_v_20) ... ok
test_parse_pythoncan_line_v_21 (test_send.SendFileParserTestCase.test_parse_pythoncan_line_v_21) ... ok
test_parse_pythoncan_line_v_21_flags (test_send.SendFileParserTestCase.test_parse_pythoncan_line_v_21_flags) ... ok
test_parse_pythoncan_line_v_30_channel (test_send.SendFileParserTestCase.test_parse_pythoncan_line_v_30_channel) ... ok
test_parse_pythoncan_line_v_30_flags (test_send.SendFileParserTestCase.test_parse_pythoncan_line_v_30_flags) ... ok

======================================================================
FAIL: test_write_did_success_skipverify (test_module_uds.UdsModuleTestCase.test_write_did_success_skipverify)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atrain/.local/lib/python3.11/site-packages/caringcaribou-0.7-py3.11.egg/caringcaribou/tests/test_module_uds.py", line 311, in test_write_did_success_skipverify
    self.assertEqual(self.ecu.WRITE_DATA, data)
AssertionError: [0, 1, 2] != b'\x06\x07\x08'

======================================================================
FAIL: test_write_did_success_verify (test_module_uds.UdsModuleTestCase.test_write_did_success_verify)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atrain/.local/lib/python3.11/site-packages/caringcaribou-0.7-py3.11.egg/caringcaribou/tests/test_module_uds.py", line 281, in test_write_did_success_verify
    self.assertEqual(self.ecu.WRITE_DATA, data)
AssertionError: [0, 1, 2] != b'\x03\x04\x05'

----------------------------------------------------------------------
Ran 34 tests in 2.032s

FAILED (failures=2)

[kasperkarlsson]

Hi there! If you send a PR covering only this specific added feature, I will have a look and try to find whatever causes the unit tests to fail 👍

[alexdetrano]

thanks kasper, i'll submit that today!