Minor changes to make it works with bleak 0.6.4

[fengxie]

For some reason, I randomly hit following error and no clue how it happens.

From error message, I guess it's because of some random events that old version of bleak can't handle properly. I made some minor changes to use the latest bleak and it works well. As my knowledge of entire code base is limited, I'm not sure if it works well for other part of the code. Attached is diff.

Traceback (most recent call last):
  File "technic_4x4.py", line 38, in <module>
    start(system)
  File "/home/fxie/dev/venv/lego/lib/python3.6/site-packages/bricknil-0.9.3-py3.6.egg/bricknil/bricknil.py", line 214, in start
  File "/home/fxie/dev/venv/lego/lib/python3.6/site-packages/bricknil-0.9.3-py3.6.egg/bricknil/bleak_interface.py", line 52, in run
  File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
    return future.result()
  File "/home/fxie/dev/venv/lego/lib/python3.6/site-packages/bricknil-0.9.3-py3.6.egg/bricknil/bleak_interface.py", line 65, in asyncio_loop
  File "/home/fxie/dev/venv/lego/lib/python3.6/site-packages/bricknil_bleak-0.3.1-py3.6.egg/bleak/backends/bluezdbus/discovery.py", line 167, in discover
    returnSignature='a{sv}').asFuture(loop)
txdbus.error.RemoteError: org.freedesktop.DBus.Error.UnknownObject: Method "GetAll" with signature "s" on interface "org.freedesktop.DBus.Properties" doesn't exist

Diffs

diff --git a/bricknil/ble_queue.py b/bricknil/ble_queue.py
index b32179a..0e85d20 100644
--- a/bricknil/ble_queue.py
+++ b/bricknil/ble_queue.py
@@ -1,11 +1,11 @@
-# Copyright 2019 Virantha N. Ekanayake
-#
+# Copyright 2019 Virantha N. Ekanayake
+#
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
-#
+#
 # http://www.apache.org/licenses/LICENSE-2.0
-#
+#
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -117,9 +117,9 @@ class BLEventQ(Process):

     def _check_devices_for(self, devices, name, manufacturer_id, address):
         """Check if any of the devices match what we're looking for
-
+
            First, check to make sure the manufacturer_id matches.  If the
-           manufacturer_id is not present in the BLE advertised data from the
+           manufacturer_id is not present in the BLE advertised data from the
            device, then fall back to the name (although this is unreliable because
            the name on the device can be changed by the user through the LEGO apps).

@@ -168,11 +168,11 @@ class BLEventQ(Process):
                     devices = await self.ble.out_queue.get() # Wait for discovered devices
                     await self.ble.out_queue.task_done()
                     # Filter out no-matching uuid
-                    devices = [d for d in devices if str(uart_uuid) in d.uuids]
+                    devices = [d for d in devices if str(uart_uuid) in d.metadata['uuids']]
                     # NOw, extract the manufacturer_id
                     for device in devices:
-                        assert len(device.manufacturer_data) == 1
-                        data = next(iter(device.manufacturer_data.values())) # Get the one and only key
+                        assert len(device.metadata['manufacturer_data']) == 1
+                        data = next(iter(device.metadata['manufacturer_data'].values())) # Get the one and only key
                         device.manufacturer_id = data[1]
                 else:
                     devices = self.ble.find_devices(service_uuids=[uart_uuid])
@@ -221,7 +221,7 @@ class BLEventQ(Process):
         try:
             ble_id = uuid.UUID(hub.ble_id) if hub.ble_id else None
         except ValueError:
-            # In case the user passed in a
+            # In case the user passed in a
             self.message_info(f"ble_id {hub.ble_id} is not a parseable UUID, so assuming it's a BLE network addresss")
             ble_id = hub.ble_id

@@ -234,7 +234,7 @@ class BLEventQ(Process):
             device = await self.ble.out_queue.get()
             await self.ble.out_queue.task_done()
             hub.ble_id = self.device.address
-            self.message_info(f'Device advertised: {device.characteristics}')
+            self.message_info(f'Device advertised: {device.services.characteristics}')
             hub.tx = (device, hub.char_uuid)   # Need to store device because the char is not an object in Bleak, unlike Bluefruit library
             # Hack to fix device name on Windows
             if self.device.name == "Unknown" and hasattr(device._requester, 'Name'):
diff --git a/setup.py b/setup.py
index 411f139..df31a47 100644
--- a/setup.py
+++ b/setup.py
@@ -80,7 +80,7 @@ setup (
     include_package_data = True,
     packages = packages,
     install_requires = required + ['pyobjc ; sys.platform == "darwin"',
-                                   'bricknil-bleak ; sys.platform != "darwin"'],
+                                   'bleak ; sys.platform != "darwin"'],
     dependency_links = dependency_links,
     entry_points = {
             'console_scripts': [

[esklarski]

I was encountering the same issue and just started poking about when I saw this. I took the liberty of creating a fork to implement and test these changes. See: https://github.com/esklarski/bricknil

I was picking through the two versions and noticed that the current version of BLEAK has a lot more error catching than the bricknil version, and I suspect the errors we are getting are for things not handled at the time of the fork.

Thanks for starting this, i will be doing testing as I have free time.

[esklarski]

Found the first issue, seems there is a problem unwrapping values from a list returned by a sensor.

File ".../bricknil/sensor/sensor.py", line 534, in update_value if self.value[ss] & (1<<15): # negative sign bit TypeError: unsupported operand type(s) for &: 'list' and 'int'

I am just playing with the DuploTrain example (link).

I can fix the problem by eliminating one of the values from the attach statement.

Does not work, and produces the above error:
@attach(DuploSpeedSensor, name='speed_sensor', capabilities=['sense_speed', 'sense_count'])

Works fine with no error:
@attach(DuploSpeedSensor, name='speed_sensor', capabilities=['sense_speed'])

[esklarski]

@virantha if you're about still, what modifications were necessary when you forked Bleak originally?

[LasseD]

I am getting the same error from both @virantha and your branch @esklarski . This is observed in MacOS 10.14.6 when trying to run vernie_remote.py, and upgrading to bleak 0.7.1 results in the same issue:

inside curio run loop
INFO:BLE Event Q.0:Clearing BLE cache data
INFO:BLE Event Q.0:Found adapter Default Adapter
INFO:BLE Event Q.0:Powering up adapter Default Adapter
INFO:BLE Event Q.0:Starting scan for UART 00001623-1212-efde-1623-785feabcd123
INFO:BLE Event Q.0:Looking for first matching hub
INFO:BLE Event Q.0:Rescanning for 00001623-1212-efde-1623-785feabcd123 (60 tries left)
INFO:BLE Event Q.0:Rescanning for 00001623-1212-efde-1623-785feabcd123 (59 tries left)
INFO:BLE Event Q.0:Rescanning for 00001623-1212-efde-1623-785feabcd123 (58 tries left)
INFO:BLE Event Q.0:Rescanning for 00001623-1212-efde-1623-785feabcd123 (57 tries left)
INFO:BLE Event Q.0:Rescanning for 00001623-1212-efde-1623-785feabcd123 (56 tries left)
INFO:BLE Event Q.0:Rescanning for 00001623-1212-efde-1623-785feabcd123 (55 tries left)
INFO:BLE Event Q.0:Rescanning for 00001623-1212-efde-1623-785feabcd123 (54 tries left)
2020-07-03 17:44:45.104 Python[14025:2500771] *** Terminating app due to uncaught exception 'OC_PythonException', reason: '<class 'AssertionError'>: <memory at 0x102956b80>'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff3cf59acd __exceptionPreprocess + 256
1 libobjc.A.dylib 0x00007fff6765da17 objc_exception_throw + 48
2 CoreFoundation 0x00007fff3cf73629 -[NSException raise] + 9
3 _objc.cpython-38-darwin.so 0x00000001028c78fe PyObjCErr_ToObjCWithGILState + 46
4 _objc.cpython-38-darwin.so 0x00000001028bfc64 -[OC_PythonObject forwardInvocation:] + 708
5 CoreFoundation 0x00007fff3cefb67e forwarding + 780
6 CoreFoundation 0x00007fff3cefb2e8 _CF_forwarding_prep_0 + 120
7 CoreBluetooth 0x00007fff3c9cd46a -[CBCentralManager handlePeripheralDiscovered:] + 1156
8 CoreBluetooth 0x00007fff3c9ce4ba -[CBCentralManager handleMsg:args:] + 536
9 CoreBluetooth 0x00007fff3c9c97db __30-[CBXpcConnection _handleMsg:]_block_invoke + 53
10 libdispatch.dylib 0x00007fff68dde5f8 _dispatch_call_block_and_release + 12
11 libdispatch.dylib 0x00007fff68ddf63d _dispatch_client_callout + 8
12 libdispatch.dylib 0x00007fff68de58e0 _dispatch_lane_serial_drain + 602
13 libdispatch.dylib 0x00007fff68de63c6 _dispatch_lane_invoke + 433
14 libdispatch.dylib 0x00007fff68dea54b _dispatch_main_queue_callback_4CF + 813
15 CoreFoundation 0x00007fff3cea3a87 CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE + 9
16 CoreFoundation 0x00007fff3cea31b1 __CFRunLoopRun + 2289
17 CoreFoundation 0x00007fff3cea266e CFRunLoopRunSpecific + 455
18 Foundation 0x00007fff3f1082ff -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 280
19 _objc.cpython-38-darwin.so 0x00000001028c6627 ffi_call_unix64 + 79
20 ??? 0x000000010451be30 0x0 + 4367433264
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6

[esklarski]

@janvrany has a better solution in the works. See:
#9
or:
https://github.com/janvrany/bricknil/tree/devel

I've tried the devel branch and it seems to work well.

[fengxie]

@janvrany has a better solution in the works. See:
#9
or:
https://github.com/janvrany/bricknil/tree/devel

I've tried the devel branch and it seems to work well.

Thanks. I tried this branch and it works well to output control signals. But it seems not returning input correctly.

I'm reading sensor like this. I expect it returns value of position and speed but output it [None].

        print("DEBUG", self.rear_drive.value)

DEBUG: Rear [None]
DEBUG {<capability.sense_speed: 1>: [None], <capability.sense_pos: 2>: [None]}

#!/usr/bin/env python3

import logging

# from curio import sleep
from asyncio import sleep
from bricknil import attach, start
from bricknil.hub import CPlusHub
from bricknil.sensor.motor import CPlusXLMotor, CPlusLargeMotor, TachoMotor


@attach(CPlusLargeMotor, name='front_drive', capabilities=['sense_speed', ('sense_pos', 1)], port=1)
@attach(CPlusXLMotor, name='rear_drive', capabilities=['sense_speed', 'sense_pos'], port=3)
class Truck(CPlusHub):
    def __init__(self, name, query_port_info=False, ble_id=None):
        super().__init__(name, query_port_info, ble_id)
        self.set_rear_speed = 100

    async def front_drive_change(self):
        front_speed = self.front_drive.value[TachoMotor.capability.sense_speed]
        front_pos = self.front_drive.value[TachoMotor.capability.sense_pos]

    async def rear_drive_change(self):
        rear_speed = self.rear_drive.value[TachoMotor.capability.sense_speed]

    async def run(self):
        self.message_info("Running")
        await self.rear_drive.activate_updates()
        await sleep(5) # Give it enough time to gather data

        await self.rear_drive.set_pos(0)
        print("DEBUG", self.rear_drive.value)


async def system():
    hub = Truck('truck', True)

if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    start(system)

Is there any idea about this?

My bleak version == 0.7.1

λ uname -a
Linux jetson-nano 4.9.140-tegra #1 SMP PREEMPT Wed Apr 8 18:10:49 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux

[fengxie]

@janvrany has a better solution in the works. See:
#9
or:
https://github.com/janvrany/bricknil/tree/devel
I've tried the devel branch and it seems to work well.

Thanks. I tried this branch and it works well to output control signals. But it seems not returning input correctly.

I'm reading sensor like this. I expect it returns value of position and speed but output it [None].

        print("DEBUG", self.rear_drive.value)

DEBUG: Rear [None]
DEBUG {<capability.sense_speed: 1>: [None], <capability.sense_pos: 2>: [None]}

#!/usr/bin/env python3

import logging

# from curio import sleep
from asyncio import sleep
from bricknil import attach, start
from bricknil.hub import CPlusHub
from bricknil.sensor.motor import CPlusXLMotor, CPlusLargeMotor, TachoMotor


@attach(CPlusLargeMotor, name='front_drive', capabilities=['sense_speed', ('sense_pos', 1)], port=1)
@attach(CPlusXLMotor, name='rear_drive', capabilities=['sense_speed', 'sense_pos'], port=3)
class Truck(CPlusHub):
    def __init__(self, name, query_port_info=False, ble_id=None):
        super().__init__(name, query_port_info, ble_id)
        self.set_rear_speed = 100

    async def front_drive_change(self):
        front_speed = self.front_drive.value[TachoMotor.capability.sense_speed]
        front_pos = self.front_drive.value[TachoMotor.capability.sense_pos]

    async def rear_drive_change(self):
        rear_speed = self.rear_drive.value[TachoMotor.capability.sense_speed]

    async def run(self):
        self.message_info("Running")
        await self.rear_drive.activate_updates()
        await sleep(5) # Give it enough time to gather data

        await self.rear_drive.set_pos(0)
        print("DEBUG", self.rear_drive.value)


async def system():
    hub = Truck('truck', True)

if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    start(system)

Is there any idea about this?

My bleak version == 0.7.1

λ uname -a
Linux jetson-nano 4.9.140-tegra #1 SMP PREEMPT Wed Apr 8 18:10:49 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux

I figured out what's wrong. Seems exception inside async routines are ignored. If they failed then it doesn't produce meaningful value.

[esklarski]

What was causing the exceptions?

[fengxie]

What was causing the exceptions?

My code was bad as capability is no longer inherit from TachoMotor.

    async def front_drive_change(self):
        front_speed = self.front_drive.value[TachoMotor.capability.sense_speed]
        front_pos = self.front_drive.value[TachoMotor.capability.sense_pos]

This triggered exception. Didn't figure out where it is handled. But the result is invalid value read back and the process seems dead.