Extending BlueZ's HAL protocol

Extending BlueZ's HAL protocol

[Thomas Zimmermann]

Hello,

On Mozilla's Firefox OS, we use a daemon for running Bluedroid drivers
in their own address space, and communicate with the daemon via BlueZ's
HAL protocol. [1]

We'd like to use the HAL protocol as-is, but several commands are not
defined in the spec. We especially ran into this problem with
Bluedroid's |bt_interface_t::config_hci_snoop_log|. BlueZ has a
completely different mechanism for handling the command and probably
won't ever require an opcode for it. Just adding an opcode for
|config_hci_snoop_log| to our code base will probably collide with BlueZ
official spec in the long run.

Our proposal is to reserve two opcodes per service for extensions to
commands and notifications. This would allow us to add the missing
pieces without opcode collisions.

Best regards
Thomas

[1]
https://git.kernel.org/cgit/bluetooth/bluez.git/tree/android/hal-ipc-api.txt

Re: Extending BlueZ's HAL protocol

[Marcel Holtmann]

Hi Thomas,

> On Mozilla's Firefox OS, we use a daemon for running Bluedroid drivers
> in their own address space, and communicate with the daemon via BlueZ's
> HAL protocol. [1]
> 
> We'd like to use the HAL protocol as-is, but several commands are not
> defined in the spec. We especially ran into this problem with
> Bluedroid's |bt_interface_t::config_hci_snoop_log|. BlueZ has a
> completely different mechanism for handling the command and probably
> won't ever require an opcode for it. Just adding an opcode for
> |config_hci_snoop_log| to our code base will probably collide with BlueZ
> official spec in the long run.
> 
> Our proposal is to reserve two opcodes per service for extensions to
> commands and notifications. This would allow us to add the missing
> pieces without opcode collisions.

when you are using BlueZ for Android, we run the bluetoothd-snoop daemon to collect the traces. The reason why it is not in bluetoothd that provides all the other functionality is pretty simple. We do not have to and that is why we never bothered to define an opcode for it. That is the advantage of not being a huge monolithic block like Bluedroid.

We rely on the Bluetooth subsystem and its monitor (snooping) functionality. The big advantage having this in a separate process is that it will not interfere with any other operation bluetoothd has to do. The kernel can schedule bluetoothd and bluetoothd-snoop individually.

So bluetoothd-snoop is essentially a mini version of btmon where its only feature is to store traces in the old BTSnoop format (Frontline compatible). That said, nothing is stopping you from actually running btmon or hcidump on the target platform or multiple of them. We could even write a USB gadget driver that exports the traces over USB and you could run btmon / Wireshark on a PC to collect them.

Can you just run bluetoothd-snoop as well and get the tracing feature that way? If you build BlueZ for Android and run it that way, that is what happens through the developer menu. It is fully integrated into the Android UI and we have been running it that way for a long time now.

Regards

Marcel

Re: Extending BlueZ's HAL protocol

[Thomas Zimmermann]

Hi Marcel

Am 30.01.2015 um 18:59 schrieb Marcel Holtmann:
> Hi Thomas,
>
>> On Mozilla's Firefox OS, we use a daemon for running Bluedroid drivers
>> in their own address space, and communicate with the daemon via BlueZ's
>> HAL protocol. [1]
>>
>> We'd like to use the HAL protocol as-is, but several commands are not
>> defined in the spec. We especially ran into this problem with
>> Bluedroid's |bt_interface_t::config_hci_snoop_log|. BlueZ has a
>> completely different mechanism for handling the command and probably
>> won't ever require an opcode for it. Just adding an opcode for
>> |config_hci_snoop_log| to our code base will probably collide with BlueZ
>> official spec in the long run.
>>
>> Our proposal is to reserve two opcodes per service for extensions to
>> commands and notifications. This would allow us to add the missing
>> pieces without opcode collisions.
> when you are using BlueZ for Android, we run the bluetoothd-snoop daemon to collect the traces. The reason why it is not in bluetoothd that provides all the other functionality is pretty simple. We do not have to and that is why we never bothered to define an opcode for it. That is the advantage of not being a huge monolithic block like Bluedroid.

It's not that easy in our case, I guess. We still run Bluedroid, but
wrapped it in it's own daemon and use BlueZ HAL protocol for
communication. We try to be compatible to BlueZ, so that we can replace
Bluedroid for BlueZ easily. However, we cannot drop Bluedroid, because
that's what Android chipset vendors support.

I think we somehow have to make our Bluedroid daemon call
|config_hci_snoop_log|. One idea is the proposed opcode for vendor
extensions (Mozilla would be the vendor in our case). The other idea
would be to write our own |bluetooth-snoop|, which uses a different
channel to instruct the Bluedroid daemon to call |config_hci_snoop_log|.

Best regards
Thomas

> We rely on the Bluetooth subsystem and its monitor (snooping) functionality. The big advantage having this in a separate process is that it will not interfere with any other operation bluetoothd has to do. The kernel can schedule bluetoothd and bluetoothd-snoop individually.
>
> So bluetoothd-snoop is essentially a mini version of btmon where its only feature is to store traces in the old BTSnoop format (Frontline compatible). That said, nothing is stopping you from actually running btmon or hcidump on the target platform or multiple of them. We could even write a USB gadget driver that exports the traces over USB and you could run btmon / Wireshark on a PC to collect them.
>
> Can you just run bluetoothd-snoop as well and get the tracing feature that way? If you build BlueZ for Android and run it that way, that is what happens through the developer menu. It is fully integrated into the Android UI and we have been running it that way for a long time now.
>
> Regards
>
> Marcel
>

Re: Extending BlueZ's HAL protocol

[Marcel Holtmann]

Hi Thomas,

>>> On Mozilla's Firefox OS, we use a daemon for running Bluedroid drivers
>>> in their own address space, and communicate with the daemon via BlueZ's
>>> HAL protocol. [1]
>>> 
>>> We'd like to use the HAL protocol as-is, but several commands are not
>>> defined in the spec. We especially ran into this problem with
>>> Bluedroid's |bt_interface_t::config_hci_snoop_log|. BlueZ has a
>>> completely different mechanism for handling the command and probably
>>> won't ever require an opcode for it. Just adding an opcode for
>>> |config_hci_snoop_log| to our code base will probably collide with BlueZ
>>> official spec in the long run.
>>> 
>>> Our proposal is to reserve two opcodes per service for extensions to
>>> commands and notifications. This would allow us to add the missing
>>> pieces without opcode collisions.
>> when you are using BlueZ for Android, we run the bluetoothd-snoop daemon to collect the traces. The reason why it is not in bluetoothd that provides all the other functionality is pretty simple. We do not have to and that is why we never bothered to define an opcode for it. That is the advantage of not being a huge monolithic block like Bluedroid.
> 
> It's not that easy in our case, I guess. We still run Bluedroid, but
> wrapped it in it's own daemon and use BlueZ HAL protocol for
> communication. We try to be compatible to BlueZ, so that we can replace
> Bluedroid for BlueZ easily. However, we cannot drop Bluedroid, because
> that's what Android chipset vendors support.

can you tell me what's your chipset manufacturer? I mean there is almost no chance that you are running on a Bluetooth chip that can not be easily supported with BlueZ. All of them talk HCI and the only difference is some minimal firmware loading, UART baudrate setting or other tiny details. Supporting a Bluetooth chip with BlueZ is most likely a lot easier than with Bluedroid.

You also realize that you can run Bluedroid on top of Bluetooth HCI User Channel feature in the kernel. That way you use the kernel side HCI infrastructure and existing drivers. At that point tools like hcidump, btmon and even hcitool will just work as usual. It is a pretty nifty trick actually. There is a libbt-vendor from Intel that will allow you to run Bluedroid on top of Bluetooth kernel subsystem using HCI User Channel. It is only a few hundred lines of code.

Or have you actually tried to use the vendor provided libbt-vendor and hook it into BlueZ via /dev/vhci. That should actually work easily. So like a tiny libhybris for Bluetooth. I say tiny since all Bluetooth chips talk HCI and it will be pretty much super simple.

> I think we somehow have to make our Bluedroid daemon call
> |config_hci_snoop_log|. One idea is the proposed opcode for vendor
> extensions (Mozilla would be the vendor in our case). The other idea
> would be to write our own |bluetooth-snoop|, which uses a different
> channel to instruct the Bluedroid daemon to call |config_hci_snoop_log|.

We could potentially add such an opcode that does emulate this piece of HAL detail. We did not add it since for us running HCI tracing / snooping in the same core process was never a viable option.

If you would try to use HCI User Channel or wrap libbt-vendor into a /dev/vhci as mentioned above, then you would actually not have to and could just run bluetoothd-snoop like we do.

Regards

Marcel

Re: Extending BlueZ's HAL protocol

[Thomas Zimmermann]

Hi Marcel

> can you tell me what's your chipset manufacturer? I mean there is almost no chance that you are running on a Bluetooth chip that can not be easily supported with BlueZ. All of them talk HCI and the only difference is some minimal firmware loading, UART baudrate setting or other tiny details. Supporting a Bluetooth chip with BlueZ is most likely a lot easier than with Bluedroid.

Usually Qualcomm, but we try to support any chipset. Using BlueZ is not
just a technical issue though. I'd guess that there are possibly other
requirements (legal, partners, etc). Our plan is to provide BlueZ as a
drop-in replacement at some later point.

>
> You also realize that you can run Bluedroid on top of Bluetooth HCI User Channel feature in the kernel. That way you use the kernel side HCI infrastructure and existing drivers. At that point tools like hcidump, btmon and even hcitool will just work as usual. It is a pretty nifty trick actually. There is a libbt-vendor from Intel that will allow you to run Bluedroid on top of Bluetooth kernel subsystem using HCI User Channel. It is only a few hundred lines of code.

TBH I wouldn't rely on devices to have Bluetooth support enabled in the
kernel. My understanding is that Bluedroid is completely independent
from the kernel, so why should OEMs enable the kernel's BT module as well?

>
> Or have you actually tried to use the vendor provided libbt-vendor and hook it into BlueZ via /dev/vhci. That should actually work easily. So like a tiny libhybris for Bluetooth. I say tiny since all Bluetooth chips talk HCI and it will be pretty much super simple.
>
>> I think we somehow have to make our Bluedroid daemon call
>> |config_hci_snoop_log|. One idea is the proposed opcode for vendor
>> extensions (Mozilla would be the vendor in our case). The other idea
>> would be to write our own |bluetooth-snoop|, which uses a different
>> channel to instruct the Bluedroid daemon to call |config_hci_snoop_log|.
> We could potentially add such an opcode that does emulate this piece of HAL detail. We did not add it since for us running HCI tracing / snooping in the same core process was never a viable option.
>
> If you would try to use HCI User Channel or wrap libbt-vendor into a /dev/vhci as mentioned above, then you would actually not have to and could just run bluetoothd-snoop like we do.

Thanks so far for all the information you provided. I think we'll have
to explore the possible options for the best solution; ideally something
that's compatible with BlueZ's daemon.

Best regards
Thomas

>
> Regards
>
> Marcel
>

Re: Extending BlueZ's HAL protocol

[Luiz Augusto von Dentz]

Hi Thomas,

On Fri, Feb 6, 2015 at 2:04 PM, Thomas Zimmermann
<tzimmermann@mozilla.com> wrote:
> Hi Marcel
>
>> can you tell me what's your chipset manufacturer? I mean there is almost=
 no chance that you are running on a Bluetooth chip that can not be easily =
supported with BlueZ. All of them talk HCI and the only difference is some =
minimal firmware loading, UART baudrate setting or other tiny details. Supp=
orting a Bluetooth chip with BlueZ is most likely a lot easier than with Bl=
uedroid.
>
> Usually Qualcomm, but we try to support any chipset. Using BlueZ is not
> just a technical issue though. I'd guess that there are possibly other
> requirements (legal, partners, etc). Our plan is to provide BlueZ as a
> drop-in replacement at some later point.
>
>>
>> You also realize that you can run Bluedroid on top of Bluetooth HCI User=
 Channel feature in the kernel. That way you use the kernel side HCI infras=
tructure and existing drivers. At that point tools like hcidump, btmon and =
even hcitool will just work as usual. It is a pretty nifty trick actually. =
There is a libbt-vendor from Intel that will allow you to run Bluedroid on =
top of Bluetooth kernel subsystem using HCI User Channel. It is only a few =
hundred lines of code.
>
> TBH I wouldn't rely on devices to have Bluetooth support enabled in the
> kernel. My understanding is that Bluedroid is completely independent
> from the kernel, so why should OEMs enable the kernel's BT module as well=
?

Only, and really only, in the context of Android, many other Linux
based OSes, which includes Chrome OS, Tizen, Ubuntu, etc, are using
the Bluetooth kernel modules so chances are that there is already some
support in the kernel. I must say that if you stay with Bluedroid's
camp you are just contributing to the fragmentation of Bluetooth
hardware adaptation in Linux, after this why would anyone here bother
helping you? Marcel already pointed out what are the benefits of using
the User Channel, together with the idea of supporting any Linux based
platform, regardless what you run on userspace, should already be
enough arguments to convince any OEM.

>>
>> Or have you actually tried to use the vendor provided libbt-vendor and h=
ook it into BlueZ via /dev/vhci. That should actually work easily. So like =
a tiny libhybris for Bluetooth. I say tiny since all Bluetooth chips talk H=
CI and it will be pretty much super simple.
>>
>>> I think we somehow have to make our Bluedroid daemon call
>>> |config_hci_snoop_log|. One idea is the proposed opcode for vendor
>>> extensions (Mozilla would be the vendor in our case). The other idea
>>> would be to write our own |bluetooth-snoop|, which uses a different
>>> channel to instruct the Bluedroid daemon to call |config_hci_snoop_log|=
.
>> We could potentially add such an opcode that does emulate this piece of =
HAL detail. We did not add it since for us running HCI tracing / snooping i=
n the same core process was never a viable option.
>>
>> If you would try to use HCI User Channel or wrap libbt-vendor into a /de=
v/vhci as mentioned above, then you would actually not have to and could ju=
st run bluetoothd-snoop like we do.
>
> Thanks so far for all the information you provided. I think we'll have
> to explore the possible options for the best solution; ideally something
> that's compatible with BlueZ's daemon.
>
> Best regards
> Thomas
>
>>
>> Regards
>>
>> Marcel
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bluetooth=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



--=20
Luiz Augusto von Dentz