[PATCH] doc: Add experimental handsfree-audio API

[PATCH] doc: Add experimental handsfree-audio API

[Mikel Astiz]

[-- Attachment #1: Type: text/plain, Size: 3306 bytes --]

From: Denis Kenzior <denkenz@gmail.com>

---
Either I have a problem with my subscription or this patch was never submitted to the mailing-list.

Not only it helps follow Claudio's patchset, but it's in general worth discussion such API publicly.
 doc/handsfree-audio-api.txt | 96 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 96 insertions(+)
 create mode 100644 doc/handsfree-audio-api.txt

diff --git a/doc/handsfree-audio-api.txt b/doc/handsfree-audio-api.txt
new file mode 100644
index 0000000..78ba9e1
--- /dev/null
+++ b/doc/handsfree-audio-api.txt
@@ -0,0 +1,96 @@
+Handsfree Audio Manager hierarchy [experimental]
+=================================
+
+Service		org.ofono
+Interface	org.ofono.HandsfreeAudioManager
+Object path	/
+
+Methods		array{object,dict} GetCards()
+
+			Get an array of card objects and properties
+			that represent the currently attached devices.
+
+			This method call should only be used once when an
+			application starts up. Further device additions
+			and removal shall be monitored via CardAdded and
+			CardRemoved signals.
+
+		void Register(object path, array{byte})
+
+			Registers a Handsfree Audio agent with a specific
+			path (freely selectable by the audio subsystem) and
+			list of supported codecs.  Available codec
+			identifiers:
+
+				1	CVSD
+				2	mSBC
+
+		void Unregister(object path)
+
+			Unregisters a Handsfree Audio agent registered
+			through the Register method.
+
+Signals		CardAdded(object path, dict properties)
+
+			Signal that is sent when a new card is added.  It
+			contains the object path of new card and its
+			properties.
+
+		CardRemoved(object path)
+
+			Signal that is sent when a card has been removed.
+			The object path is no longer accessible after this
+			signal and only emitted for reference.
+
+
+Handsfree Audio Card hierarchy [experimental]
+==============================
+
+Service		org.ofono
+Interface	org.ofono.HandsfreeAudioCard
+Object path	/{device0,device1,...}
+
+Methods		dict GetProperties()
+
+			Returns properties for the device object. See
+			the properties section for available properties.
+
+		void Connect()
+
+			Attempts to establish the SCO audio connection.
+			The Agent NewConnection() method will be called
+			whenever the SCO audio link has been established.  If
+			the audio connection could not be established, this
+			method will return an error.
+
+Signals		PropertyChanged(string name, variant value)
+
+			This signal indicates a changed value of the given
+			property.
+
+Properties	string RemoteAddress [readonly]
+
+			Bluetooth address of the remote peer.
+
+		string LocalAddress [readonly]
+
+			Bluetooth address of the local adapter.
+
+
+Handsfree Audio Agent hierarchy [experimental]
+===============================
+
+Service		<freely defined>
+Interface	org.ofono.HandsfreeAudioAgent
+Object		<freely defined>
+
+Methods		void NewConnection(object card, fd sco, byte codec)
+
+			Notifies the handler that a new SCO connection is
+			available. Returning an error will cause oFono to
+			disconnect the SCO connection.
+
+		void Release()
+
+			Notifies the Agent that it is no longer registered
+			to oFono.
-- 
1.8.1

Re: [PATCH] doc: Add experimental handsfree-audio API

[Mikel Astiz]

[-- Attachment #1: Type: text/plain, Size: 5985 bytes --]

Hi Denis,

On Wed, Feb 20, 2013 at 9:04 AM, Mikel Astiz <mikel.astiz.oss@gmail.com> wrote:
> From: Denis Kenzior <denkenz@gmail.com>
>
> ---
> Either I have a problem with my subscription or this patch was never submitted to the mailing-list.
>
> Not only it helps follow Claudio's patchset, but it's in general worth discussion such API publicly.
>  doc/handsfree-audio-api.txt | 96 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 96 insertions(+)
>  create mode 100644 doc/handsfree-audio-api.txt
>
> diff --git a/doc/handsfree-audio-api.txt b/doc/handsfree-audio-api.txt
> new file mode 100644
> index 0000000..78ba9e1
> --- /dev/null
> +++ b/doc/handsfree-audio-api.txt
> @@ -0,0 +1,96 @@
> +Handsfree Audio Manager hierarchy [experimental]
> +=================================
> +
> +Service                org.ofono
> +Interface      org.ofono.HandsfreeAudioManager
> +Object path    /
> +
> +Methods                array{object,dict} GetCards()
> +
> +                       Get an array of card objects and properties
> +                       that represent the currently attached devices.
> +
> +                       This method call should only be used once when an
> +                       application starts up. Further device additions
> +                       and removal shall be monitored via CardAdded and
> +                       CardRemoved signals.
> +
> +               void Register(object path, array{byte})
> +
> +                       Registers a Handsfree Audio agent with a specific
> +                       path (freely selectable by the audio subsystem) and
> +                       list of supported codecs.  Available codec
> +                       identifiers:
> +
> +                               1       CVSD
> +                               2       mSBC
> +
> +               void Unregister(object path)
> +
> +                       Unregisters a Handsfree Audio agent registered
> +                       through the Register method.
> +
> +Signals                CardAdded(object path, dict properties)
> +
> +                       Signal that is sent when a new card is added.  It
> +                       contains the object path of new card and its
> +                       properties.
> +
> +               CardRemoved(object path)
> +
> +                       Signal that is sent when a card has been removed.
> +                       The object path is no longer accessible after this
> +                       signal and only emitted for reference.
> +
> +
> +Handsfree Audio Card hierarchy [experimental]
> +==============================
> +
> +Service                org.ofono
> +Interface      org.ofono.HandsfreeAudioCard
> +Object path    /{device0,device1,...}

I'd propose dropping this interface entirely or at least using the
conventional modem object path, assuming that every HandsfreeAudioCard
would necessarily have an associated modem.

After all, the Audio agent would typically have to match cards with
modems, probably because the associated voicecall status needs to be
known (e.g. if the HS wants to ignore the inband-ringing and instead
use another ring tone).

> +
> +Methods                dict GetProperties()
> +
> +                       Returns properties for the device object. See
> +                       the properties section for available properties.
> +
> +               void Connect()
> +
> +                       Attempts to establish the SCO audio connection.
> +                       The Agent NewConnection() method will be called
> +                       whenever the SCO audio link has been established.  If
> +                       the audio connection could not be established, this
> +                       method will return an error.

Isn't Disconnect() missing here? How would the agent release an SCO?

This method (along with Disconnect(), if needed) could be moved to
org.ofono.HandsfreeAudioManager, dropping HandsfreeAudioCard entirely.

Or alternatively, if you think the interface is actually important,
I'd be in favor of registering it in the same modem object path,
avoiding the CardAdded/CardRemoved signals.

> +
> +Signals                PropertyChanged(string name, variant value)
> +
> +                       This signal indicates a changed value of the given
> +                       property.
> +
> +Properties     string RemoteAddress [readonly]
> +
> +                       Bluetooth address of the remote peer.
> +
> +               string LocalAddress [readonly]
> +
> +                       Bluetooth address of the local adapter.
> +
> +
> +Handsfree Audio Agent hierarchy [experimental]
> +===============================
> +
> +Service                <freely defined>
> +Interface      org.ofono.HandsfreeAudioAgent
> +Object         <freely defined>
> +
> +Methods                void NewConnection(object card, fd sco, byte codec)
> +
> +                       Notifies the handler that a new SCO connection is
> +                       available. Returning an error will cause oFono to
> +                       disconnect the SCO connection.

Same here. How would the agent receive a release-notification? Perhaps
this would be implicit in the socket HUP?

If so, I have some concerns about the potential race conditions. If
SCO was closed and reopen immediately afterwards, the client could
receive the (second) NewConnection before the HUP.

Besides, is this D-Bus API intended to support SCO-over-PCM as well?
If yes, several considerations seems to be missing, specially
regarding the socket fd. In particular, the HUP approach for
disconnections would not be valid anymore.

> +
> +               void Release()
> +
> +                       Notifies the Agent that it is no longer registered
> +                       to oFono.
> --
> 1.8.1
>

Cheers,
Mikel

Re: [PATCH] doc: Add experimental handsfree-audio API

[Frederic Danis]

[-- Attachment #1: Type: text/plain, Size: 1085 bytes --]

Hello Mikel,

On 20/02/2013 09:55, Mikel Astiz wrote:
<snip>

>> +Handsfree Audio Card hierarchy [experimental]
>> +==============================
>> +
>> +Service                org.ofono
>> +Interface      org.ofono.HandsfreeAudioCard
>> +Object path    /{device0,device1,...}
>
> I'd propose dropping this interface entirely or at least using the
> conventional modem object path, assuming that every HandsfreeAudioCard
> would necessarily have an associated modem.
>
> After all, the Audio agent would typically have to match cards with
> modems, probably because the associated voicecall status needs to be
> known (e.g. if the HS wants to ignore the inband-ringing and instead
> use another ring tone).
>
In case of HFP AG, there could be connections with multiple handsfrees, 
so multiple emulator and HandsFreeAudioCard for a modem.

So, we can not use modem path directly.

Regards

Fred


-- 
Frederic Danis                            Open Source Technology Center
frederic.danis(a)intel.com                              Intel Corporation

Re: [PATCH] doc: Add experimental handsfree-audio API

[Johan Hedberg]

[-- Attachment #1: Type: text/plain, Size: 3175 bytes --]

Hi Mikel,

On Wed, Feb 20, 2013, Mikel Astiz wrote:
> > +Handsfree Audio Card hierarchy [experimental]
> > +==============================
> > +
> > +Service                org.ofono
> > +Interface      org.ofono.HandsfreeAudioCard
> > +Object path    /{device0,device1,...}
> 
> I'd propose dropping this interface entirely or at least using the
> conventional modem object path, assuming that every HandsfreeAudioCard
> would necessarily have an associated modem.
> 
> After all, the Audio agent would typically have to match cards with
> modems, probably because the associated voicecall status needs to be
> known (e.g. if the HS wants to ignore the inband-ringing and instead
> use another ring tone).

What you describe would only cover the HF role of HFP but not AG. With
AG you can have multiple HF devices connected to you (hence multiple
"cards") while still having just one oFono modem object.

> > +Methods                dict GetProperties()
> > +
> > +                       Returns properties for the device object. See
> > +                       the properties section for available properties.
> > +
> > +               void Connect()
> > +
> > +                       Attempts to establish the SCO audio connection.
> > +                       The Agent NewConnection() method will be called
> > +                       whenever the SCO audio link has been established.  If
> > +                       the audio connection could not be established, this
> > +                       method will return an error.
> 
> Isn't Disconnect() missing here? How would the agent release an SCO?

I think the idea was that the client can just use the shutdown() system
call.

> > +Handsfree Audio Agent hierarchy [experimental]
> > +===============================
> > +
> > +Service                <freely defined>
> > +Interface      org.ofono.HandsfreeAudioAgent
> > +Object         <freely defined>
> > +
> > +Methods                void NewConnection(object card, fd sco, byte codec)
> > +
> > +                       Notifies the handler that a new SCO connection is
> > +                       available. Returning an error will cause oFono to
> > +                       disconnect the SCO connection.
> 
> Same here. How would the agent receive a release-notification? Perhaps
> this would be implicit in the socket HUP?

I think that was the idea (but I'll let Denis clarify if I'm wrong).

> If so, I have some concerns about the potential race conditions. If
> SCO was closed and reopen immediately afterwards, the client could
> receive the (second) NewConnection before the HUP.
> 
> Besides, is this D-Bus API intended to support SCO-over-PCM as well?
> If yes, several considerations seems to be missing, specially
> regarding the socket fd. In particular, the HUP approach for
> disconnections would not be valid anymore.

I don't see why it would not be valid. Even if you do SCO over PCM the
SCO socket will give you the right connection state info since it's
hooked to the same connect and disconnect HCI events which occur
regardless of how you route the SCO data.

Johan

Re: [PATCH] doc: Add experimental handsfree-audio API

[Mikel Astiz]

[-- Attachment #1: Type: text/plain, Size: 4890 bytes --]

Hi Johan,

On Wed, Feb 20, 2013 at 10:19 AM, Johan Hedberg <johan.hedberg@gmail.com> wrote:
> Hi Mikel,
>
> On Wed, Feb 20, 2013, Mikel Astiz wrote:
>> > +Handsfree Audio Card hierarchy [experimental]
>> > +==============================
>> > +
>> > +Service                org.ofono
>> > +Interface      org.ofono.HandsfreeAudioCard
>> > +Object path    /{device0,device1,...}
>>
>> I'd propose dropping this interface entirely or at least using the
>> conventional modem object path, assuming that every HandsfreeAudioCard
>> would necessarily have an associated modem.
>>
>> After all, the Audio agent would typically have to match cards with
>> modems, probably because the associated voicecall status needs to be
>> known (e.g. if the HS wants to ignore the inband-ringing and instead
>> use another ring tone).
>
> What you describe would only cover the HF role of HFP but not AG. With
> AG you can have multiple HF devices connected to you (hence multiple
> "cards") while still having just one oFono modem object.

I see the point now.

However, I still see some value in having a more coupled relation
between modems and audio-cards, beyond the RemoteAddress/LocalAddress
pair. This is specially interesting for the HS role, since otherwise
both modems and audio-cards need to be tracked and matched.

The two roles might actually need to be separated. After all, there
are some fundamental differences: audio-cards representing a remote
gateway would be statically associated to a modem. Actually the modem
*represents* this device, so signals such as CardAdded/CardRemoved
provide little value.

On the other hand, audio-cards representing remote headsets are more
dynamic: they come and go depending on the number of paired headsets
(or more likely, the connected ones - it's not clear what "attached"
means in the doc).

One possibility would be to register HandsfreeAudioCard in the modem
object path for the HS role (as I proposed before) and dynamically
register the necessary objects for the AG role. In the later case, the
connected headsets could be announced as in Denis' original proposal
(perhaps renamed to HeadsetAdded/HeadsetRemoved).

The ability to distinguish HS and AG roles in the client's side is
desirable (at least PulseAudio heavily depends on this information).
So even if the proposal above is not accepted, at least the role
(UUID) would have to be exposed somehow.

>
>> > +Methods                dict GetProperties()
>> > +
>> > +                       Returns properties for the device object. See
>> > +                       the properties section for available properties.
>> > +
>> > +               void Connect()
>> > +
>> > +                       Attempts to establish the SCO audio connection.
>> > +                       The Agent NewConnection() method will be called
>> > +                       whenever the SCO audio link has been established.  If
>> > +                       the audio connection could not be established, this
>> > +                       method will return an error.
>>
>> Isn't Disconnect() missing here? How would the agent release an SCO?
>
> I think the idea was that the client can just use the shutdown() system
> call.
>
>> > +Handsfree Audio Agent hierarchy [experimental]
>> > +===============================
>> > +
>> > +Service                <freely defined>
>> > +Interface      org.ofono.HandsfreeAudioAgent
>> > +Object         <freely defined>
>> > +
>> > +Methods                void NewConnection(object card, fd sco, byte codec)
>> > +
>> > +                       Notifies the handler that a new SCO connection is
>> > +                       available. Returning an error will cause oFono to
>> > +                       disconnect the SCO connection.
>>
>> Same here. How would the agent receive a release-notification? Perhaps
>> this would be implicit in the socket HUP?
>
> I think that was the idea (but I'll let Denis clarify if I'm wrong).
>
>> If so, I have some concerns about the potential race conditions. If
>> SCO was closed and reopen immediately afterwards, the client could
>> receive the (second) NewConnection before the HUP.
>>
>> Besides, is this D-Bus API intended to support SCO-over-PCM as well?
>> If yes, several considerations seems to be missing, specially
>> regarding the socket fd. In particular, the HUP approach for
>> disconnections would not be valid anymore.
>
> I don't see why it would not be valid. Even if you do SCO over PCM the
> SCO socket will give you the right connection state info since it's
> hooked to the same connect and disconnect HCI events which occur
> regardless of how you route the SCO data.

You're right, I thought no socket would exist in this scenario, but
that's not the case. So my point was not valid.

Cheers,
Mikel

Re: [PATCH] doc: Add experimental handsfree-audio API

[Denis Kenzior]

[-- Attachment #1: Type: text/plain, Size: 2382 bytes --]

Hi Mikel,

> However, I still see some value in having a more coupled relation
> between modems and audio-cards, beyond the RemoteAddress/LocalAddress
> pair. This is specially interesting for the HS role, since otherwise
> both modems and audio-cards need to be tracked and matched.
>
> The two roles might actually need to be separated. After all, there
> are some fundamental differences: audio-cards representing a remote
> gateway would be statically associated to a modem. Actually the modem
> *represents* this device, so signals such as CardAdded/CardRemoved
> provide little value.
>

This API is meant specifically for PA or another Audio subsystem.  What 
you want to be tracking and cross-referencing I'm not quite sure of.

> On the other hand, audio-cards representing remote headsets are more
> dynamic: they come and go depending on the number of paired headsets
> (or more likely, the connected ones - it's not clear what "attached"
> means in the doc).

In practice, attached means an established SLC connection.

>
> One possibility would be to register HandsfreeAudioCard in the modem
> object path for the HS role (as I proposed before) and dynamically
> register the necessary objects for the AG role. In the later case, the
> connected headsets could be announced as in Denis' original proposal
> (perhaps renamed to HeadsetAdded/HeadsetRemoved).
>

Perhaps, but why would you be using this API and not one from PA?

> The ability to distinguish HS and AG roles in the client's side is
> desirable (at least PulseAudio heavily depends on this information).
> So even if the proposal above is not accepted, at least the role
> (UUID) would have to be exposed somehow.
>

The type information was left out on purpose from this proposal.  Likely 
it is necessary, but for the initial phase it does not seem relevant.  I 
also have yet to hear concise and clear summary on the utility of 
providing this information.

>>> If so, I have some concerns about the potential race conditions. If
>>> SCO was closed and reopen immediately afterwards, the client could
>>> receive the (second) NewConnection before the HUP.
>>>

You are going over-the-air vs local machine.  I really do not see this 
ever happening.  But even if it does, is this really a hard problem for 
the Agent to deal with?

Regards,
-Denis

Re: [PATCH] doc: Add experimental handsfree-audio API

[Mikel Astiz]

[-- Attachment #1: Type: text/plain, Size: 5061 bytes --]

Hi Denis,

On Wed, Feb 20, 2013 at 4:45 PM, Denis Kenzior <denkenz@gmail.com> wrote:
> Hi Mikel,
>
>
>> However, I still see some value in having a more coupled relation
>> between modems and audio-cards, beyond the RemoteAddress/LocalAddress
>> pair. This is specially interesting for the HS role, since otherwise
>> both modems and audio-cards need to be tracked and matched.
>>
>> The two roles might actually need to be separated. After all, there
>> are some fundamental differences: audio-cards representing a remote
>> gateway would be statically associated to a modem. Actually the modem
>> *represents* this device, so signals such as CardAdded/CardRemoved
>> provide little value.
>>
>
> This API is meant specifically for PA or another Audio subsystem.  What you
> want to be tracking and cross-referencing I'm not quite sure of.

I already mentioned one example: if a headset wants to ignore the
inband ringtone, and use it's own custom one, it needs to match the
audio-card and the voicecall state. During the ringtone (callsetup=1),
the SCO stream would be ignored, as suggested in section 4.13.4 of the
spec.

I could list more examples too if needed, but they would involve
multiple connected AGs.

According to this API, the audio subsystem would have to find the
relevant modem by comparing the remote BT address of the audio-card
with the serial number of the modems. This is not very nice, not to
mention that the serial number doesn't expose the adapter's BT adress.
So I think having an object path to the modem would be preferable, or
as discussed, even registering the interface in the modem's object
path.

>
>
>> On the other hand, audio-cards representing remote headsets are more
>> dynamic: they come and go depending on the number of paired headsets
>> (or more likely, the connected ones - it's not clear what "attached"
>> means in the doc).
>
>
> In practice, attached means an established SLC connection.
>
>
>>
>> One possibility would be to register HandsfreeAudioCard in the modem
>> object path for the HS role (as I proposed before) and dynamically
>> register the necessary objects for the AG role. In the later case, the
>> connected headsets could be announced as in Denis' original proposal
>> (perhaps renamed to HeadsetAdded/HeadsetRemoved).
>>
>
> Perhaps, but why would you be using this API and not one from PA?

I'm not following here. My proposal is concerning how a
client/audio-agent would be implemented, so all my comments above were
trying to improve the API in order to simplify client code.

Not that I agree with the idea of re-engineering the Media API, but anyway...

>
>
>> The ability to distinguish HS and AG roles in the client's side is
>> desirable (at least PulseAudio heavily depends on this information).
>> So even if the proposal above is not accepted, at least the role
>> (UUID) would have to be exposed somehow.
>>
>
> The type information was left out on purpose from this proposal.  Likely it
> is necessary, but for the initial phase it does not seem relevant.  I also
> have yet to hear concise and clear summary on the utility of providing this
> information.

I can summarize PulseAudio's case. The current policies slightly
differ in the way SCO is handled. The HS role is more passive,
accepting remote SCO connections and releases. The AG, on the other
hand, initiates an SCO connection as soon as some audio starts
streaming, or the user sets the card's profile to "hsp" manually.

The HS role currently switches between HSP/HFP and A2DP automatically.
So if SCO starts streaming, the profile is switched and the audio gets
routed. This is not the case for the AG role (desktop use-case) where
no automatic switch exists and the user-setting applies. I believe
this shouldn't be necessary in the long term, if the routing problem
was solved, but that alone is a big topic.

Finally, PA exposes per-profile information in the UI, so the user
actually distinguishes AG and HS roles. So if you pair a Nokia support
both profiles, you would see them both in the UI. I guess this could
be changed, but not without a controversial discussion. Even several
BlueZ developers have argued that this information is very convenient
for development and testing.

Note that the HSP vs HFP difference is not relevant, so I'm just
talking about separating AG vs HS roles.

>
>>>> If so, I have some concerns about the potential race conditions. If
>>>> SCO was closed and reopen immediately afterwards, the client could
>>>> receive the (second) NewConnection before the HUP.
>>>>
>
> You are going over-the-air vs local machine.  I really do not see this ever
> happening.  But even if it does, is this really a hard problem for the Agent
> to deal with?

I think it would be quite obscure to assume that the previous socket
will HUP shortly after this second NewConnection is received. So if
this approach is encouraged, I'd expect some reference in the doc.

Cheers,
Mikel

Re: [PATCH] doc: Add experimental handsfree-audio API

[Denis Kenzior]

[-- Attachment #1: Type: text/plain, Size: 4386 bytes --]

Hi Mikel,

>> This API is meant specifically for PA or another Audio subsystem.  What you
>> want to be tracking and cross-referencing I'm not quite sure of.
>
> I already mentioned one example: if a headset wants to ignore the
> inband ringtone, and use it's own custom one, it needs to match the
> audio-card and the voicecall state. During the ringtone (callsetup=1),
> the SCO stream would be ignored, as suggested in section 4.13.4 of the
> spec.

Why? Just select a different sound device if the call status is 
incoming.  Why do you need to know this down to the lowest level of 
detail?  Or better yet, just turn the in-band ringing off at the AG side 
and be done with it.

The AG can start an audio connection for any reason whatsoever, trying 
to 'outsmart' it will likely never work due to deficiencies in the 
protocol.  I would love to hear how you plan to implement voice 
recognition policies.

If you want to control everything down to the nitty gritty, you probably 
need a dedicated application.  There's no way all of this level of 
detail is ever going to be exposed over IPC.  Even oFono core lacks much 
of this information due to abstractions.

>
> I could list more examples too if needed, but they would involve
> multiple connected AGs.

Please do.

> Not that I agree with the idea of re-engineering the Media API, but anyway...
>

Lets put it this way, Media API has a snowball's chance in hell of going 
to oFono upstream.  So lets put this discussion to rest already.

>>
>>
>>> The ability to distinguish HS and AG roles in the client's side is
>>> desirable (at least PulseAudio heavily depends on this information).
>>> So even if the proposal above is not accepted, at least the role
>>> (UUID) would have to be exposed somehow.
>>>
>>
>> The type information was left out on purpose from this proposal.  Likely it
>> is necessary, but for the initial phase it does not seem relevant.  I also
>> have yet to hear concise and clear summary on the utility of providing this
>> information.
>
> I can summarize PulseAudio's case. The current policies slightly
> differ in the way SCO is handled. The HS role is more passive,
> accepting remote SCO connections and releases. The AG, on the other
> hand, initiates an SCO connection as soon as some audio starts
> streaming, or the user sets the card's profile to "hsp" manually.
>
> The HS role currently switches between HSP/HFP and A2DP automatically.
> So if SCO starts streaming, the profile is switched and the audio gets
> routed. This is not the case for the AG role (desktop use-case) where
> no automatic switch exists and the user-setting applies. I believe
> this shouldn't be necessary in the long term, if the routing problem
> was solved, but that alone is a big topic.
>
> Finally, PA exposes per-profile information in the UI, so the user
> actually distinguishes AG and HS roles. So if you pair a Nokia support
> both profiles, you would see them both in the UI. I guess this could
> be changed, but not without a controversial discussion. Even several
> BlueZ developers have argued that this information is very convenient
> for development and testing.
>
> Note that the HSP vs HFP difference is not relevant, so I'm just
> talking about separating AG vs HS roles.
>

I'm still lost.  But that is okay.  Can we please implement the basic 
proposal as it is now and then start adding features?  Remember the 
motto, 'make it work, then make it work better'.

>>
>>>>> If so, I have some concerns about the potential race conditions. If
>>>>> SCO was closed and reopen immediately afterwards, the client could
>>>>> receive the (second) NewConnection before the HUP.
>>>>>
>>
>> You are going over-the-air vs local machine.  I really do not see this ever
>> happening.  But even if it does, is this really a hard problem for the Agent
>> to deal with?
>
> I think it would be quite obscure to assume that the previous socket
> will HUP shortly after this second NewConnection is received. So if
> this approach is encouraged, I'd expect some reference in the doc.

Uhhh, this situation will never happen.  Too much thinking, stop it ;) 
If you want to be ultra-paranoid just close the previous socket when you 
get a NewConnection with the same card id, but I digress...

Regards,
-Denis

Re: [PATCH] doc: Add experimental handsfree-audio API

[Mikel Astiz]

[-- Attachment #1: Type: text/plain, Size: 6147 bytes --]

Hi Denis,

On Wed, Feb 20, 2013 at 6:34 PM, Denis Kenzior <denkenz@gmail.com> wrote:
> Hi Mikel,
>
>
>>> This API is meant specifically for PA or another Audio subsystem.  What
>>> you
>>> want to be tracking and cross-referencing I'm not quite sure of.
>>
>>
>> I already mentioned one example: if a headset wants to ignore the
>> inband ringtone, and use it's own custom one, it needs to match the
>> audio-card and the voicecall state. During the ringtone (callsetup=1),
>> the SCO stream would be ignored, as suggested in section 4.13.4 of the
>> spec.
>
>
> Why? Just select a different sound device if the call status is incoming.
> Why do you need to know this down to the lowest level of detail?  Or better
> yet, just turn the in-band ringing off at the AG side and be done with it.

I'm not following any of these proposals.

For the first one, in order to know the call status, it's necessary to
known the association between the audio-card and the modem. I don't
know what you mean with "lowest level of detail". Just the mapping
between org.ofono.Modem and org.ofono.AudioCard needs to be known, and
ideally, without having to use the modem's serial number.

Regarding the second proposal, I'm not aware of this being possible.
Perhaps you can point to some part in the spec where this is defined?

>
> The AG can start an audio connection for any reason whatsoever, trying to
> 'outsmart' it will likely never work due to deficiencies in the protocol.  I
> would love to hear how you plan to implement voice recognition policies.

The AG reports when voice recognition is active, so I see no big deal
with this one.

>
> If you want to control everything down to the nitty gritty, you probably
> need a dedicated application.  There's no way all of this level of detail is
> ever going to be exposed over IPC.  Even oFono core lacks much of this
> information due to abstractions.

What exactly do you mean? I don't see any information missing so far.
Just the mapping between audio-cards and modems is not explicit enough
IMO in the proposed Handsfree-Audio API. I'm just trying to argue that
this coupling could be more explicit.

>
>>
>> I could list more examples too if needed, but they would involve
>> multiple connected AGs.
>
>
> Please do.

Examples include two connected AGs (phone A and phone B) where one of
them (A) contains an active call, and the other one (B) contains a
held call. In this case the audio from phone A should be routed, and
the one from phone B ignored.

This one is very similar to the first example: the audio-subsystem
needs to know the call status associated to each audio-card.

>
>
>> Not that I agree with the idea of re-engineering the Media API, but
>> anyway...
>>
>
> Lets put it this way, Media API has a snowball's chance in hell of going to
> oFono upstream.  So lets put this discussion to rest already.

You're certainly aware that this duplicates the client code since they
have to have separate implementations for A2DP and HSP/HFP. The motto
of "simplifying client code" is not relevant anymore, as it seems.

>
>
>>>
>>>
>>>> The ability to distinguish HS and AG roles in the client's side is
>>>> desirable (at least PulseAudio heavily depends on this information).
>>>> So even if the proposal above is not accepted, at least the role
>>>> (UUID) would have to be exposed somehow.
>>>>
>>>
>>> The type information was left out on purpose from this proposal.  Likely
>>> it
>>> is necessary, but for the initial phase it does not seem relevant.  I
>>> also
>>> have yet to hear concise and clear summary on the utility of providing
>>> this
>>> information.
>>
>>
>> I can summarize PulseAudio's case. The current policies slightly
>> differ in the way SCO is handled. The HS role is more passive,
>> accepting remote SCO connections and releases. The AG, on the other
>> hand, initiates an SCO connection as soon as some audio starts
>> streaming, or the user sets the card's profile to "hsp" manually.
>>
>> The HS role currently switches between HSP/HFP and A2DP automatically.
>> So if SCO starts streaming, the profile is switched and the audio gets
>> routed. This is not the case for the AG role (desktop use-case) where
>> no automatic switch exists and the user-setting applies. I believe
>> this shouldn't be necessary in the long term, if the routing problem
>> was solved, but that alone is a big topic.
>>
>> Finally, PA exposes per-profile information in the UI, so the user
>> actually distinguishes AG and HS roles. So if you pair a Nokia support
>> both profiles, you would see them both in the UI. I guess this could
>> be changed, but not without a controversial discussion. Even several
>> BlueZ developers have argued that this information is very convenient
>> for development and testing.
>>
>> Note that the HSP vs HFP difference is not relevant, so I'm just
>> talking about separating AG vs HS roles.
>>
>
> I'm still lost.  But that is okay.  Can we please implement the basic
> proposal as it is now and then start adding features?  Remember the motto,
> 'make it work, then make it work better'.

Sure, I'm completely fine with that.

>
>>>
>>>>>> If so, I have some concerns about the potential race conditions. If
>>>>>> SCO was closed and reopen immediately afterwards, the client could
>>>>>> receive the (second) NewConnection before the HUP.
>>>>>>
>>>
>>> You are going over-the-air vs local machine.  I really do not see this
>>> ever
>>> happening.  But even if it does, is this really a hard problem for the
>>> Agent
>>> to deal with?
>>
>>
>> I think it would be quite obscure to assume that the previous socket
>> will HUP shortly after this second NewConnection is received. So if
>> this approach is encouraged, I'd expect some reference in the doc.
>
>
> Uhhh, this situation will never happen.  Too much thinking, stop it ;) If
> you want to be ultra-paranoid just close the previous socket when you get a
> NewConnection with the same card id, but I digress...

Cheers,
Mikel

Re: [PATCH] doc: Add experimental handsfree-audio API

[Denis Kenzior]

[-- Attachment #1: Type: text/plain, Size: 4701 bytes --]

Hi Mikel,

On 02/22/2013 04:06 AM, Mikel Astiz wrote:
> Hi Denis,
>
> On Wed, Feb 20, 2013 at 6:34 PM, Denis Kenzior<denkenz@gmail.com>  wrote:
>> Hi Mikel,
>>
>>
>>>> This API is meant specifically for PA or another Audio subsystem.  What
>>>> you
>>>> want to be tracking and cross-referencing I'm not quite sure of.
>>>
>>>
>>> I already mentioned one example: if a headset wants to ignore the
>>> inband ringtone, and use it's own custom one, it needs to match the
>>> audio-card and the voicecall state. During the ringtone (callsetup=1),
>>> the SCO stream would be ignored, as suggested in section 4.13.4 of the
>>> spec.
>>
>>
>> Why? Just select a different sound device if the call status is incoming.
>> Why do you need to know this down to the lowest level of detail?  Or better
>> yet, just turn the in-band ringing off at the AG side and be done with it.
>
> I'm not following any of these proposals.
>
> For the first one, in order to know the call status, it's necessary to
> known the association between the audio-card and the modem. I don't
> know what you mean with "lowest level of detail". Just the mapping
> between org.ofono.Modem and org.ofono.AudioCard needs to be known, and
> ideally, without having to use the modem's serial number.

Why? You have an incoming call on Modem A.  You know the contact as 
Contact X.  You are ignoring the SCO in-band ring anyway.  Just play the 
'ring-tone for contact X' until you have an active call.  There is no 
need for you to do any cross-referencing that I see.

>
> Regarding the second proposal, I'm not aware of this being possible.
> Perhaps you can point to some part in the spec where this is defined?
>

I went back and re-read the spec.  I mis-spoke on this one.  Indeed only 
the AG can change this setting, which in practice means it is fully 
pointless part of the protocol.  The fact that HF cannot control in-band 
ringing setting is a major oversight in the protocol in my opinion.

>>
>> The AG can start an audio connection for any reason whatsoever, trying to
>> 'outsmart' it will likely never work due to deficiencies in the protocol.  I
>> would love to hear how you plan to implement voice recognition policies.
>
> The AG reports when voice recognition is active, so I see no big deal
> with this one.
>

Yes it does.  My point here was that you still do not need any 
cross-referencing here.  Just pipe the data to SCO when it is asked for.

>>>
>>> I could list more examples too if needed, but they would involve
>>> multiple connected AGs.
>>
>>
>> Please do.
>
> Examples include two connected AGs (phone A and phone B) where one of
> them (A) contains an active call, and the other one (B) contains a
> held call. In this case the audio from phone A should be routed, and
> the one from phone B ignored.
>
> This one is very similar to the first example: the audio-subsystem
> needs to know the call status associated to each audio-card.
>

So we're off into advanced use-cases lala land, where we have multiple 
SCO streams, have you even tested whether multiple simultaneous mSBC 
streams work?

And how do you even get to this point?  If phone A is on the call and 
you get a call on Phone B, what do you even do?  Auto-hold Phone A? 
Play Ring Tone while Phone A is active?  More importantly, why would you 
keep both SCO links active in the first place?

Actually, I'm not sure I want to know ;)  Right now we don't even have 
the simple 1:1 connection case working properly, so lets get that 
working first before running off into the weeds.

>>
>>
>>> Not that I agree with the idea of re-engineering the Media API, but
>>> anyway...
>>>
>>
>> Lets put it this way, Media API has a snowball's chance in hell of going to
>> oFono upstream.  So lets put this discussion to rest already.
>
> You're certainly aware that this duplicates the client code since they
> have to have separate implementations for A2DP and HSP/HFP. The motto
> of "simplifying client code" is not relevant anymore, as it seems.
>

I disagree completely.  Media API was trying to abstract both protocols 
which are completely different.  It didn't do a very good job, whatever 
was being encapsulated still bled through into the API, particularly 
things oFono would never ever care about.  In short, the oFono side of 
things looked just plain awful.

By the way, whatever points you are raising above, would still apply to 
the Media API, you just have to deal with even more complexity.  In the 
end I rather have more code that is simpler to understand than less code 
that is unreadable.

Regards,
-Denis