Unreliable next_msg_id (was Re: [PATCH 2/2] Added SQLite history plugin)

Unreliable next_msg_id (was Re: [PATCH 2/2] Added SQLite history plugin)

[Nicola Mfb]

[-- Attachment #1: Type: text/plain, Size: 919 bytes --]

Hi!

At first look it seems that ofono stores permanently the next_msg_id
only when the "sms_remove" functions is called, and that's does not
happen when ofono quits in a not clean way.

During our tests, users reported bad behaivour of the sqlite history
plugins and error messages about sqlite constraints violation, after a
rude power off of the device (battery drained, forced shutdown with
long press power button, and so on).

That's becouse ofono on the next start reads an "old" next_msg_id from
the permanent storage, so when sending/receiving a new message the
history plugin fails to insert a new row as one with the same msg_id
was already inserted in the prev session.

Any suggestion to fix that?

Finally, we'd like to use some "panic" function in ofono, that should
power down all modems and warn clients when critical conditions
happen.

Is that possible?

Regards

    Niko

Re: Unreliable next_msg_id (was Re: [PATCH 2/2] Added SQLite history plugin)

[Denis Kenzior]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]

Hi Niko,

> Hi!
> 
> At first look it seems that ofono stores permanently the next_msg_id
> only when the "sms_remove" functions is called, and that's does not
> happen when ofono quits in a not clean way.
> 
> During our tests, users reported bad behaivour of the sqlite history
> plugins and error messages about sqlite constraints violation, after a
> rude power off of the device (battery drained, forced shutdown with
> long press power button, and so on).

So the immediate fix is to set & sync the SMS settings whenever the next_msg_id 
is updated.

> 
> That's becouse ofono on the next start reads an "old" next_msg_id from
> the permanent storage, so when sending/receiving a new message the
> history plugin fails to insert a new row as one with the same msg_id
> was already inserted in the prev session.
> 
> Any suggestion to fix that?

The proper fix is to generate a SHA1 or MD5 hash over the contents, local 
reception/send time, remote reception / send time, destination / originator of 
the message.

> 
> Finally, we'd like to use some "panic" function in ofono, that should
> power down all modems and warn clients when critical conditions
> happen.
> 
> Is that possible?

Send HUP to the daemon?  Or do you want oFono to keep running?

Regards,
-Denis

Re: Unreliable next_msg_id (was Re: [PATCH 2/2] Added SQLite history plugin)

[Nicola Mfb]

[-- Attachment #1: Type: text/plain, Size: 2507 bytes --]

On Thu, Apr 15, 2010 at 8:35 PM, Denis Kenzior <denkenz@gmail.com> wrote:
[...]
>> During our tests, users reported bad behaivour of the sqlite history
>> plugins and error messages about sqlite constraints violation, after a
>> rude power off of the device (battery drained, forced shutdown with
>> long press power button, and so on).
>
> So the immediate fix is to set & sync the SMS settings whenever the next_msg_id
> is updated.

Yes, this for sure will help a lot.

>> That's becouse ofono on the next start reads an "old" next_msg_id from
>> the permanent storage, so when sending/receiving a new message the
>> history plugin fails to insert a new row as one with the same msg_id
>> was already inserted in the prev session.
>>
>> Any suggestion to fix that?
>
> The proper fix is to generate a SHA1 or MD5 hash over the contents, local
> reception/send time, remote reception / send time, destination / originator of
> the message.

Dario suggested that to me too, the only problem I see is that an hash
signature does not provide ">" operator that may help in some sync
scenarios, but this may be fixed in several ways (for example using an
additional autoincrement serial column that may expose to applications
an unique, ordered and reliable id without exposing the ofono internal
one).

Will check if all history callbacks lets us rehash the messages properly.

A shot in the dark: what's about delegating the id generation to the
history plugin (if present and capable)?
That should fix the above problems, avoid hash generation and provide
a possible base to reuse the id while resubmitting failed/pending
messages providing the right id to ofono, avoiding the introduction of
race conditions or complex implementations.

Anyway this is a delicate and upcoming use case, we just created our
git tracking to experiment some solutions, when ready will submit
patches again.

>> Finally, we'd like to use some "panic" function in ofono, that should
>> power down all modems and warn clients when critical conditions
>> happen.
>>
>> Is that possible?
>
> Send HUP to the daemon?  Or do you want oFono to keep running?

I mean the best way to shutdown modems and ofono inside the code (in
the history for example if an sql statement fails, or if it cannot
save id to the permanent storage, big dbus troubles or other similiar
cases).
We just discovered our users may become crazy if they lose a single sms/call :)

Regards

     Niko

Re: Unreliable next_msg_id (was Re: [PATCH 2/2] Added SQLite history plugin)

[Denis Kenzior]

[-- Attachment #1: Type: text/plain, Size: 2034 bytes --]

Hi Niko,

> > The proper fix is to generate a SHA1 or MD5 hash over the contents, local
> > reception/send time, remote reception / send time, destination /
> > originator of the message.
> 
> Dario suggested that to me too, the only problem I see is that an hash
> signature does not provide ">" operator that may help in some sync
> scenarios, but this may be fixed in several ways (for example using an
> additional autoincrement serial column that may expose to applications
> an unique, ordered and reliable id without exposing the ofono internal
> one).

I suggest using a sync timestamp or some other way to accomplish this.

> 
> Will check if all history callbacks lets us rehash the messages properly.
> 
> A shot in the dark: what's about delegating the id generation to the
> history plugin (if present and capable)?
> That should fix the above problems, avoid hash generation and provide
> a possible base to reuse the id while resubmitting failed/pending
> messages providing the right id to ofono, avoiding the introduction of
> race conditions or complex implementations.

oFono will keep pending messages around across reboots, that is an enhancement 
we're already working on.  Resubmitting of failed messages should result in a 
new ID anyway.

And I honestly don't think N implementations of hashing SMS messages in each 
history plugin is a good idea.  Let oFono do it so we get it right for 
everyone.

> >> Finally, we'd like to use some "panic" function in ofono, that should
> >> power down all modems and warn clients when critical conditions
> >> happen.
> >>
> >> Is that possible?
> >
> > Send HUP to the daemon?  Or do you want oFono to keep running?
> 
> I mean the best way to shutdown modems and ofono inside the code (in
> the history for example if an sql statement fails, or if it cannot
> save id to the permanent storage, big dbus troubles or other similiar
> cases).

Yikes.  The answer is "Please don't even think about it" :)

Regards,
-Denis