oFono upstream test results_20130517

oFono upstream test results_20130517

[Nicolas Paccou]

[-- Attachment #1: Type: text/plain, Size: 5804 bytes --]

Hello all,

 

Please find the test report of oFono v1.12 commit 1c75edf.

Due to Network Simulator License issue, no report has been sent last month,
I'm sorry for the inconvenience. This problem is now resolved.

 

During this testing, we ran 462 functional positive cases. 372 cases passed,
1 failed and 89 are blocked due to missing software/missing hardware/Feature
not supported. The pass rate is 81% (=) which is equal than previous test
cycle (v1.12 commit 4fcd084). No new issue has been found.

 

If you have any comment about this report, contact me please.

 

 

Test Objective

 

The aim of this test cycle was to validate the state of oFono upstream by
testing all its features over all material we had (according to what feature
was supported and by priority order: Smartphone with real SIM or with
Network Simulator, on Laptop with 3G dongle or with Phonesim). oFono has
been installed and tested on smartphone device running Tizen 2.0 and Ubuntu
12.04 device.

 

 

Test Environment

 

For all Setup:

oFono: v1.12 (updated to commit 1c75edf)

 

-      Smartphone device Setup:

Image: Tizen Mobile 2.0

Operator & SIM Card: several french and UK SIM Cards according to features

 

-      Laptop + 3G Dongle/Phonesim:

usb_modeswitch: v1.2.3

modeswitch data: 20120120

Hardware: Laptop

Ubuntu: v12.04

Modem: Huawei E173u-2 - Operator & SIM Card: several french and UK SIM Cards
according to features

Phonesim: v1.19 (updated to commit c94e6c0)

     

-      Laptop + HFP

HFP Server:

Hardware: Laptop

Ubuntu: v12.04

Bluez 4.101 (updated to commit 67ef3ac)

oFono has been installed with "--enable-bluez4" option instead of using
default bluez5 plugin (unable to make HFP using this one)

HFP Client: Android JB device

 

 

Issue Summary

 

New bug: 0

None

 

Known bug: 1

Bug 25982 <https://bugs.meego.com/show_bug.cgi?id=25982>  - Going back from
a select item list releases the session

 

Closed bug: 0

None

 

 

Test Result

 

SUMMARY

Total Test Case                  462

Passed                              372

Failed                                1

Blocked                             89

TCs completed                   100,0%

Run rate                            81%

Pass rate total                    81% (=)

Blocked rate total               19% (=)

Pass rate of executed          100%

 

FEATURES                                   TOTAL           PASS  FAIL
BLOCKED      PASS %          MODEM USED

Modem                                        16                16      0
0                  100%            Smartphone using real SIM and Laptop +
HFP Modem

SIM                                             20                20      0
0                  100%            Smartphone using real SIM

Phonebook                                   1                  1        0
0                  100%            Smartphone using real SIM

Network                                      18                18      0
0                  100%            Smartphone using real SIM

Radio                                          22                17      0
5                  77%             Smartphone using real SIM

Connectivity                                28                28      0
0                  100%            Smartphone using real SIM and Laptop + 3G
Dongle (for PPP tests)

Voice Calls                                   50                45      0
5                  90%             Smartphone using real SIM or Network
Simulator

Messaging                                   24                22      0
2                  92%             Smartphone using real SIM or Phonesim

Smart Messaging                         15                15      0        0
100%            Laptop + Phonesim

Message Waiting                          8                  8        0
0                  100%            Laptop + Phonesim

Cell Broadcast                              15                12      0
3                  80%             Laptop + Phonesim

Call Settings                                13                9        0
4                  69%             Smartphone using real SIM

Supplementary Services               59                56      0        3
95%             Smartphone (using appropriate SIM Card)

Call Meter                                    15                13      0
2                  87%             Smartphone using Network Simulator

Call Barring                                 49                33      0
16                67%             Smartphone (using appropriate SIM Cards)

Call Forwarding                            5                  5        0
0                  100%            Smartphone (using appropriate SIM Cards)

SimToolKit                                  104               54      1
49                52%             Laptop + Phonesim

 

Please find details in the attached file.

 

 

Notes

 

Please note that I have found a crash of oFono when running two STK TCs in a
row: 

Bug <https://bugs.meego.com/show_bug.cgi?id=25998>  25998 - oFono crashes
when doing a NAA Initialization+File Change Notification after having done a
NAA Initialization+Full File Change Notification with Phonesim.

 

19% of cases are blocked due to:

-      Missing software (e.g.: Missing feature in Phonesim or missing a
visual application for some blocked feature as Setup Idle ModeText, Select
Item, Send SMS, etc.)

-      Missing hardware (e.g.: Missing other SIM (having appropriate service
activated).

-      Feature not supported by oFono

 

Best regards,

Nicolas


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 27209 bytes --]

[-- Attachment #3: oFono1.12_1c75edf_TCs_TitlesOnly.xlsx --]
[-- Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, Size: 87219 bytes --]