Hello all,

Please find the test report of oFono v1.13 commit 7a6da06.

During this testing, we ran 34 functional positive cases. 32 cases passed, 1 failed and 1 blocked due to bug (see Issue Summary section for details). The Pass Rate is 94% (-6% vs previous session). One regression has been found concerning Lock/Unlock PIN code feature.

Specifically for 1.13 release which supports HFP 1.6 features, a PTS test session has been also run to validate it. We ran a total of 75 functional positive cases. 72 cases passed, 0 failed and 3 blocked due to issue (see Notes section for details) or feature not supported. The Pass Rate is 96%.

If you have any comment about this report, tell me please.

-----------------------------------------------

Test Objective

The aim of this session was to validate the state of oFono upstream by testing only major tests of most important features over all material we had (according to what feature was supported and by priority order: PR3, Network Simulator, 3G dongle) and specifically for 1.13 release which supports HFP 1.6 features a PTS test session has been also run to validate it. oFono has been installed and tested on PR3 running Tizen 2.0 and Ubuntu 12.04 device for standard Telephony features and installed on IVI board for HFP features.

-----------------------------------------------

Test Environment

For all Setup:
oFono: v1.13 (updated to commit 7a6da06)

-      PR3 Setup:
    Image: Tizen_Mobile_ofono_branch_09_06_2013 (Tizen 2.0 image)
    Operator & SIM Card: SFR SIM Card – Phone number +33623312183, Bouygues SIM Card – Phone number +33760660848

- Laptop + 3G Dongle/Phonesim:
usb_modeswitch: v1.2.5
modeswitch data: 20120120
Hardware: Laptop
Ubuntu: v12.04
Modem: Huawei E173u-2 - Operator & SIM Card: SFR SIM Card – Phone number +33623312183, Bouygues SIM Card – Phone number +33663999052

    -      IVI Setup (for PTS tests)
    Device: Ndis 166 IVI board
    Image: tizen_20131030.3_ivi-mbr-i586-sdb
    Packages installed (which was missing in the image):
        bluez-cups-5.10-1.1.i686.rpm
        bluez-devel-5.10-1.1.i686.rpm
        bluez-docs-5.10-1.1.i686.rpm
        bluez-test-5.10-1.1.i686.rpm
        libpulse-devel-4.0-4.1.i686.rpm
        ofono-devel-1.13-1.i686.rpm (updated to commit 7a6da06)
        ofono-docs-1.13-1.i686.rpm (updated to commit 7a6da06)
        ofono-test-1.13-1.i686.rpm (updated to commit 7a6da06)
        pulseaudio-docs-4.0-4.1.i686.rpm
        pulseaudio-module-augment-properties-4.0-4.1.i686.rpm
        pulseaudio-module-combine-sink-4.0-4.1.i686.rpm
        pulseaudio-module-dbus-protocol-4.0-4.1.i686.rpm
        pulseaudio-module-devel-4.0-4.1.i686.rpm
        pulseaudio-module-filter-4.0-4.1.i686.rpm
        pulseaudio-module-null-source-4.0-4.1.i686.rpm
        pulseaudio-module-raop-4.0-4.1.i686.rpm
        pulseaudio-module-switch-on-connect-4.0-4.1.i686.rpm
        pulseaudio-vala-bindings-4.0-4.1.i686.rpm

-----------------------------------------------

Issue Summary

New bug: 2

OF-168 - Unable to lock/unlock PIN Code while modem is online
https://01.org/jira/browse/OF-168

Issue: Unable to initiate audio connection (error in btmon when executing "set-card-profile x hfgw" pacmd: Unsupported Remote Feature / Unsupported LMP Feature (0x1a))
Components: To be analyzed (hardware issue?)
Severity: Normal
Status: Open
Assignee: Luiz Von Dentz

Known bug: 1

OF-162 - Going back from a select item list releases the session
https://01.org/jira/browse/OF-162

Closed bug: 0
None

-----------------------------------------------

Test Result

Result Summary – Basic Features

Total Test Case	34
Passed	32
Failed	1
Blocked	1
TCs completed	100,0%
Run rate	97%
Pass rate total	94%
Blocked rate total	3%
Pass rate of executed	97%

Test Result by feature

Features (and their status in color)	Total	Pass	Fail	Blocked	Pass %	Modem Used
Modem	5	5	0	0	100%	Tested on smartphone using a real SIM or on Laptop using 3G Dongle
SIM	4	2	1	1	50%	Tested on Laptop using 3G Dongle
Network	2	2	0	0	100%	Tested on smartphone using a real SIM
Connectivity	8	8	0	0	100%	Tested on smartphone using a real SIM or on Laptop using 3G Dongle
Voice Calls	12	12	0	0	100%	Tested on smartphone with appropriate SIM Card and Laptop + HFP connected with a smartphone
Messaging	2	2	0	0	100%	Tested on smartphone using a real SIM
Message Waiting	1	1	0	0	100%	Tested on smartphone using a real SIM

Please find details in the attached file.

Result Summary – PTS HFP 1.6 Features

Total Test Case	75
Passed	72
Failed	0
Blocked	3
TCs completed	100,0%
Run rate	96%
Pass rate total	96%
Blocked rate total	4%
Pass rate of executed	100%

Test Result by feature

Features (and their status in color)	Total	Pass	Fail	Blocked	Pass %
Transfer of Registration Status	1	1	0	0	100%
Phone status information	4	4	0	0	100%
Audio Connection handling	4	4	0	0	100%
Audio Connection release	2	2	0	0	100%
Accept an incoming voice call	1	1	0	0	100%
Incoming voice call audio scenarios	6	6	0	0	100%
Reject an incoming voice call	2	2	0	0	100%
Terminate a voice call	4	4	0	0	100%
Audio Connection transfer during an ongoing call to HF	5	3	0	2	60%
Audio Connection transfer during an ongoing call to AG	2	2	0	0	100%
Place a call with the phone number supplied by the HF	1	1	0	0	100%
Call waiting notification - Three way calling	6	6	0	0	100%
Incoming voice call interrupted by remote	1	1	0	0	100%
Echo canceling (EC) and noise reduction (NR)	1	1	0	0	100%
Attach a phone number to a voice tag	1	0	0	1	0%
Ability to transmit DTMF codes	1	1	0	0	100%
Remote audio speaker volume control	3	3	0	0	100%
Remote audio micro volume control	3	3	0	0	100%
Query current call status	3	3	0	0	100%
Enhanced Call Control	2	2	0	0	100%
Service Level Connection establishment - Codec Negotiation	7	7	0	0	100%
Codec Connection set up	7	7	0	0	100%
Wide Band Speech	2	2	0	0	100%
Connection management	2	2	0	0	100%
Connection management - Service Search Request	3	3	0	0	100%
Place a call with the phone number supplied by the AG	1	1	0	0	100%

-----------------------------------------------

Notes

Please note that one oFono crash still occurs on the following conditions:
-      OF-163 - oFono crashes when doing a NAA Initialization+File Change Notification after having done a NAA Initialization+Full File Change Notification with Phonesim.

A bug in PTS software (11515 which sends to oFono the wrong HFP version supported) preventing to execute several TCs has been avoided during test session.
    Workaround details:
        1.       Change the HFP version in /var/lib/Bluetooth/<Local mac address>/cache/<PTS mac address>
        2.       Kill Bluetoothd
        3.       Relaunch Bluetoothd
        4.       Redo the PTS tests that was blocked by this issue.

Best regards,
Nicolas