* oFono upstream test results_20131115
@ 2013-11-18 12:04 Nicolas Paccou
0 siblings, 0 replies; only message in thread
From: Nicolas Paccou @ 2013-11-18 12:04 UTC (permalink / raw)
To: ofono
[-- Attachment #1: Type: text/plain, Size: 8984 bytes --]
Hello all,
Please find the test report of oFono v1.13 commit 7a6da06.
During this testing, we ran 34 functional positive cases. 32 cases
passed, 1 failed and 1 blocked due to bug (see Issue Summary section for
details). The Pass Rate is 94% (-6% vs previous session). One regression
has been found concerning Lock/Unlock PIN code feature.
Specifically for 1.13 release which supports HFP 1.6 features, a PTS
test session has been also run to validate it. We ran a total of 75
functional positive cases. 72 cases passed, 0 failed and 3 blocked due
to issue (see Notes section for details) or feature not supported. The
Pass Rate is 96%.
If you have any comment about this report, tell me please.
-----------------------------------------------
*Test Objective *
The aim of this session was to validate the state of oFono upstream by
testing only major tests of most important features over all material we
had (according to what feature was supported and by priority order: PR3,
Network Simulator, 3G dongle) and specifically for 1.13 release which
supports HFP 1.6 features a PTS test session has been also run to
validate it. oFono has been installed and tested on PR3 running Tizen
2.0 and Ubuntu 12.04 device for standard Telephony features and
installed on IVI board for HFP features.
-----------------------------------------------
*Test Environment *
For all Setup:
oFono: v1.13 (updated to commit 7a6da06)
-PR3 Setup:
Image: Tizen_Mobile_ofono_branch_09_06_2013 (Tizen 2.0 image)
Operator & SIM Card: SFR SIM Card -- Phone number +33623312183,
Bouygues SIM Card -- Phone number +33760660848
-Laptop + 3G Dongle/Phonesim:
usb_modeswitch: v1.2.5
modeswitch data: 20120120
Hardware: Laptop
Ubuntu: v12.04
Modem: Huawei E173u-2 - Operator & SIM Card: SFR SIM Card -- Phone
number +33623312183, Bouygues SIM Card -- Phone number +33663999052
-IVI Setup (for PTS tests)
Device: Ndis 166 IVI board
Image: tizen_20131030.3_ivi-mbr-i586-sdb
Packages installed (which was missing in the image):
bluez-cups-5.10-1.1.i686.rpm
bluez-devel-5.10-1.1.i686.rpm
bluez-docs-5.10-1.1.i686.rpm
bluez-test-5.10-1.1.i686.rpm
libpulse-devel-4.0-4.1.i686.rpm
ofono-devel-1.13-1.i686.rpm (updated to commit 7a6da06)
ofono-docs-1.13-1.i686.rpm (updated to commit 7a6da06)
ofono-test-1.13-1.i686.rpm (updated to commit 7a6da06)
pulseaudio-docs-4.0-4.1.i686.rpm
pulseaudio-module-augment-properties-4.0-4.1.i686.rpm
pulseaudio-module-combine-sink-4.0-4.1.i686.rpm
pulseaudio-module-dbus-protocol-4.0-4.1.i686.rpm
pulseaudio-module-devel-4.0-4.1.i686.rpm
pulseaudio-module-filter-4.0-4.1.i686.rpm
pulseaudio-module-null-source-4.0-4.1.i686.rpm
pulseaudio-module-raop-4.0-4.1.i686.rpm
pulseaudio-module-switch-on-connect-4.0-4.1.i686.rpm
pulseaudio-vala-bindings-4.0-4.1.i686.rpm
-----------------------------------------------
*Issue Summary *
New bug: 2
OF-168 - Unable to lock/unlock PIN Code while modem is online
https://01.org/jira/browse/OF-168
Issue: Unable to initiate audio connection (error in btmon when
executing "set-card-profile x hfgw" pacmd: Unsupported Remote Feature /
Unsupported LMP Feature (0x1a))
Components: To be analyzed (hardware issue?)
Severity: Normal
Status: Open
Assignee:Luiz Von Dentz
Known bug: 1
OF-162 - Going back from a select item list releases the session
https://01.org/jira/browse/OF-162
Closed bug: 0
None
-----------------------------------------------
*Test Result *
*Result Summary -- Basic Features*
Total Test Case
*34*
Passed
32
Failed
1
Blocked
1
TCs completed
*100,0%*
Run rate
*97%*
Pass rate total
*94%*
Blocked rate total
*3%*
Pass rate of executed
*97%*
*Test Result by feature
*
*Features (and their status in color)*
*Total*
*Pass*
*Fail*
*Blocked*
*Pass %*
*Modem Used*
Modem
*5*
5
0
0
*100%*
Tested on smartphone using a real SIM or on Laptop using 3G Dongle
SIM
*4*
2
1
1
*50%*
Tested on Laptop using 3G Dongle
Network
*2*
2
0
0
*100%*
Tested on smartphone using a real SIM
Connectivity
*8*
8
0
0
*100%*
Tested on smartphone using a real SIM or on Laptop using 3G Dongle
Voice Calls
*12*
12
0
0
*100%*
Tested on smartphone with appropriate SIM Card and Laptop + HFP
connected with a smartphone
Messaging
*2*
2
0
0
*100%*
Tested on smartphone using a real SIM
Message Waiting
*1*
1
0
0
*100%*
Tested on smartphone using a real SIM
Please find details in the attached file.*
**Result Summary -- PTS HFP 1.6 Features***
Total Test Case
*75*
Passed
72
Failed
0
Blocked
3
TCs completed
*100,0%*
Run rate
*96%*
Pass rate total
*96%*
Blocked rate total
*4%*
Pass rate of executed
*100%*
*Test Result by feature*
*Features (and their status in color)*
*Total*
*Pass*
*Fail*
*Blocked*
*Pass %*
Transfer of Registration Status
*1*
1
0
0
*100%*
Phone status information
*4*
4
0
0
*100%*
Audio Connection handling
*4*
4
0
0
*100%*
Audio Connection release
*2*
2
0
0
*100%*
Accept an incoming voice call
*1*
1
0
0
*100%*
Incoming voice call audio scenarios
*6*
6
0
0
*100%*
Reject an incoming voice call
*2*
2
0
0
*100%*
Terminate a voice call
*4*
4
0
0
*100%*
Audio Connection transfer during an ongoing call to HF
*5*
3
0
2
*60%*
Audio Connection transfer during an ongoing call to AG
*2*
2
0
0
*100%*
Place a call with the phone number supplied by the HF
*1*
1
0
0
*100%*
Call waiting notification - Three way calling
*6*
6
0
0
*100%*
Incoming voice call interrupted by remote
*1*
1
0
0
*100%*
Echo canceling (EC) and noise reduction (NR)
*1*
1
0
0
*100%*
Attach a phone number to a voice tag
*1*
0
0
1
*0%*
Ability to transmit DTMF codes
*1*
1
0
0
*100%*
Remote audio speaker volume control
*3*
3
0
0
*100%*
Remote audio micro volume control
*3*
3
0
0
*100%*
Query current call status
*3*
3
0
0
*100%*
Enhanced Call Control
*2*
2
0
0
*100%*
Service Level Connection establishment - Codec Negotiation
*7*
7
0
0
*100%*
Codec Connection set up
*7*
7
0
0
*100%*
Wide Band Speech
*2*
2
0
0
*100%*
Connection management
*2*
2
0
0
*100%*
Connection management - Service Search Request
*3*
3
0
0
*100%*
Place a call with the phone number supplied by the AG
*1*
1
0
0
*100%*
-----------------------------------------------
*Notes***
Please note that one oFono crash still occurs on the following conditions:
-OF-163 <https://01.org/jira/browse/OF-163>- oFono crashes when doing a
NAA Initialization+File Change Notification after having done a NAA
Initialization+Full File Change Notification with Phonesim.
A bug in PTS software (11515
<https://www.bluetooth.org/pts/issues/view_issue.cfm?id=11515> which
sends to oFono the wrong HFP version supported) preventing to execute
several TCs has been avoided during test session.
/ Workaround details:/
/ 1.//Change the HFP version in /var/lib/Bluetooth/<Local mac
address>/cache/<PTS mac address>/
/ 2.//Kill Bluetoothd/
/ 3.//Relaunch Bluetoothd/
/ 4.//Redo the PTS tests that was blocked by this issue./
Best regards,
Nicolas
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 135885 bytes --]
[-- Attachment #3: oFono1.13_7a6da06_BAT_TitlesOnly.xlsx --]
[-- Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, Size: 1509062 bytes --]
[-- Attachment #4: oFono1.13_7a6da06_HFPv1.6_PTS_TCs.xlsx --]
[-- Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, Size: 21439 bytes --]
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2013-11-18 12:04 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-11-18 12:04 oFono upstream test results_20131115 Nicolas Paccou
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.