* [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
@ 2016-01-20 12:58 Jaap Jan Meijer
2016-01-21 11:12 ` Marcel Holtmann
0 siblings, 1 reply; 6+ messages in thread
From: Jaap Jan Meijer @ 2016-01-20 12:58 UTC (permalink / raw)
To: linux-bluetooth
Hi,
WIth Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
kernel. ioctl fails with ENOTTY.
static int attach_proto(const char *path, unsigned int proto,
unsigned int flags)
{
int fd, dev_id;
fd = open_serial(path);
if (fd < 0)
return -1;
if (ioctl(fd, HCIUARTSETFLAGS, flags) < 0) {
perror("Failed to set flags");
close(fd);
return -1;
}
.....
Kind regards,
J.J.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
2016-01-20 12:58 [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit Jaap Jan Meijer
@ 2016-01-21 11:12 ` Marcel Holtmann
2016-01-21 21:38 ` Jaap Jan Meijer
0 siblings, 1 reply; 6+ messages in thread
From: Marcel Holtmann @ 2016-01-21 11:12 UTC (permalink / raw)
To: Jaap Jan Meijer; +Cc: linux-bluetooth
Hi Jaap,
> WIth Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
> kernel. ioctl fails with ENOTTY.
>
> static int attach_proto(const char *path, unsigned int proto,
> unsigned int flags)
> {
> int fd, dev_id;
>
> fd = open_serial(path);
> if (fd < 0)
> return -1;
>
> if (ioctl(fd, HCIUARTSETFLAGS, flags) < 0) {
> perror("Failed to set flags");
> close(fd);
> return -1;
> }
> .....
I am a little bit lost on how you get ENOTTY. The only way I can see this happening if the kernel falls through towards n_tty_ioctl_helper and that means that cmd is messed up.
If you go into drivers/bluetooth/hci_ldisc.c and add this line
hci_uart_ldisc.compat_ioctl = hci_uart_tty_ioctl;
it might fix it. It is a guess right now until you figure out where the ENOTTY is coming from.
Regards
Marcel
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
2016-01-21 11:12 ` Marcel Holtmann
@ 2016-01-21 21:38 ` Jaap Jan Meijer
2016-01-22 1:07 ` Marcel Holtmann
2016-01-22 13:38 ` Marcel Holtmann
0 siblings, 2 replies; 6+ messages in thread
From: Jaap Jan Meijer @ 2016-01-21 21:38 UTC (permalink / raw)
To: Marcel Holtmann; +Cc: linux-bluetooth
Hi Marcel,
Thank you, that was very good guess work. Btattach is now able to
attach by bcm43241 in the mixed arch situation. Should I supply a
patch or will you just add it? I'll will send-email a patch anyway.
Kind regards,
J.J.
2016-01-21 12:12 GMT+01:00 Marcel Holtmann <marcel@holtmann.org>:
> Hi Jaap,
>
>> WIth Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
>> kernel. ioctl fails with ENOTTY.
>>
>> static int attach_proto(const char *path, unsigned int proto,
>> unsigned int flags)
>> {
>> int fd, dev_id;
>>
>> fd = open_serial(path);
>> if (fd < 0)
>> return -1;
>>
>> if (ioctl(fd, HCIUARTSETFLAGS, flags) < 0) {
>> perror("Failed to set flags");
>> close(fd);
>> return -1;
>> }
>> .....
>
> I am a little bit lost on how you get ENOTTY. The only way I can see this happening if the kernel falls through towards n_tty_ioctl_helper and that means that cmd is messed up.
>
> If you go into drivers/bluetooth/hci_ldisc.c and add this line
>
> hci_uart_ldisc.compat_ioctl = hci_uart_tty_ioctl;
>
> it might fix it. It is a guess right now until you figure out where the ENOTTY is coming from.
>
> Regards
>
> Marcel
>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
2016-01-21 21:38 ` Jaap Jan Meijer
@ 2016-01-22 1:07 ` Marcel Holtmann
2016-01-22 13:38 ` Marcel Holtmann
1 sibling, 0 replies; 6+ messages in thread
From: Marcel Holtmann @ 2016-01-22 1:07 UTC (permalink / raw)
To: Jaap Jan Meijer; +Cc: linux-bluetooth
Hi Jaap,
> Thank you, that was very good guess work. Btattach is now able to
> attach by bcm43241 in the mixed arch situation. Should I supply a
> patch or will you just add it? I'll will send-email a patch anyway.
please send a patch for it.
Regards
Marcel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
2016-01-21 21:38 ` Jaap Jan Meijer
2016-01-22 1:07 ` Marcel Holtmann
@ 2016-01-22 13:38 ` Marcel Holtmann
2016-01-25 7:55 ` Marcel Holtmann
1 sibling, 1 reply; 6+ messages in thread
From: Marcel Holtmann @ 2016-01-22 13:38 UTC (permalink / raw)
To: Jaap Jan Meijer; +Cc: linux-bluetooth
Hi Jaap,
> Thank you, that was very good guess work. Btattach is now able to
> attach by bcm43241 in the mixed arch situation. Should I supply a
> patch or will you just add it? I'll will send-email a patch anyway.
I just send a patch that should be more correct on all platforms. Can you try that one out please.
Regards
Marcel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit
2016-01-22 13:38 ` Marcel Holtmann
@ 2016-01-25 7:55 ` Marcel Holtmann
0 siblings, 0 replies; 6+ messages in thread
From: Marcel Holtmann @ 2016-01-25 7:55 UTC (permalink / raw)
To: Jaap Jan Meijer; +Cc: linux-bluetooth
Hi Jaap,
>> Thank you, that was very good guess work. Btattach is now able to
>> attach by bcm43241 in the mixed arch situation. Should I supply a
>> patch or will you just add it? I'll will send-email a patch anyway.
>
> I just send a patch that should be more correct on all platforms. Can you try that one out please.
actually it might be just some missing COMPATIBLE_IOCTL declarations. I just send a patch for that. The patch is correct by itself, but it might actually fix your issue and no .compat_ioctl is needed.
Regards
Marcel
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-01-25 7:55 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-01-20 12:58 [BUG] Btattach, ioctl fails to set flags from 32-bit user space to 64-bit Jaap Jan Meijer
2016-01-21 11:12 ` Marcel Holtmann
2016-01-21 21:38 ` Jaap Jan Meijer
2016-01-22 1:07 ` Marcel Holtmann
2016-01-22 13:38 ` Marcel Holtmann
2016-01-25 7:55 ` Marcel Holtmann
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).