[Qemu-devel] slirp-smb broken with Samba 4.1

[Qemu-devel] slirp-smb broken with Samba 4.1

[Jan Kiszka]

Hi Michael,

writing to you as you provided a fix for the last related issue:

I just noticed that the samba-based share is broken again with smbd
version 4.1.11. Tried to look briefly at it, realized that it is a
permission thing (different error when qemu runs as root) but also some
more nasty problem with configuring r/w guest access to a share. If you
have experience in that area, maybe you have an idea.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] slirp-smb broken with Samba 4.1

[Michael Tokarev]

On 10/24/2014 08:55 PM, Jan Kiszka wrote:
> Hi Michael,
> 
> writing to you as you provided a fix for the last related issue:
> 
> I just noticed that the samba-based share is broken again with smbd
> version 4.1.11. Tried to look briefly at it, realized that it is a
> permission thing (different error when qemu runs as root) but also some
> more nasty problem with configuring r/w guest access to a share. If you
> have experience in that area, maybe you have an idea.

Oh well.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=747636#15 -- note the
date.  I don't know if that my old analisys is correct and whenever
anything has changed since that time...

Not that it helps much, I know, but...

Thanks,

/mjt

Re: [Qemu-devel] slirp-smb broken with Samba 4.1

[Michael Tokarev]

On 10/24/2014 10:29 PM, Michael Tokarev wrote:
> On 10/24/2014 08:55 PM, Jan Kiszka wrote:
>> Hi Michael,
>>
>> writing to you as you provided a fix for the last related issue:
>>
>> I just noticed that the samba-based share is broken again with smbd
>> version 4.1.11. Tried to look briefly at it, realized that it is a
>> permission thing (different error when qemu runs as root) but also some
>> more nasty problem with configuring r/w guest access to a share. If you
>> have experience in that area, maybe you have an idea.
> 
> Oh well.
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=747636#15 -- note the
> date.  I don't know if that my old analisys is correct and whenever
> anything has changed since that time...

Actually it seems wrong.

My 4.1.11 version tries to open /var/log/samba/log.smbd.bin file and
gives up.  After I created that file and chowned it to me it works.

I'm trying to find out what tells smbd to use that .bin file...

What error are you getting when run it as root?

/mjt

Re: [Qemu-devel] slirp-smb broken with Samba 4.1

[Michael Tokarev]

On 10/24/2014 11:58 PM, Michael Tokarev wrote:

> My 4.1.11 version tries to open /var/log/samba/log.smbd.bin file and
> gives up.  After I created that file and chowned it to me it works.

(that .bin comes from /me renaming smbd to smbd.bin to run it under
strace from a tiny shell wrapper under original name).

I just sent a small patch which adds a command-line switch to smbd.
It looks like if smbd needs to log something before it parses smb.conf,
it will log to /var/log/samba/log.$progname, but if that fails smbd
will exit.

And together with samba4 something else has been upgraded on my
system (or maybe samba started to load/init more stuff), so now when
smbd loads its libraries and initializes stuff it tries to warn about
obsolete name /etc/pkcs11/modules/gnome-keyring-module and does that
_before_ reading smb.conf.

That patch makes my samba to work.

So my conclusion about missing inetd mode was wrong.

But I really wonder if we should run a helper script to set things
up instead of hardcoding it all into qemu binary...  It is so much
easier to modify and debug the script.

> What error are you getting when run it as root?

Is there anything else missing on your side?

/mjt

Re: [Qemu-devel] slirp-smb broken with Samba 4.1

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

On 2014-10-24 22:43, Michael Tokarev wrote:
> But I really wonder if we should run a helper script to set things
> up instead of hardcoding it all into qemu binary...  It is so much
> easier to modify and debug the script.

Yes, seems reasonable. I would review and test a patch if you like to
look into this.

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [Qemu-devel] slirp-smb broken with Samba 4.1

[Peter Wu]

Hi,

On Friday 24 October 2014 18:55:07 Jan Kiszka wrote:
> writing to you as you provided a fix for the last related issue:
> 
> I just noticed that the samba-based share is broken again with smbd
> version 4.1.11. Tried to look briefly at it, realized that it is a
> permission thing (different error when qemu runs as root) but also some
> more nasty problem with configuring r/w guest access to a share. If you
> have experience in that area, maybe you have an idea.

It appears that the issue is solved in Samba 4.2 (tested with
samba-4.2.0rc1-388-ga3b333a). Using that samba version fixes the problem, I'll
try to do a bisect to see where it got fixed in Samba.

As an aside, would it be possible to override the samba binary at runtime,
without compiling? Either an envvar (SMBD) or an option (-net
user,smb=...,smbd=...).

-- 
Kind regards,
Peter
https://lekensteyn.nl

Re: [Qemu-devel] slirp-smb broken with Samba 4.1

[Michael Tokarev]

03.11.2014 15:22, Peter Wu wrote:
[]
> As an aside, would it be possible to override the samba binary at runtime,
> without compiling? Either an envvar (SMBD) or an option (-net
> user,smb=...,smbd=...).

The whole thing needs to be rewritten to use a shell script to create
smb.conf and to call smbd.  It's something I wanted to do for very long
time, the only question is where to put this script so a user can
easily customize it.  In debian we have a rule that anything in /etc
is a conffile, and it gets upgraded (when upgrading the corresponding
package) only if it hasn't been modified by the user.  Everything else
(in /usr/lib, /usr/share etc) gets upgraded unconditionally.  So maybe
storing it in /etc/qemu/run-smbd.sh or somehting like that makes sense.
The rest is a shell code to create smb.conf in a temp dir -- the same
as current code does in qemu, plus all your additions, plus whatever
else might be needed (that script can even query which samba version
is in use and enable one or another option).

But this is definitely not a 2.2 matherial.

Thanks,

/mjt

[Qemu-devel] Improved shared folders support (was: Re: slirp-smb broken with Samba 4.1)

[Peter Wu]

On Monday 03 November 2014 15:58:35 Michael Tokarev wrote:
> 03.11.2014 15:22, Peter Wu wrote:
> []
> > As an aside, would it be possible to override the samba binary at runtime,
> > without compiling? Either an envvar (SMBD) or an option (-net
> > user,smb=...,smbd=...).
> 
> The whole thing needs to be rewritten to use a shell script to create
> smb.conf and to call smbd.  It's something I wanted to do for very long
> time, the only question is where to put this script so a user can
> easily customize it.  In debian we have a rule that anything in /etc
> is a conffile, and it gets upgraded (when upgrading the corresponding
> package) only if it hasn't been modified by the user.  Everything else
> (in /usr/lib, /usr/share etc) gets upgraded unconditionally.  So maybe
> storing it in /etc/qemu/run-smbd.sh or somehting like that makes sense.

I do not think that everyone is (or wants to be) a Samba wizard. For
that reason, I see little value in giving the user too much control over
the creation of the samba config file. There exists helpers for setting up
bridges, but that is because it may need more privileges, and starting samba is
not such a case.

> The rest is a shell code to create smb.conf in a temp dir -- the same
> as current code does in qemu, plus all your additions,

If by "my additions" you mean specifying a custom samba path, that is
just a single program, why not allow to override it at runtime?

> plus whatever else might be needed (that script can even query which
> samba version is in use and enable one or another option).

That single program could be a wrapper script that modifies the files as
needed.

I propose to keep a simple interface which do not need users to modify a
shell script or learn Samba. Consider Virtual Box' and VMWare' shared
folders functionality. It allows multiple folders to be specified with
these properties:

 - share name,
 - folder to be shared,
 - read-only,
 - automount it in the guest (feature of VBox Guest additions),
 - disable after session (after reboot/poweroff, VMWare feature).

Only the first three options are interesting enough for QEMU's
interface. What about extending the current '-net user,smb' option as
follows (lines are wrapped for readability):

    smb=[sharename:]folder,
    [smb=sharename2:folder2[:readonly]] (etc)

If 'sharename' is omitted, 'qemu' is used. Duplicate share names must
result in a fatal error. 'readonly' requires specification of the
sharename to avoid an ambiguity. Further share options could be added
in the future as needed using ':' as separator (the comma is already
parsed by -net).

Alternative option (defaults to rw:):

    smb=[rw:|ro:][sharename:]folder,
    smb=[rw:|ro:][sharename2:]folder2 (etc)

It would also be nice if a QMP/HMP was also available to add shares on
runtime. In the current implementation, that could be done by writing a
new config file, renaming it to smb.conf when done. Samba appears to
have support to do this without restarting samba, but I have no idea
whether it can be used in the current implementation where
communication happens through stdin/stdout.

By the way, how well does '-net user,smb' work with multiple
connections? The documentation for guestfwd suggests that the command
gets executed for each connection. Won't that cause concurrency issues
in the access to databases?
-- 
Kind regards,
Peter
https://lekensteyn.nl

Re: [Qemu-devel] Improved shared folders support

[Michael Tokarev]

03.11.2014 22:46, Peter Wu wrote:
> On Monday 03 November 2014 15:58:35 Michael Tokarev wrote:
>> 03.11.2014 15:22, Peter Wu wrote:
>> []
>>> As an aside, would it be possible to override the samba binary at runtime,
>>> without compiling? Either an envvar (SMBD) or an option (-net
>>> user,smb=...,smbd=...).
>>
>> The whole thing needs to be rewritten to use a shell script to create
>> smb.conf and to call smbd.  It's something I wanted to do for very long
>> time, the only question is where to put this script so a user can
>> easily customize it.  In debian we have a rule that anything in /etc
>> is a conffile, and it gets upgraded (when upgrading the corresponding
>> package) only if it hasn't been modified by the user.  Everything else
>> (in /usr/lib, /usr/share etc) gets upgraded unconditionally.  So maybe
>> storing it in /etc/qemu/run-smbd.sh or somehting like that makes sense.
> 
> I do not think that everyone is (or wants to be) a Samba wizard. For
> that reason, I see little value in giving the user too much control over
> the creation of the samba config file. There exists helpers for setting up
> bridges, but that is because it may need more privileges, and starting samba is
> not such a case.

The talk is not about being a wizard or relying on users creating that
script.  The talk is about letting user a quick way to fix stuff with
new or unusual samba combination, without a need to recompile qemu
binary.  Changing a shell script is trivial, including trial and error
(so you dont really need to be a wizard), while recompiling is much
more difficult.

>> The rest is a shell code to create smb.conf in a temp dir -- the same
>> as current code does in qemu, plus all your additions,
> 
> If by "my additions" you mean specifying a custom samba path, that is
> just a single program, why not allow to override it at runtime?

No, I mean the other patch - the printing one you sent the other day.

[]
> That single program could be a wrapper script that modifies the files as
> needed.
> 
> I propose to keep a simple interface which do not need users to modify a
> shell script or learn Samba. Consider Virtual Box' and VMWare' shared
> folders functionality. It allows multiple folders to be specified with
> these properties:

Again, my variant does not mean users will need to learn anything, it
is only needed in emergency cases.  My proposal is to move all current
smb.conf file creation from qemu binary to an external shell script
which is *shipped by qemu*, and which creates the same setup as currently
done by qemu binary.  But with it being a shell script, it will be easy
to modify it *if needed*.  Modifying already created smb.conf is not a
good idea, that requires even more wizardy skills.

[skip changing user interface, it is for another email]
[]
> By the way, how well does '-net user,smb' work with multiple
> connections? The documentation for guestfwd suggests that the command
> gets executed for each connection. Won't that cause concurrency issues
> in the access to databases?

I for one never tried that ;)

/mjt

Re: [Qemu-devel] Improved shared folders support

[Peter Wu]

On Tuesday 04 November 2014 09:13:31 Michael Tokarev wrote:
> >> The whole thing needs to be rewritten to use a shell script to create
> >> smb.conf and to call smbd.  It's something I wanted to do for very long
> >> time, the only question is where to put this script so a user can
> >> easily customize it.  In debian we have a rule that anything in /etc
> >> is a conffile, and it gets upgraded (when upgrading the corresponding
> >> package) only if it hasn't been modified by the user.  Everything else
> >> (in /usr/lib, /usr/share etc) gets upgraded unconditionally.  So maybe
> >> storing it in /etc/qemu/run-smbd.sh or somehting like that makes sense.
> > 
> > I do not think that everyone is (or wants to be) a Samba wizard. For
> > that reason, I see little value in giving the user too much control over
> > the creation of the samba config file. There exists helpers for setting up
> > bridges, but that is because it may need more privileges, and starting samba is
> > not such a case.
> 
> The talk is not about being a wizard or relying on users creating that
> script.  The talk is about letting user a quick way to fix stuff with
> new or unusual samba combination, without a need to recompile qemu
> binary.  Changing a shell script is trivial, including trial and error
> (so you dont really need to be a wizard), while recompiling is much
> more difficult.

So this is more to workaround bugs in Samba should they occur? I
previously mentioned a possible smbd=/path/to/smbd option, but it could
also become a smbhelper=/path/to/helper.sh. Possible options:

 - Generate the smb.conf and pass the temp dir containing it to that
   helper. smbd is started by the helper.
 - Do not create smb.conf nor create a tempdir, let the helper control
   this. smbd is started by the helper.
 - Same as previous, but allow smb and smbhelper to co-exist and pass
   the smb option via an envvar to smbhelper.

> > That single program could be a wrapper script that modifies the files as
> > needed.
> > 
> > I propose to keep a simple interface which do not need users to modify a
> > shell script or learn Samba. Consider Virtual Box' and VMWare' shared
> > folders functionality. It allows multiple folders to be specified with
> > these properties:
> 
> Again, my variant does not mean users will need to learn anything, it
> is only needed in emergency cases.  My proposal is to move all current
> smb.conf file creation from qemu binary to an external shell script
> which is *shipped by qemu*, and which creates the same setup as currently
> done by qemu binary.  But with it being a shell script, it will be easy
> to modify it *if needed*.

If Samba options have be to be modified in this way, then that is a bug
in QEMU and must be fixed (consider the Samba config an implementation
detail which does not need to be exposed to the user in one way or
another).

I would like to have the ability to let QEMU generate the config file
*without* having to rely on an external script with a hard-coded path.
Consider out-of-tree builds and testing such binaries. If a helper is
wanted, see above for a possible smbhelper.

> Modifying already created smb.conf is not a good idea, that requires
> even more wizardy skills.

For manual *testing* you can actually modify the smb.conf between
invocations of smbd (that is, before the guest accesses port 139 or
445). I actually did this to add "log level". In the proposed extension
of the smb option I mentioned rewriting smb.conf on config changes, but
that would then be done by QEMU, not the user.

Btw, what do you think about the proposed smb extension to allow more
shares with custom names?
-- 
Kind regards,
Peter
https://lekensteyn.nl

Re: [Qemu-devel] slirp-smb broken with Samba 4.1

[Peter Wu]

On Monday 03 November 2014 13:22:19 Peter Wu wrote:
> On Friday 24 October 2014 18:55:07 Jan Kiszka wrote:
> > writing to you as you provided a fix for the last related issue:
> > 
> > I just noticed that the samba-based share is broken again with smbd
> > version 4.1.11. Tried to look briefly at it, realized that it is a
> > permission thing (different error when qemu runs as root) but also some
> > more nasty problem with configuring r/w guest access to a share. If you
> > have experience in that area, maybe you have an idea.
> 
> It appears that the issue is solved in Samba 4.2 (tested with
> samba-4.2.0rc1-388-ga3b333a). Using that samba version fixes the
> problem, I'll try to do a bisect to see where it got fixed in Samba.

Unfortunately I was wrong. I have tested it by forwarding a smbd daemon
which listens on a port, not one that runs in inetd mode. The referenced
version was still broken. The earliest version I tested in the Samba 4
series is samba-4.0.0alpha17 and that is still broken.

I have reported the issue here, it turns out to affect instances running
as non-root user:
https://bugzilla.samba.org/show_bug.cgi?id=10919
-- 
Kind regards,
Peter
https://lekensteyn.nl