RE: [PATCH 00/13] drivers: hv: kvp

RE: [PATCH 00/13] drivers: hv: kvp

[KY Srinivasan]

> -----Original Message-----
> From: Olaf Hering [mailto:olaf@aepfle.de]
> Sent: Thursday, June 28, 2012 10:24 AM
> To: KY Srinivasan
> Cc: Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> virtualization@lists.osdl.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 00/13] drivers: hv: kvp
> 
> On Tue, Jun 26, KY Srinivasan wrote:
> 
> > > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > > The fact that it was Red Hat specific was the main part, this should be
> > > done in a standard way, with standard tools, right?
> >
> > The reason I asked this question was to make sure I address these
> > issues in addition to whatever I am debugging now. I use the standard
> > tools and calls to retrieve all the IP configuration. As I look at
> > each distribution the files they keep persistent IP configuration
> > Information is different and that is the reason I chose to start with
> > RedHat. If there is a standard way to store the configuration, I will
> > do that.
> 
> 
> KY,
> 
> instead of using system() in kvp_get_ipconfig_info and kvp_set_ip_info,
> wouldnt it be easier to call an external helper script which does all
> the distribution specific work? Just define some API to pass values to
> the script, and something to read values collected by the script back
> into the daemon.

On the "Get" side I mostly use standard commands/APIs to get all the information:

1) IP address information and subnet mask: getifaddrs()
2) DNS information:  Parsing /etc/resolv.conf
3) /sbin/ip command for all the routing information
4)  Parse /etc/sysconfig/network-scripts/ifcfg-ethx for boot protocol

As you can see, all but the boot protocol is gathered using the "standard distro
independent mechanisms. I was looking at NetworkManager cli and it looks
like I could gather all the information except the boot protocol information. I am 
not sure how to gather the boot protocol information in a distro independent fashion.

On the SET side, I need to persistently store the settings in an appropriate configuration
file and flush these settings down so that the interface is appropriately configured. It is here
that I am struggling to find a distro independent way of doing things. It would be great if I can
use NetworkManager cli (nmcli) to accomplish this. Any help here would be greatly appreciated.

While I toyed with your proposal, I feel it just pushes the problem out of the daemon code -
we would still need to write distro specific scripts. If this approach is something that everybody
is comfortable with, I can take a stab at implementing that. 

> 
> If the work is done in a script it will be much easier for an admin to
> debug and adjust it.
> 
> I think there is no standard way to configure all relevant distros in
> the same way. Maybe one day NetworkManager can finally handle all
> possible ways to configure network related things. But until that
> happens the config files need to be adjusted manually.
> 
> 
> 
> Some of the functions have deep indention levels due to 'while() {
> switch() }' usage. Perhaps such code could be moved into its own
> function so that lines dont need to be wrapped that much due to the odd
> 80 column limit.

I will take care of this. As suggested by Greg, I am adding netdev developers here to
seek their input. 

Regards,

K. Y

Re: [PATCH 00/13] drivers: hv: kvp

[Ben Hutchings]

On Mon, Jul 02, 2012 at 03:22:25PM +0000, KY Srinivasan wrote:
> 
> 
> > -----Original Message-----
> > From: Olaf Hering [mailto:olaf@aepfle.de]
> > Sent: Thursday, June 28, 2012 10:24 AM
> > To: KY Srinivasan
> > Cc: Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> > virtualization@lists.osdl.org; linux-kernel@vger.kernel.org
> > Subject: Re: [PATCH 00/13] drivers: hv: kvp
> > 
> > On Tue, Jun 26, KY Srinivasan wrote:
> > 
> > > > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > > > The fact that it was Red Hat specific was the main part, this should be
> > > > done in a standard way, with standard tools, right?
> > >
> > > The reason I asked this question was to make sure I address these
> > > issues in addition to whatever I am debugging now. I use the standard
> > > tools and calls to retrieve all the IP configuration. As I look at
> > > each distribution the files they keep persistent IP configuration
> > > Information is different and that is the reason I chose to start with
> > > RedHat. If there is a standard way to store the configuration, I will
> > > do that.
> > 
> > 
> > KY,
> > 
> > instead of using system() in kvp_get_ipconfig_info and kvp_set_ip_info,
> > wouldnt it be easier to call an external helper script which does all
> > the distribution specific work? Just define some API to pass values to
> > the script, and something to read values collected by the script back
> > into the daemon.
> 
> On the "Get" side I mostly use standard commands/APIs to get all the information:
> 
> 1) IP address information and subnet mask: getifaddrs()
> 2) DNS information:  Parsing /etc/resolv.conf
> 3) /sbin/ip command for all the routing information

If you're interested in the *current* configuration then (1) and (3)
are OK but you should really use the rtnetlink API.

However, I suspect that Hyper-V assumes that current and persistent
configuration are the same thing, which is obviously not true in
general on Linux.  But if NetworkManager is running then you can
assume they are.

> 4)  Parse /etc/sysconfig/network-scripts/ifcfg-ethx for boot protocol
> 
> As you can see, all but the boot protocol is gathered using the "standard distro
> independent mechanisms. I was looking at NetworkManager cli and it looks
> like I could gather all the information except the boot protocol information. I am 
> not sure how to gather the boot protocol information in a distro independent fashion.
> 
> On the SET side, I need to persistently store the settings in an appropriate configuration
> file and flush these settings down so that the interface is appropriately configured. It is here
> that I am struggling to find a distro independent way of doing things. It would be great if I can
> use NetworkManager cli (nmcli) to accomplish this. Any help here would be greatly appreciated.
[...]

What was wrong with the NetworkManager D-Bus API I pointed you at?
I don't see how it makes sense to use nmcli as an API.

Ben.

-- 
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
                                                              - Albert Camus

Re: [PATCH 00/13] drivers: hv: kvp

[Olaf Hering]

On Mon, Jul 02, KY Srinivasan wrote:

> While I toyed with your proposal, I feel it just pushes the problem
> out of the daemon code - we would still need to write distro specific
> scripts. If this approach is something that everybody is comfortable
> with, I can take a stab at implementing that. 

Until NetworkManager is feature complete and until every distro is using
NetworkManager per default the kvp_daemon needs distro specific code to
get and set network related settings.
Doing it with an external script will simplify debugging and changes to
the code.

Olaf

Re: [PATCH 00/13] drivers: hv: kvp

[Stephen Hemminger]

> On Mon, Jul 02, KY Srinivasan wrote:
> 
> > While I toyed with your proposal, I feel it just pushes the problem
> > out of the daemon code - we would still need to write distro
> > specific
> > scripts. If this approach is something that everybody is
> > comfortable
> > with, I can take a stab at implementing that.
> 
> Until NetworkManager is feature complete and until every distro is
> using
> NetworkManager per default the kvp_daemon needs distro specific code
> to
> get and set network related settings.
> Doing it with an external script will simplify debugging and changes
> to
> the code.

Although,  Network Manager is a good tool for what it does;
it is not appropriate for every distro. It is overkill
in embedded systems, and it's GUI dependency makes it unmanageable
on servers. 

RE: [PATCH 00/13] drivers: hv: kvp

[KY Srinivasan]

> -----Original Message-----
> From: Ben Hutchings [mailto:ben@decadent.org.uk]
> Sent: Monday, July 02, 2012 3:57 PM
> To: KY Srinivasan
> Cc: Olaf Hering; Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> virtualization@lists.osdl.org; linux-kernel@vger.kernel.org;
> netdev@vger.kernel.org
> Subject: Re: [PATCH 00/13] drivers: hv: kvp
> 
> On Mon, Jul 02, 2012 at 03:22:25PM +0000, KY Srinivasan wrote:
> >
> >
> > > -----Original Message-----
> > > From: Olaf Hering [mailto:olaf@aepfle.de]
> > > Sent: Thursday, June 28, 2012 10:24 AM
> > > To: KY Srinivasan
> > > Cc: Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> > > virtualization@lists.osdl.org; linux-kernel@vger.kernel.org
> > > Subject: Re: [PATCH 00/13] drivers: hv: kvp
> > >
> > > On Tue, Jun 26, KY Srinivasan wrote:
> > >
> > > > > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > > > > The fact that it was Red Hat specific was the main part, this should be
> > > > > done in a standard way, with standard tools, right?
> > > >
> > > > The reason I asked this question was to make sure I address these
> > > > issues in addition to whatever I am debugging now. I use the standard
> > > > tools and calls to retrieve all the IP configuration. As I look at
> > > > each distribution the files they keep persistent IP configuration
> > > > Information is different and that is the reason I chose to start with
> > > > RedHat. If there is a standard way to store the configuration, I will
> > > > do that.
> > >
> > >
> > > KY,
> > >
> > > instead of using system() in kvp_get_ipconfig_info and kvp_set_ip_info,
> > > wouldnt it be easier to call an external helper script which does all
> > > the distribution specific work? Just define some API to pass values to
> > > the script, and something to read values collected by the script back
> > > into the daemon.
> >
> > On the "Get" side I mostly use standard commands/APIs to get all the
> information:
> >
> > 1) IP address information and subnet mask: getifaddrs()
> > 2) DNS information:  Parsing /etc/resolv.conf
> > 3) /sbin/ip command for all the routing information
> 
> If you're interested in the *current* configuration then (1) and (3)
> are OK but you should really use the rtnetlink API.
> 
> However, I suspect that Hyper-V assumes that current and persistent
> configuration are the same thing, which is obviously not true in
> general on Linux.  But if NetworkManager is running then you can
> assume they are.

I am only interested in the currently active information. Why do you
recommend the use of rtnetlink API over the "ip" command. If I am not
mistaken, the ip command uses netlink to get the information. 

> 
> > 4)  Parse /etc/sysconfig/network-scripts/ifcfg-ethx for boot protocol

This is the only information that requires parsing a distro specific configuration file. Do
you have any suggestion on how I may get this information in a distro independent way.
 
> >
> > As you can see, all but the boot protocol is gathered using the "standard distro
> > independent mechanisms. I was looking at NetworkManager cli and it looks
> > like I could gather all the information except the boot protocol information. I am
> > not sure how to gather the boot protocol information in a distro independent
> fashion.
> >
> > On the SET side, I need to persistently store the settings in an appropriate
> configuration
> > file and flush these settings down so that the interface is appropriately
> configured. It is here
> > that I am struggling to find a distro independent way of doing things. It would
> be great if I can
> > use NetworkManager cli (nmcli) to accomplish this. Any help here would be
> greatly appreciated.
> [...]
> 
> What was wrong with the NetworkManager D-Bus API I pointed you at?
> I don't see how it makes sense to use nmcli as an API.

I saw some documentation that claimed that nmcli could be used to accomplish
all that can be done with the GUI interface. I am looking for a portable way
to accomplish configuring an interface. If nmcli can do that, I would use it. With
regards to D-BUS API, I took a cursory look at the APIs. I am still evaluating
my options.

Regards,

K. Y

RE: [PATCH 00/13] drivers: hv: kvp

[KY Srinivasan]

> -----Original Message-----
> From: Olaf Hering [mailto:olaf@aepfle.de]
> Sent: Tuesday, July 03, 2012 9:21 AM
> To: KY Srinivasan
> Cc: Greg KH; apw@canonical.com; devel@linuxdriverproject.org; linux-
> kernel@vger.kernel.org; netdev@vger.kernel.org
> Subject: Re: [PATCH 00/13] drivers: hv: kvp
> 
> On Mon, Jul 02, KY Srinivasan wrote:
> 
> > While I toyed with your proposal, I feel it just pushes the problem
> > out of the daemon code - we would still need to write distro specific
> > scripts. If this approach is something that everybody is comfortable
> > with, I can take a stab at implementing that.
> 
> Until NetworkManager is feature complete and until every distro is using
> NetworkManager per default the kvp_daemon needs distro specific code to
> get and set network related settings.
> Doing it with an external script will simplify debugging and changes to
> the code.

Fair enough. I will keep my current implementation of the GET operation as is since
it is distro independent. On the SET side, I will implement a script as you have suggested.

Regards,

K. Y

> 
> Olaf
> 
> 

RE: [PATCH 00/13] drivers: hv: kvp

[KY Srinivasan]

> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen.hemminger@vyatta.com]
> Sent: Tuesday, July 03, 2012 11:03 AM
> To: Olaf Hering
> Cc: Greg KH; apw@canonical.com; devel@linuxdriverproject.org; linux-
> kernel@vger.kernel.org; netdev@vger.kernel.org; KY Srinivasan
> Subject: Re: [PATCH 00/13] drivers: hv: kvp
> 
> 
> > On Mon, Jul 02, KY Srinivasan wrote:
> >
> > > While I toyed with your proposal, I feel it just pushes the problem
> > > out of the daemon code - we would still need to write distro
> > > specific
> > > scripts. If this approach is something that everybody is
> > > comfortable
> > > with, I can take a stab at implementing that.
> >
> > Until NetworkManager is feature complete and until every distro is
> > using
> > NetworkManager per default the kvp_daemon needs distro specific code
> > to
> > get and set network related settings.
> > Doing it with an external script will simplify debugging and changes
> > to
> > the code.
> 
> Although,  Network Manager is a good tool for what it does;
> it is not appropriate for every distro. It is overkill
> in embedded systems, and it's GUI dependency makes it unmanageable
> on servers.

Thanks Stephen. I will retain the code that I currently have for the "GET" side and
I will implement a script as Olaf suggested that can be distro specific to implement
the SET operation.

Regards,

K. Y

Re: [PATCH 00/13] drivers: hv: kvp

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 3244 bytes --]

On Tue, 2012-07-03 at 15:24 +0000, KY Srinivasan wrote:
> 
> > -----Original Message-----
> > From: Ben Hutchings [mailto:ben@decadent.org.uk]
> > Sent: Monday, July 02, 2012 3:57 PM
> > To: KY Srinivasan
> > Cc: Olaf Hering; Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> > virtualization@lists.osdl.org; linux-kernel@vger.kernel.org;
> > netdev@vger.kernel.org
> > Subject: Re: [PATCH 00/13] drivers: hv: kvp
> > 
> > On Mon, Jul 02, 2012 at 03:22:25PM +0000, KY Srinivasan wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Olaf Hering [mailto:olaf@aepfle.de]
> > > > Sent: Thursday, June 28, 2012 10:24 AM
> > > > To: KY Srinivasan
> > > > Cc: Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> > > > virtualization@lists.osdl.org; linux-kernel@vger.kernel.org
> > > > Subject: Re: [PATCH 00/13] drivers: hv: kvp
> > > >
> > > > On Tue, Jun 26, KY Srinivasan wrote:
> > > >
> > > > > > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > > > > > The fact that it was Red Hat specific was the main part, this should be
> > > > > > done in a standard way, with standard tools, right?
> > > > >
> > > > > The reason I asked this question was to make sure I address these
> > > > > issues in addition to whatever I am debugging now. I use the standard
> > > > > tools and calls to retrieve all the IP configuration. As I look at
> > > > > each distribution the files they keep persistent IP configuration
> > > > > Information is different and that is the reason I chose to start with
> > > > > RedHat. If there is a standard way to store the configuration, I will
> > > > > do that.
> > > >
> > > >
> > > > KY,
> > > >
> > > > instead of using system() in kvp_get_ipconfig_info and kvp_set_ip_info,
> > > > wouldnt it be easier to call an external helper script which does all
> > > > the distribution specific work? Just define some API to pass values to
> > > > the script, and something to read values collected by the script back
> > > > into the daemon.
> > >
> > > On the "Get" side I mostly use standard commands/APIs to get all the
> > information:
> > >
> > > 1) IP address information and subnet mask: getifaddrs()
> > > 2) DNS information:  Parsing /etc/resolv.conf
> > > 3) /sbin/ip command for all the routing information
> > 
> > If you're interested in the *current* configuration then (1) and (3)
> > are OK but you should really use the rtnetlink API.
> > 
> > However, I suspect that Hyper-V assumes that current and persistent
> > configuration are the same thing, which is obviously not true in
> > general on Linux.  But if NetworkManager is running then you can
> > assume they are.
> 
> I am only interested in the currently active information. Why do you
> recommend the use of rtnetlink API over the "ip" command. If I am not
> mistaken, the ip command uses netlink to get the information. 
[...]

'Screen-scraping' the output of administrative tools is not good
practice.  It may be the best you can do when writing a shell script,
but for a C program it's generaly less reliable and often more difficult
than using the underlying C API.

Ben.

-- 
Ben Hutchings
73.46% of all statistics are made up.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

RE: [PATCH 00/13] drivers: hv: kvp

[KY Srinivasan]

> -----Original Message-----
> From: Ben Hutchings [mailto:ben@decadent.org.uk]
> Sent: Saturday, July 21, 2012 10:51 PM
> To: KY Srinivasan
> Cc: Olaf Hering; Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> virtualization@lists.osdl.org; linux-kernel@vger.kernel.org;
> netdev@vger.kernel.org
> Subject: Re: [PATCH 00/13] drivers: hv: kvp
> 
> On Tue, 2012-07-03 at 15:24 +0000, KY Srinivasan wrote:
> >
> > > -----Original Message-----
> > > From: Ben Hutchings [mailto:ben@decadent.org.uk]
> > > Sent: Monday, July 02, 2012 3:57 PM
> > > To: KY Srinivasan
> > > Cc: Olaf Hering; Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> > > virtualization@lists.osdl.org; linux-kernel@vger.kernel.org;
> > > netdev@vger.kernel.org
> > > Subject: Re: [PATCH 00/13] drivers: hv: kvp
> > >
> > > On Mon, Jul 02, 2012 at 03:22:25PM +0000, KY Srinivasan wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Olaf Hering [mailto:olaf@aepfle.de]
> > > > > Sent: Thursday, June 28, 2012 10:24 AM
> > > > > To: KY Srinivasan
> > > > > Cc: Greg KH; apw@canonical.com; devel@linuxdriverproject.org;
> > > > > virtualization@lists.osdl.org; linux-kernel@vger.kernel.org
> > > > > Subject: Re: [PATCH 00/13] drivers: hv: kvp
> > > > >
> > > > > On Tue, Jun 26, KY Srinivasan wrote:
> > > > >
> > > > > > > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > > > > > > The fact that it was Red Hat specific was the main part, this should be
> > > > > > > done in a standard way, with standard tools, right?
> > > > > >
> > > > > > The reason I asked this question was to make sure I address these
> > > > > > issues in addition to whatever I am debugging now. I use the standard
> > > > > > tools and calls to retrieve all the IP configuration. As I look at
> > > > > > each distribution the files they keep persistent IP configuration
> > > > > > Information is different and that is the reason I chose to start with
> > > > > > RedHat. If there is a standard way to store the configuration, I will
> > > > > > do that.
> > > > >
> > > > >
> > > > > KY,
> > > > >
> > > > > instead of using system() in kvp_get_ipconfig_info and kvp_set_ip_info,
> > > > > wouldnt it be easier to call an external helper script which does all
> > > > > the distribution specific work? Just define some API to pass values to
> > > > > the script, and something to read values collected by the script back
> > > > > into the daemon.
> > > >
> > > > On the "Get" side I mostly use standard commands/APIs to get all the
> > > information:
> > > >
> > > > 1) IP address information and subnet mask: getifaddrs()
> > > > 2) DNS information:  Parsing /etc/resolv.conf
> > > > 3) /sbin/ip command for all the routing information
> > >
> > > If you're interested in the *current* configuration then (1) and (3)
> > > are OK but you should really use the rtnetlink API.
> > >
> > > However, I suspect that Hyper-V assumes that current and persistent
> > > configuration are the same thing, which is obviously not true in
> > > general on Linux.  But if NetworkManager is running then you can
> > > assume they are.
> >
> > I am only interested in the currently active information. Why do you
> > recommend the use of rtnetlink API over the "ip" command. If I am not
> > mistaken, the ip command uses netlink to get the information.
> [...]
> 
> 'Screen-scraping' the output of administrative tools is not good
> practice.  It may be the best you can do when writing a shell script,
> but for a C program it's generaly less reliable and often more difficult
> than using the underlying C API.

Ben,
Based on the input I have gotten, the consensus appears to be to have external
scripts to both GET and SET IP related configuration information. So, the KVP
daemon will need to parse information returned from these external distro specific
scripts (on the GET side).  While I agree with you that it is good to use C APIs, I currently
have the implementation using the "ip" command and it appears to be quite simple.
Furthermore, given that the information I need to pass back to the host needs to be
appropriately formatted (based on host specified format), I suspect using the "ip" command
may actually simplify the code.

Regards,

K. Y