[PATCH net-next] net: macb: allow MTU change when the interface is up

[PATCH net-next] net: macb: allow MTU change when the interface is up

[thomas.perrot]

From: Thomas Perrot <thomas.perrot@bootlin.com>

The macb_change_mtu callback doesn't allow MTU changes when the
interface is up because the MTU is used to set the buffer size on
GEM.

So, to be able to change the MTU from userspace it is necessary to
explicitly down then to up the interface, which is problematic in
some cases.

Then we suggest to reset the interface when the interface is already
up to allow to change the MTU, regardless of if the interface is up
or not.

Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
---
 drivers/net/ethernet/cadence/macb_main.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 898debfd4db3..5c20b162c5da 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -3019,11 +3019,19 @@ static int macb_close(struct net_device *dev)
 
 static int macb_change_mtu(struct net_device *dev, int new_mtu)
 {
-	if (netif_running(dev))
-		return -EBUSY;
+	int reset = 0;
+
+	/* On GEM the buffer size is based on the MTU */
+	if (netif_running(dev)) {
+		macb_close(dev);
+		reset = 1;
+	}
 
 	dev->mtu = new_mtu;
 
+	if (reset)
+		macb_open(dev);
+
 	return 0;
 }
 
-- 
2.44.0

Re: [PATCH net-next] net: macb: allow MTU change when the interface is up

[Simon Horman]

On Mon, Mar 25, 2024 at 04:27:35PM +0100, thomas.perrot@bootlin.com wrote:
> From: Thomas Perrot <thomas.perrot@bootlin.com>
> 
> The macb_change_mtu callback doesn't allow MTU changes when the
> interface is up because the MTU is used to set the buffer size on
> GEM.
> 
> So, to be able to change the MTU from userspace it is necessary to
> explicitly down then to up the interface, which is problematic in
> some cases.
> 
> Then we suggest to reset the interface when the interface is already
> up to allow to change the MTU, regardless of if the interface is up
> or not.
> 
> Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>

Hi Thomas,

I'm not sure that it is expected behaviour for an interface
to reset like this when a change of MTU is requested.
While conversely I think it is common (if not entirely desirable)
to prohibit changing the MTU when an interface is up.
What is the problem being addressed here?

> ---
>  drivers/net/ethernet/cadence/macb_main.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> index 898debfd4db3..5c20b162c5da 100644
> --- a/drivers/net/ethernet/cadence/macb_main.c
> +++ b/drivers/net/ethernet/cadence/macb_main.c
> @@ -3019,11 +3019,19 @@ static int macb_close(struct net_device *dev)
>  
>  static int macb_change_mtu(struct net_device *dev, int new_mtu)
>  {
> -	if (netif_running(dev))
> -		return -EBUSY;
> +	int reset = 0;

nit: reset could be bool and set to true and false.

> +
> +	/* On GEM the buffer size is based on the MTU */
> +	if (netif_running(dev)) {
> +		macb_close(dev);
> +		reset = 1;
> +	}
>  
>  	dev->mtu = new_mtu;
>  
> +	if (reset)
> +		macb_open(dev);
> +
>  	return 0;
>  }
>  
> -- 
> 2.44.0
> 
> 

Re: [PATCH net-next] net: macb: allow MTU change when the interface is up

[Jakub Kicinski]

On Mon, 25 Mar 2024 20:54:01 +0000 Simon Horman wrote:
> I'm not sure that it is expected behaviour for an interface
> to reset like this when a change of MTU is requested.
> While conversely I think it is common (if not entirely desirable)
> to prohibit changing the MTU when an interface is up.
> What is the problem being addressed here?

Right..

> >  	dev->mtu = new_mtu;
> >  
> > +	if (reset)
> > +		macb_open(dev);

.. imagine admin does this over SSH on a remote box and system 
is under memory pressure. Even ignoring the fact you're not checking
the return value, the result of changing MTU should be either having 
the requested MTU (success) or having the old MTU (failure).
Not "machine drops off the network" :(

Re: [PATCH net-next] net: macb: allow MTU change when the interface is up

[Thomas Perrot]

[-- Attachment #1: Type: text/plain, Size: 2279 bytes --]

Hello,

On Mon, 2024-03-25 at 18:56 -0700, Jakub Kicinski wrote:
> On Mon, 25 Mar 2024 20:54:01 +0000 Simon Horman wrote:
> > I'm not sure that it is expected behaviour for an interface
> > to reset like this when a change of MTU is requested.
> > While conversely I think it is common (if not entirely desirable)
> > to prohibit changing the MTU when an interface is up.
> > What is the problem being addressed here?
> 

The problem being addressed here, is that NetworkManager isn't able to
apply the MTU value set in the connection configuration file because
the link is already up, then the change_mtu callback returns an error:

"NetworkManager[412]: <debug> [1709629970.1735] platform: (eth0) link:
setting mtu 1400                                                      
NetworkManager[412]: <trace> [1709629970.1737] platform-linux: delayed-
action: schedule wait-for-response-rtnl (seq 41, timeout in
0.199992796, response-type 0)                                         
NetworkManager[412]: <trace> [1709629970.1737] platform-linux: delayed-
action: schedule refresh-link (ifindex 2)                             
NetworkManager[412]: <trace> [1709629970.1738] platform-linux: delayed-
action: handle refresh-link (ifindex 2)                               
NetworkManager[412]: <debug> [1709629970.1738] platform-linux: do-
request-link: 2                                                       
NetworkManager[412]: <trace> [1709629970.1739] platform-linux: rtnl:
recvmsg: new message NLMSG_ERROR, flags 0, seq 41                     
NetworkManager[412]: <debug> [1709629970.1740] platform-linux: rtnl:
recvmsg: error message from kernel: Device or resource busy (-16) for
request 41"

Kind regards,
Thomas Perrot

> Right..
> 
> > >  	dev->mtu = new_mtu;
> > >  
> > > +	if (reset)
> > > +		macb_open(dev);
> 
> .. imagine admin does this over SSH on a remote box and system 
> is under memory pressure. Even ignoring the fact you're not checking
> the return value, the result of changing MTU should be either having 
> the requested MTU (success) or having the old MTU (failure).
> Not "machine drops off the network" :(
> 

-- 
Thomas Perrot, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH net-next] net: macb: allow MTU change when the interface is up

[Simon Horman]

On Tue, Mar 26, 2024 at 09:49:19AM +0100, Thomas Perrot wrote:
> Hello,
> 
> On Mon, 2024-03-25 at 18:56 -0700, Jakub Kicinski wrote:
> > On Mon, 25 Mar 2024 20:54:01 +0000 Simon Horman wrote:
> > > I'm not sure that it is expected behaviour for an interface
> > > to reset like this when a change of MTU is requested.
> > > While conversely I think it is common (if not entirely desirable)
> > > to prohibit changing the MTU when an interface is up.
> > > What is the problem being addressed here?
> > 
> 
> The problem being addressed here, is that NetworkManager isn't able to
> apply the MTU value set in the connection configuration file because
> the link is already up, then the change_mtu callback returns an error:
> 
> "NetworkManager[412]: <debug> [1709629970.1735] platform: (eth0) link:
> setting mtu 1400                                                      
> NetworkManager[412]: <trace> [1709629970.1737] platform-linux: delayed-
> action: schedule wait-for-response-rtnl (seq 41, timeout in
> 0.199992796, response-type 0)                                         
> NetworkManager[412]: <trace> [1709629970.1737] platform-linux: delayed-
> action: schedule refresh-link (ifindex 2)                             
> NetworkManager[412]: <trace> [1709629970.1738] platform-linux: delayed-
> action: handle refresh-link (ifindex 2)                               
> NetworkManager[412]: <debug> [1709629970.1738] platform-linux: do-
> request-link: 2                                                       
> NetworkManager[412]: <trace> [1709629970.1739] platform-linux: rtnl:
> recvmsg: new message NLMSG_ERROR, flags 0, seq 41                     
> NetworkManager[412]: <debug> [1709629970.1740] platform-linux: rtnl:
> recvmsg: error message from kernel: Device or resource busy (-16) for
> request 41"

Hi Thomas,

I understand this is not ideal. But from a driver level perspective, if the
HW doesn't support changing the MTU while the device is up, rejecting the
change MTU operation is actually the best practice (or least worst
depending on your perspective).

As Jakub pointed out elsewhere in this thread, with your proposed change,
there is a real chance the NIC could drop of the network entirely and not
come back. Whereas the expected outcome is for the NIC to remain on the
network with either the new or old MTU.