RFC: changes to microcode update driver.

RFC: changes to microcode update driver.

[Tigran Aivazian]

Hi guys,

It's been a long time that I was going to send to Linux the patch with the
following changes, but the birth of my first child intervened and caused
delays (though can't call them "unexpected" :). Now, if I hear from people
"no, no! we are using this feature!"  then I will reconsider:

1. Remove ->read() method for /dev/cpu/microcode device node and do not
hold a copy of applied microcode chunks in kernel memory. In the days when
we had a regular devfs file with a non-zero size this had at least some
potential use but now this feature is almost useless and removing it would
allow a lot of code cleanup and simplification.

2. remove MICROCODE_IOCFREE ioctl for freeing the copy of held microcode
(because there won't be such copy, see 1.)

Please let me know your thoughts.

Kind regards
Tigran

Re: RFC: changes to microcode update driver.

[Tigran Aivazian]

On Tue, 7 Oct 2003, Tigran Aivazian wrote:
> 2. remove MICROCODE_IOCFREE ioctl for freeing the copy of held microcode
> (because there won't be such copy, see 1.)

Obviously this can't be done before the microcode_ctl userspace utility is
updated. So, for now we can just quietly suceed and only later remove the
actual ->ioctl() method.

Kind regards
Tigran

Re: RFC: changes to microcode update driver.

[Giacomo A. Catenazzi]

Tigran Aivazian wrote:

> Hi guys,
> 
> It's been a long time that I was going to send to Linux the patch with the
> following changes, but the birth of my first child intervened and caused
> delays (though can't call them "unexpected" :). Now, if I hear from people
> "no, no! we are using this feature!"  then I will reconsider:

> Please let me know your thoughts.

Hello.

I would like to have a common model to load microcodes (and in the new boot 
procedure), but I've lost the status of such project, I think things for 2.7.
Any news?

Is microcode_ctl still maintained? I've made some correction/extentions but now 
new from the maintainer.

Intel give us the new microcode? I had contact with the new contact/maintainer/? 
person in Intel, but still no new microcode since summer 2001. So maybe before 
changing the driver, could you check the Intel vision about Linux and microcode?

ciao
	giacomo

Re: RFC: changes to microcode update driver.

[Tigran Aivazian]

On Tue, 7 Oct 2003, Giacomo A. Catenazzi wrote:
> Is microcode_ctl still maintained? I've made some correction/extentions but now
> new from the maintainer.

Yes, I believe Simon is alive and well. (but busy, as we all are)

> Intel give us the new microcode? I had contact with the new contact/maintainer/?
> person in Intel, but still no new microcode since summer 2001. So maybe before
> changing the driver, could you check the Intel vision about Linux and microcode?

I am communicating with Intel guys from time to time and there are some
interesting changes from Intel in the pipeline to update the driver, but I
thought it is worthwhile to cleanup and throw away unnecessary bits before
applying a major update (otherwise we would be wasting time debugging an
update to code which is no longer needed).

As for the microcode data itself, no, I haven't received anything new from
Intel yet but please be patient. I have received unofficial latest
"hacked"  version of microcode data from someone (outside Intel) but it
will not be uploaded because it will cause support problems both to Intel
and myself.

I find it is wiser to be friendly with Intel than to annoy them with
constant questions "where is the latest microcode data" :)

Kind regards
Tigran

Re: RFC: changes to microcode update driver.

[Dave Jones]

On Tue, Oct 07, 2003 at 03:58:00AM -0700, Tigran Aivazian wrote:
 > Hi guys,
 > 
 > It's been a long time that I was going to send to Linux the patch with the
 > following changes, but the birth of my first child intervened and caused
 > delays (though can't call them "unexpected" :)

Congrats 8-)

 > 1. Remove ->read() method for /dev/cpu/microcode device node and do not
 > hold a copy of applied microcode chunks in kernel memory. In the days when
 > we had a regular devfs file with a non-zero size this had at least some
 > potential use but now this feature is almost useless and removing it would
 > allow a lot of code cleanup and simplification.
 > 
 > 2. remove MICROCODE_IOCFREE ioctl for freeing the copy of held microcode
 > (because there won't be such copy, see 1.)

Assuming that it can be done without the old tools breaking, sounds good
to me.  How will microcode_ctl -i react if you remove the ioctl ?
Folks _will_ upgrade kernels without updating userspace.

		Dave

-- 
 Dave Jones     http://www.codemonkey.org.uk

Re: RFC: changes to microcode update driver.

[Tigran Aivazian]

On Tue, 7 Oct 2003, Dave Jones wrote:
> Assuming that it can be done without the old tools breaking, sounds good
> to me.  How will microcode_ctl -i react if you remove the ioctl ?
> Folks _will_ upgrade kernels without updating userspace.

It will fail and Red Hat's startup scripts will show the red [FAILED]
thing which users are always afraid of.

So, I thought initially we should just return 0 in that ioctl (with the 
comment that it's going away) and then remove it completely, after 
microcode_ctl has been updated.

The patch was almost ready (together with Intel's changes) but I 
discovered that microcode module (or in fact ANY module that is loaded 
first on my system, 2.6.0-test6) is not unloadable, i.e. usage count stays 
at 1 even though nothing is using it. I am not aware of this general 
problem being discussed on linux-kernel list, so I thought I should debug 
it first (after all, it may be something I am doing that causes it). And 
yes my kernel is configured to allow unloading, even forced unloading.

(still feeling ashamed after yesterday's thing with me forgetting to 
configure siimage driver and complaining that generic ATA is too slow :)

Kind regards
Tigran

Re: RFC: changes to microcode update driver.

[Mikael Pettersson]

Tigran Aivazian writes:
 > The patch was almost ready (together with Intel's changes) but I 
 > discovered that microcode module (or in fact ANY module that is loaded 
 > first on my system, 2.6.0-test6) is not unloadable, i.e. usage count stays 
 > at 1 even though nothing is using it. I am not aware of this general 

Maybe unrelated, but 2.6.0-test6 has a bug in drivers/char/misc.c's
module autoloading code, which causes device open() failures, and
subsequent unloading failures due to the usage count being 1 too high.

I posted the fix below, and Linus included it in a recent -bk snapshot.

/Mikael

diff -ruN linux-2.6.0-test6/drivers/char/misc.c linux-2.6.0-test6.char-misc-fix/drivers/char/misc.c
--- linux-2.6.0-test6/drivers/char/misc.c	2003-09-28 12:19:40.000000000 +0200
+++ linux-2.6.0-test6.char-misc-fix/drivers/char/misc.c	2003-10-04 14:55:45.000000000 +0200
@@ -157,12 +157,11 @@
 		list_for_each_entry(c, &misc_list, list) {
 			if (c->minor == minor) {
 				new_fops = fops_get(c->fops);
-				if (!new_fops)
-					goto fail;
 				break;
 			}
 		}
-		goto fail;
+		if (!new_fops)
+			goto fail;
 	}
 
 	err = 0;

Re: RFC: changes to microcode update driver.

[Jamie Lokier]

Tigran Aivazian wrote:
> 1. Remove ->read() method for /dev/cpu/microcode device node and do not
> hold a copy of applied microcode chunks in kernel memory. In the days when
> we had a regular devfs file with a non-zero size this had at least some
> potential use but now this feature is almost useless and removing it would
> allow a lot of code cleanup and simplification.

It would be nice to record the MD5 or SHA checksum of the loaded
microcode, so it's possible to check which one was loaded.

-- Jamie

RE: RFC: changes to microcode update driver.

[Nakajima, Jun]

As Tigran pointed out, we are active in this area too. At this point we
want to add support of the extended update format to the driver, before
we ship the latest microcode data. Some of them require the new format.

	Jun
> -----Original Message-----
> From: Tigran Aivazian [mailto:tigran@veritas.com]
> Sent: Tuesday, October 07, 2003 6:48 AM
> To: Giacomo A. Catenazzi
> Cc: linux-kernel@vger.kernel.org; simon@urbanmyth.org; Nakajima, Jun
> Subject: Re: RFC: changes to microcode update driver.
> 
> On Tue, 7 Oct 2003, Giacomo A. Catenazzi wrote:
> > Is microcode_ctl still maintained? I've made some
correction/extentions
> but now
> > new from the maintainer.
> 
> Yes, I believe Simon is alive and well. (but busy, as we all are)
> 
> > Intel give us the new microcode? I had contact with the new
> contact/maintainer/?
> > person in Intel, but still no new microcode since summer 2001. So
maybe
> before
> > changing the driver, could you check the Intel vision about Linux
and
> microcode?
> 
> I am communicating with Intel guys from time to time and there are
some
> interesting changes from Intel in the pipeline to update the driver,
but I
> thought it is worthwhile to cleanup and throw away unnecessary bits
before
> applying a major update (otherwise we would be wasting time debugging
an
> update to code which is no longer needed).
> 
> As for the microcode data itself, no, I haven't received anything new
from
> Intel yet but please be patient. I have received unofficial latest
> "hacked"  version of microcode data from someone (outside Intel) but
it
> will not be uploaded because it will cause support problems both to
Intel
> and myself.
> 
> I find it is wiser to be friendly with Intel than to annoy them with
> constant questions "where is the latest microcode data" :)
> 
> Kind regards
> Tigran
> 
> 
> 
> 

RE: RFC: changes to microcode update driver.

[Maciej Zenczykowski]

> As Tigran pointed out, we are active in this area too. At this point we
> want to add support of the extended update format to the driver, before
> we ship the latest microcode data. Some of them require the new format.

Do we even have any use for a userspace utility anymore?
strace'ing the microcode_ctl -u process results in information that the 
microcode.dat file is converted to binary and written to 
/dev/cpu/microcode.  The following code:
#include <stdio.h>
int main (void) {
  int x[4], i;
  while ((i = scanf("%x, %x, %x, %x,\n", &x[0], &x[1], &x[2], &x[3])) > 0) 
fwrite(x, 4, i, stdout);
  return 0;
};
does the conversion to binary:
cat /etc/microcode.dat | grep -v "^/" | ./a.out > microcode.raw
and the following loads it:
dd bs=`ls -s --block-size=1 microcode.raw | cut -f 1 -d " "` 
if=microcode.raw of=/dev/cpu/microcode

Either distribute the microcode in binary form and load it via dd (in the 
/etc/rc.d/init.d/microcode script)
or include the text file parser in the microcode module - since the module 
is only needed during loading of the update this tiny amount of extra code 
is likely acceptable.  For reducing kernel size for embedded systems (any 
based on ia32 lacking the few kb?) try compiling the 2kb update relevant 
for the given processor directly into the kernel...

Just a few ideas...

MaZe.

RE: RFC: changes to microcode update driver.

[Tigran Aivazian]

On Tue, 7 Oct 2003, Maciej Zenczykowski wrote:

> > As Tigran pointed out, we are active in this area too. At this point we
> > want to add support of the extended update format to the driver, before
> > we ship the latest microcode data. Some of them require the new format.
>
> Do we even have any use for a userspace utility anymore?
> strace'ing the microcode_ctl -u process results in information that the
  ~~~~~~~~~~

If you read the code instead of stracing you would have discovered that
there is a little (i.e. undocummented :) switch to do everything you
described below.

As for compiling the microcode data into the kernel (as scsi firmware etc)
I recall that Alan Cox said we are moving away from that sort of thing,
so it's not a good idea.

Kind regards.
Tigran

> microcode.dat file is converted to binary and written to
> /dev/cpu/microcode.  The following code:
> #include <stdio.h>
> int main (void) {
>   int x[4], i;
>   while ((i = scanf("%x, %x, %x, %x,\n", &x[0], &x[1], &x[2], &x[3])) > 0)
> fwrite(x, 4, i, stdout);
>   return 0;
> };
> does the conversion to binary:
> cat /etc/microcode.dat | grep -v "^/" | ./a.out > microcode.raw
> and the following loads it:
> dd bs=`ls -s --block-size=1 microcode.raw | cut -f 1 -d " "`
> if=microcode.raw of=/dev/cpu/microcode
>
> Either distribute the microcode in binary form and load it via dd (in the
> /etc/rc.d/init.d/microcode script)
> or include the text file parser in the microcode module - since the module
> is only needed during loading of the update this tiny amount of extra code
> is likely acceptable.  For reducing kernel size for embedded systems (any
> based on ia32 lacking the few kb?) try compiling the 2kb update relevant
> for the given processor directly into the kernel...
>
> Just a few ideas...
>
> MaZe.
>
>