Memory coherency on MPC8272

Memory coherency on MPC8272

[Dmitri Petchkine]

Hi All,

I am working on a user-space application interacting with the security 
engine on MPC8272, which runs Linux 2.6.10_mvl401 (Montavista Pro 
4.0.1). The driver (version 1.2 22-Feb-2006) for the security engine is 
written by Freescale.

The application calls the driver through ioctl() to perform a crypto 
operation. It is non-blocking call, so the app sits and waits for the 
result when it becomes available. Once the driver notifies the app via a 
signal, the latter does another ioctl() to get the result back to the 
user space from the kernel-space.

Sometimes the returned buffer is just all zeroes which is incorrect.

Here is the piece of code in the driver which copies the data back to 
the user space:

sec1_ioctl.c:
int SEC1_ioctl(struct inode  *nd,
                struct file   *fil,
                unsigned int   ioctlCode,
                unsigned long  param)
{
...
         case IOCTL_COPYTO:
             mem = (MALLOC_REQ *)param;
             mem->pid = current->pid;
             copy_to_user(mem->to, mem->from, mem->sz);
             status = SEC1_SUCCESS;
             break;
...
}

The definition of MALLOC_REQ (file Sec1.h):

typedef struct
{
     unsigned long sz;   /* Number of bytes to allocate
                            Zero means to use the default.  A value
                            of zero can be used to avoid fragmentation. */
     void         *ptr;  /* Pointer to the adress that is to
                            be returned by a call to KernelMalloc()
                            or a pointer to an address that is to
                            be freed when calling KernelFree()   */
     char         *to;   /* copy to pointer */
     char         *from; /* copy from pointer*/
     int           pid;  /* pid of requestor */
} MALLOC_REQ;


The problem goes away if I add printing of some buffer (it can be 
mem->to or mem->from, or even another, unrelated buffer) AFTER the call 
to copy_to_user(), i.e.:

         case IOCTL_COPYTO:
         {   char garbage[128]; /* new line */
             mem = (MALLOC_REQ *)param;
             mem->pid = current->pid;
             copy_to_user(mem->to, mem->from, mem->sz);

             PRINT_HEX( garbage, 128 ); /* new line */

             status = SEC1_SUCCESS;
             break;
          }

My understanding that the security hardware of MPC8272 uses DMA to write 
results into the memory which may cause a coherency problem.

My exposure to such hardware issues is very limited, so I appreciate any 
advice on how to fix it.

Thanks in advance,
Dmitri Pechkin

Re: Memory coherency on MPC8272

[Dan Malek]

On Jun 22, 2007, at 10:28 AM, Dmitri Petchkine wrote:

> My understanding that the security hardware of MPC8272 uses DMA to  
> write
> results into the memory which may cause a coherency problem.

Where did you get such an (incorrect) understanding?
I suggest you acquire your understanding from reading the
MPC8272 reference manual, since it describes the cache
and snooping options between the SEC and memory,
as well as among the SEC units.

> My exposure to such hardware issues is very limited, so I  
> appreciate any
> advice on how to fix it.

The code snippet clearly shows you don't understand
the user/kernel interface, since your access to "param" isn't
correct.  The SEC is complex and has subtle control
considerations, any part of this could be incorrect.  I
suspect your coding errors are hidden/highlighted
by adding the code for the printing, not the actual
operation itself.


	-- Dan

Re: Memory coherency on MPC8272

[Dmitri Petchkine]

On 06/22/2007 03:35 PM, Dan Malek wrote:
> On Jun 22, 2007, at 10:28 AM, Dmitri Petchkine wrote:
>> My understanding that the security hardware of MPC8272 uses DMA to write
>> results into the memory which may cause a coherency problem.
> 
> Where did you get such an (incorrect) understanding?
> I suggest you acquire your understanding from reading the
> MPC8272 reference manual, since it describes the cache
> and snooping options between the SEC and memory,
> as well as among the SEC units.
> 
>> My exposure to such hardware issues is very limited, so I appreciate any
>> advice on how to fix it.
> 
> The code snippet clearly shows you don't understand
> the user/kernel interface, since your access to "param" isn't
> correct.

Please note that I'm NOT writing a driver for SEC1, I'm using it. The 
driver has been written by Freescale. The piece of code I quoted in my 
previous email and you reference to has been coded by Freescale guys. If 
you want take a look, the sources can be downloaded from

http://www.freescale.com/webapp/sps/download/license.jsp?colCode=SEC1DRVRS&location=null&fpsp=1

Before performing any crypto operation, my application calls the SEC1 
driver to allocate memory buffers (via IOCTL_MALLOC) in the kernel space 
and copy all the user buffers into the kernel space (via 
IOCTL_COPYFROM), including the request structure itself.

After the completion of the crypt op, the application copies (via 
IOCTL_COPYTO) the request structure back to the user space as well as 
all the output buffers.

 > The SEC is complex and has subtle control
> considerations, any part of this could be incorrect.  I
> suspect your coding errors are hidden/highlighted
> by adding the code for the printing, not the actual
> operation itself.

Yes it might be that I use the driver improperly. Although the app's 
interaction with it is very limited: set up a structure in the kernel 
space, do a crypto op, get data back to the user space. All of it is 
done through ioctl().

The application is a quite comprehensive unit test for crypto, which 
includes DES/TDES/AES/RC4, SHA-1/SHA-2/MD5, RSA, DSA, DH, and ECC. It 
can be built also as a kernel module - this passes all the exact same 
tests, which gives me some assurance that request structures for the 
SEC1 hardware are set properly.

Thanks
Dmitri Petchkine