Re: [PATCH] acl: If build does not support sse4.2, emulate missing instructions with C code

[Neil Horman <nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org>]

On Wed, Aug 06, 2014 at 12:23:29PM +0000, Ananyev, Konstantin wrote:
> 
> 
> > -----Original Message-----
> > From: Neil Horman [mailto:nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org]
> > Sent: Wednesday, August 06, 2014 1:12 PM
> > To: Ananyev, Konstantin
> > Cc: dev-VfR2kkLFssw@public.gmane.org; Thomas Monjalon; Richardson, Bruce
> > Subject: Re: [PATCH] acl: If build does not support sse4.2, emulate missing instructions with C code
> > 
> > On Wed, Aug 06, 2014 at 10:52:09AM +0000, Ananyev, Konstantin wrote:
> > >
> > > > > For ACL there is a simple UT, that could be run as:
> > > > > ./x86_64-native-linuxapp-gcc/app/test ...
> > > > > RTE>>acl_autotest
> > > > > It takes just few seconds to run.
> > > >
> > > > It doesn't seem to work properly, at least not on any of my systems.  With a
> > > > system allocation 100 pages to hugepage memory:
> > > >
> > > > [root@hmsreliant app]# ./test -c 0x3 -n 2
> > > > EAL: Detected lcore 0 as core 0 on socket 0
> > > > EAL: Detected lcore 1 as core 1 on socket 0
> > > > EAL: Detected lcore 2 as core 2 on socket 0
> > > > EAL: Detected lcore 3 as core 3 on socket 0
> > > > EAL: Detected lcore 4 as core 0 on socket 0
> > > > EAL: Detected lcore 5 as core 1 on socket 0
> > > > EAL: Detected lcore 6 as core 2 on socket 0
> > > > EAL: Detected lcore 7 as core 3 on socket 0
> > > > EAL: Support maximum 64 logical core(s) by configuration.
> > > > EAL: Detected 8 lcore(s)
> > > > EAL:   cannot open VFIO container, error 2 (No such file or directory)
> > > > EAL: VFIO support could not be initialized
> > > > EAL: Setting up memory...
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7fef07000000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7fef06c00000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x400000 bytes
> > > > EAL: Virtual area found at 0x7fef06600000 (size = 0x400000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7fef06200000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7fef05e00000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x600000 bytes
> > > > EAL: Virtual area found at 0x7fef05600000 (size = 0x600000)
> > > > EAL: Ask a virtual area of 0x600000 bytes
> > > > EAL: Virtual area found at 0x7fef04e00000 (size = 0x600000)
> > > > EAL: Ask a virtual area of 0x800000 bytes
> > > > EAL: Virtual area found at 0x7fef04400000 (size = 0x800000)
> > > > EAL: Ask a virtual area of 0x400000 bytes
> > > > EAL: Virtual area found at 0x7fef03e00000 (size = 0x400000)
> > > > EAL: Ask a virtual area of 0xa00000 bytes
> > > > EAL: Virtual area found at 0x7fef03200000 (size = 0xa00000)
> > > > EAL: Ask a virtual area of 0x400000 bytes
> > > > EAL: Virtual area found at 0x7fef02c00000 (size = 0x400000)
> > > > EAL: Ask a virtual area of 0x400000 bytes
> > > > EAL: Virtual area found at 0x7fef02600000 (size = 0x400000)
> > > > EAL: Ask a virtual area of 0x1800000 bytes
> > > > EAL: Virtual area found at 0x7fef00c00000 (size = 0x1800000)
> > > > EAL: Ask a virtual area of 0x1200000 bytes
> > > > EAL: Virtual area found at 0x7feeff800000 (size = 0x1200000)
> > > > EAL: Ask a virtual area of 0x2600000 bytes
> > > > EAL: Virtual area found at 0x7feefd000000 (size = 0x2600000)
> > > > EAL: Ask a virtual area of 0xc00000 bytes
> > > > EAL: Virtual area found at 0x7feefc200000 (size = 0xc00000)
> > > > EAL: Ask a virtual area of 0x2200000 bytes
> > > > EAL: Virtual area found at 0x7feef9e00000 (size = 0x2200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7feef9a00000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7feef9600000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7feef9200000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7feef8e00000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7feef8a00000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7feef8600000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > EAL: Virtual area found at 0x7feef8200000 (size = 0x200000)
> > > > EAL: Ask a virtual area of 0x600000 bytes
> > > > EAL: Virtual area found at 0x7feef7a00000 (size = 0x600000)
> > > > EAL: Requesting 100 pages of size 2MB from socket 0
> > > > EAL: TSC frequency is ~3392297 KHz
> > > > EAL: Master core 0 is ready (tid=73cf880)
> > > > EAL: Core 1 is ready (tid=f71fe700)
> > > > APP: HPET is not enabled, using TSC as default timer
> > > > RTE>>acl_autotest
> > > > ACL: allocation of 25166720 bytes on socket 9 for ACL_acl_ctx failed
> > > >
> > > > This hangs forever (well, at least 30 minutes, but thats sufficiently
> > > > close to forever for me to wait).
> > > >
> > >
> > > Ok that's unusual.
> > > Never seen before.
> > > I suppose that happens with unmodified dpdk 1.7?
> > If I build unmodified dpdk 1.7 (which I can only build for the native platform),
> > it works, however it doesn't work when built with modifications for the default
> > platform, but see below.
> > 
> > > I do run acl_autotest with at least 256M of huge-pages as ACL is quite memory consuming  and, as I remember, requires at least
> > 2X32M areas of contiguous hugepages.
> > > But in that case (not enough hugepages) it should just fail with something like:
> > > ACL: allocation of 25953152 bytes on socket -1 for ACL_acl_ctx failed
> > > Any other details, so I can try to reproduce it:
> > > arch you build/run it on?
> > x86_64 default build (has to be to test the new code).
> > 
> > > amount of free memory in the system?
> > Its an 8G workstation, should have most of that free (haven't checked yet).
> 
> That's should be enough.
> I run it on 8GB workstation with ~3GB reported as 'free' by linux.
> 
> > 
> > > Wonder what pstack says?
> > It says that we're stuck in the eal library, before we've touched the ACL
> > library it seems, so I'm not sure whats going on here:
> > #0  0x0000003d11ef53c3 in epoll_wait () from /lib64/libc.so.6
> > #1  0x00000000004d315a in eal_intr_thread_main ()
> > #2  0x0000003d12607f33 in start_thread () from /lib64/libpthread.so.0
> > #3  0x0000003d11ef4ded in clone () from /lib64/libc.so.6
> > Thread 2 (Thread 0x7fee73ffe700 (LWP 14529)):
> > #0  0x0000003d1260e87d in read () from /lib64/libpthread.so.0
> > #1  0x00000000004ced0a in eal_thread_loop ()
> > #2  0x0000003d12607f33 in start_thread () from /lib64/libpthread.so.0
> > #3  0x0000003d11ef4ded in clone () from /lib64/libc.so.6
> > Thread 1 (Thread 0x7fee84100880 (LWP 14527)):
> > #0  0x000000000041adde in acl_match_check_x2.constprop ()
> > #1  0x000000000041b1af in search_sse_2 ()
> > #2  0x00000000004baab9 in rte_acl_classify ()
> > #3  0x000000000047d12d in test_acl ()
> > #4  0x000000000041c535 in cmd_autotest_parsed ()
> > #5  0x00000000004d8503 in cmdline_parse ()
> > #6  0x00000000004d7600 in cmdline_valid_buffer ()
> > #7  0x00000000004da43a in rdline_char_in ()
> > #8  0x00000000004d7809 in cmdline_interact ()
> > #9  0x000000000041bccb in main ()
> 
> Ok, seems that we are in the indefinite loop.
> Probably here:
> acl_match_check_x2 
> {
>   ...
>   while (!MM_TESTZ(temp, temp)) {..}
> }
>  
Possibly, but I don't see how that lines up with the stack trace above.  I'll
add some printfs later this morning to see whats going on.

> Did I get it right, that it works ok with x86_64-default-linuxapp-gcc binary, but hangs with modified x86_64-default-linuxapp-gcc?
> If yes, then I wonder what modifications you made?
> 
Impossible to tell, because x86_64-default-linuxapp-gcc can't build the acl
library without my changes, thats the point of the patch :).

> Konstantin
> 
> 
>