Question about process.c

Question about process.c

[Anton Ivanov]

Hi all, hi Richard,

I have a question - is it really necessary to pass a "force" flag to a 
tlb flush before doing a fork (in process.c)?

If the flush is not forced, the fork becomes significantly less 
expensive. I tried running a couple of crude tests and did not notice 
anything broken if the flush there is done without a "force" flag.

-- 
Anton R. Ivanov

Cambridge Greys Limited, England and Wales company No 10273661
http://www.cambridgegreys.com/	


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: Question about process.c

[Richard Weinberger]

Am Dienstag, 1. Mai 2018, 15:01:54 CEST schrieb Anton Ivanov:
> Hi all, hi Richard,
> 
> I have a question - is it really necessary to pass a "force" flag to a 
> tlb flush before doing a fork (in process.c)?

Where exactly?

Thanks,
//richard

-- 
sigma star gmbh - Eduard-Bodem-Gasse 6 - 6020 Innsbruck - Austria
ATU66964118 - FN 374287y

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: Question about process.c

[Anton Ivanov]

arch/um/kernel/process.c line 140 invokes force_flush_all();

which does a full tlb flush with the force flag set.

I did a quick test and I did not notice any breakage if we replace that 
with a quickly hacked together new function called gentle_flush_all() 
which does the same but without setting the force flag in the tlb routines.

At the same time I got 10% or better speedup for fork-heavy things like 
startup, etc.

In fact, this is one of the last remaining major performance bugbears. 
We got most of the other stuff to a very reasonable standard - 
networking, disk, etc is not that far off from let's say qemu and the 
overall "slowness" is now mostly down to the huge cost of fork/exec.

Example (ab)using busybox. Busybox executes cat passed as the find -exec 
argument internally shortcutting to its applet. As a result there is no 
cost of fork/exec incurred when we run a find -exec cat {} via busybox. 
Normal find executes /bin/cat instead.

This allows us to compare and attribute the cost of fork/exec in scripts 
as well as gives us a benchmark to see the effect of any changes.

Bare metal:

aivanov@amistad:/var/autofs/local/src/linux-work/linux-submit$ time 
busybox find /mnt/usr -type f -exec cat {} > /dev/null \;

real    0m16.511s
user    0m12.672s
sys    0m4.001s
aivanov@amistad:/var/autofs/local/src/linux-work/linux-submit$ time find 
/mnt/usr -type f -exec cat {} > /dev/null \;

real    0m25.329s
user    0m16.397s
sys    0m9.185s

UML:

root@uml-switch:~# time busybox find /usr -type f -exec cat {} > 
/dev/null \;

real    0m11.447s
user    0m0.000s
sys    0m8.820s
root@uml-switch:~# time find /usr -type f -exec cat {} > /dev/null \;

real    7m8.228s
user    0m0.000s
sys    6m42.780s

The filesystem is identical in both cases. The experiment is not 
perfectly "clean" as i am doing other stuff on my laptop, but it is 
reasonably indicative

In fact, in the longer term I would really like to somehow speed-up most 
of the stuff tlb.c It is painfully slow at present.

A.

On 05/01/18 14:21, Richard Weinberger wrote:
> Am Dienstag, 1. Mai 2018, 15:01:54 CEST schrieb Anton Ivanov:
>> Hi all, hi Richard,
>>
>> I have a question - is it really necessary to pass a "force" flag to a
>> tlb flush before doing a fork (in process.c)?
> Where exactly?
>
> Thanks,
> //richard
>

-- 
Anton R. Ivanov

Cambridge Greys Limited, England and Wales company No 10273661
http://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: Question about process.c

[Richard Weinberger]

Anton,

Am Dienstag, 1. Mai 2018, 15:48:48 CEST schrieb Anton Ivanov:
> arch/um/kernel/process.c line 140 invokes force_flush_all();
> 
> which does a full tlb flush with the force flag set.
> 
> I did a quick test and I did not notice any breakage if we replace that 
> with a quickly hacked together new function called gentle_flush_all() 
> which does the same but without setting the force flag in the tlb routines.
> 
> At the same time I got 10% or better speedup for fork-heavy things like 
> startup, etc.
> 
> In fact, this is one of the last remaining major performance bugbears. 
> We got most of the other stuff to a very reasonable standard - 
> networking, disk, etc is not that far off from let's say qemu and the 
> overall "slowness" is now mostly down to the huge cost of fork/exec.
> 
> Example (ab)using busybox. Busybox executes cat passed as the find -exec 
> argument internally shortcutting to its applet. As a result there is no 
> cost of fork/exec incurred when we run a find -exec cat {} via busybox. 
> Normal find executes /bin/cat instead.
> 
> This allows us to compare and attribute the cost of fork/exec in scripts 
> as well as gives us a benchmark to see the effect of any changes.
> 
> Bare metal:
> 
> aivanov@amistad:/var/autofs/local/src/linux-work/linux-submit$ time 
> busybox find /mnt/usr -type f -exec cat {} > /dev/null \;
> 
> real    0m16.511s
> user    0m12.672s
> sys    0m4.001s
> aivanov@amistad:/var/autofs/local/src/linux-work/linux-submit$ time find 
> /mnt/usr -type f -exec cat {} > /dev/null \;
> 
> real    0m25.329s
> user    0m16.397s
> sys    0m9.185s
> 
> UML:
> 
> root@uml-switch:~# time busybox find /usr -type f -exec cat {} > 
> /dev/null \;
> 
> real    0m11.447s
> user    0m0.000s
> sys    0m8.820s
> root@uml-switch:~# time find /usr -type f -exec cat {} > /dev/null \;
> 
> real    7m8.228s
> user    0m0.000s
> sys    6m42.780s
> 
> The filesystem is identical in both cases. The experiment is not 
> perfectly "clean" as i am doing other stuff on my laptop, but it is 
> reasonably indicative
> 
> In fact, in the longer term I would really like to somehow speed-up most 
> of the stuff tlb.c It is painfully slow at present.

Hmmm, not sure. I fear without going deep into the UML history we cannot know.
I think we have to do a full flush for security reasons.

Thanks,
//richard

-- 
sigma star gmbh - Eduard-Bodem-Gasse 6 - 6020 Innsbruck - Austria
ATU66964118 - FN 374287y

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: Question about process.c

[Anton Ivanov]

OK.

I will park it until I have a large timeslot to play with it and analyze
it properly.

A.

On 01/05/18 15:36, Richard Weinberger wrote:
> Anton,
>
> Am Dienstag, 1. Mai 2018, 15:48:48 CEST schrieb Anton Ivanov:
>> arch/um/kernel/process.c line 140 invokes force_flush_all();
>>
>> which does a full tlb flush with the force flag set.
>>
>> I did a quick test and I did not notice any breakage if we replace that 
>> with a quickly hacked together new function called gentle_flush_all() 
>> which does the same but without setting the force flag in the tlb routines.
>>
>> At the same time I got 10% or better speedup for fork-heavy things like 
>> startup, etc.
>>
>> In fact, this is one of the last remaining major performance bugbears. 
>> We got most of the other stuff to a very reasonable standard - 
>> networking, disk, etc is not that far off from let's say qemu and the 
>> overall "slowness" is now mostly down to the huge cost of fork/exec.
>>
>> Example (ab)using busybox. Busybox executes cat passed as the find -exec 
>> argument internally shortcutting to its applet. As a result there is no 
>> cost of fork/exec incurred when we run a find -exec cat {} via busybox. 
>> Normal find executes /bin/cat instead.
>>
>> This allows us to compare and attribute the cost of fork/exec in scripts 
>> as well as gives us a benchmark to see the effect of any changes.
>>
>> Bare metal:
>>
>> aivanov@amistad:/var/autofs/local/src/linux-work/linux-submit$ time 
>> busybox find /mnt/usr -type f -exec cat {} > /dev/null \;
>>
>> real    0m16.511s
>> user    0m12.672s
>> sys    0m4.001s
>> aivanov@amistad:/var/autofs/local/src/linux-work/linux-submit$ time find 
>> /mnt/usr -type f -exec cat {} > /dev/null \;
>>
>> real    0m25.329s
>> user    0m16.397s
>> sys    0m9.185s
>>
>> UML:
>>
>> root@uml-switch:~# time busybox find /usr -type f -exec cat {} > 
>> /dev/null \;
>>
>> real    0m11.447s
>> user    0m0.000s
>> sys    0m8.820s
>> root@uml-switch:~# time find /usr -type f -exec cat {} > /dev/null \;
>>
>> real    7m8.228s
>> user    0m0.000s
>> sys    6m42.780s
>>
>> The filesystem is identical in both cases. The experiment is not 
>> perfectly "clean" as i am doing other stuff on my laptop, but it is 
>> reasonably indicative
>>
>> In fact, in the longer term I would really like to somehow speed-up most 
>> of the stuff tlb.c It is painfully slow at present.
> Hmmm, not sure. I fear without going deep into the UML history we cannot know.
> I think we have to do a full flush for security reasons.
>
> Thanks,
> //richard
>


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: Question about process.c

[Richard Weinberger]

Anton,

On Tue, May 1, 2018 at 4:43 PM, Anton Ivanov
<anton.ivanov@kot-begemot.co.uk> wrote:
> OK.
>
> I will park it until I have a large timeslot to play with it and analyze
> it properly.

Can you please use reply-all and not top-post?

I did some research.
The force flush has its roots in uml-patch-2.4.9-4, in form of
flush_tlb_kernel_vm().
We have kind of a changelog:
http://user-mode-linux.sourceforge.net/old/changelog-uml-patch-2.4.9-4.bz2.html
"Fixed the process segfaults that happened when the system was swapping"

In uml-patch-2.4.9-8 it gained further changes.
"Fixed and cleaned up the tlb flush code".

I think it is high noon to review/rethink that code...

-- 
Thanks,
//richard

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

Re: Question about process.c

[Anton Ivanov]

On 5/1/18 10:33 PM, Richard Weinberger wrote:
> Anton,
>
> On Tue, May 1, 2018 at 4:43 PM, Anton Ivanov
> <anton.ivanov@kot-begemot.co.uk> wrote:
>> OK.
>>
>> I will park it until I have a large timeslot to play with it and analyze
>> it properly.
> Can you please use reply-all and not top-post?
>
> I did some research.
> The force flush has its roots in uml-patch-2.4.9-4, in form of
> flush_tlb_kernel_vm().
> We have kind of a changelog:
> http://user-mode-linux.sourceforge.net/old/changelog-uml-patch-2.4.9-4.bz2.html
> "Fixed the process segfaults that happened when the system was swapping"
>
> In uml-patch-2.4.9-8 it gained further changes.
> "Fixed and cleaned up the tlb flush code".
>
> I think it is high noon to review/rethink that code...
>
I have reviewed and retested it. Unfortunately, the original issue is 
still there so the tlb has to have a force flush if swap is present.

That, however, means that we can stick an

#ifdef CONFIG_SWAP
#define FORK_MM_FORCE 1
#elif
#define FORK_MM_FORCE 0
#endif

In the beginning of tlb.c.If we then make the force_flush conditional on 
SWAP

void force_flush_all(void)
{
     struct mm_struct *mm = current->mm;
     struct vm_area_struct *vma = mm->mmap;

     while (vma != NULL) {
         fix_range(mm, vma->vm_start, vma->vm_end, FORK_MM_FORCE);
         vma = vma->vm_next;
     }
}

We give 10%+ performance boost to those who would want to forgo swap.

A.


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um