* Change TCG cache size? @ 2021-09-21 21:07 Kenneth Adam Miller 2021-09-22 10:47 ` Alex Bennée 0 siblings, 1 reply; 6+ messages in thread From: Kenneth Adam Miller @ 2021-09-21 21:07 UTC (permalink / raw) To: QEMU Developers [-- Attachment #1: Type: text/plain, Size: 148 bytes --] Hello all, I just want to ask this one question: if I change the qemu tcg cache size ( TB_JMP_CACHE_SIZE), will that force any errors at run time? [-- Attachment #2: Type: text/html, Size: 353 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Change TCG cache size? 2021-09-21 21:07 Change TCG cache size? Kenneth Adam Miller @ 2021-09-22 10:47 ` Alex Bennée 2021-09-22 12:32 ` Kenneth Adam Miller 0 siblings, 1 reply; 6+ messages in thread From: Alex Bennée @ 2021-09-22 10:47 UTC (permalink / raw) To: Kenneth Adam Miller; +Cc: qemu-devel Kenneth Adam Miller <kennethadammiller@gmail.com> writes: > Hello all, > > I just want to ask this one question: if I change the qemu tcg cache > size (TB_JMP_CACHE_SIZE), will that force any errors at run time? Hopefully not - for both user-mode and softmmu we take some care to ensure tb_jmp_cache_hash_func and tb_jmp_cache_hash_page return appropriately masked values for the table lookup. What has not been done since Emilio's work in 6f1653180f (tb-hash: improve tb_jmp_cache hash function in user mode) is a deeper look at the hit rate and bounce rate of the softmmu jump table hashing. Any suggested changes will need some benchmarking to show what difference it makes. -- Alex Bennée ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Change TCG cache size? 2021-09-22 10:47 ` Alex Bennée @ 2021-09-22 12:32 ` Kenneth Adam Miller 2021-09-22 17:32 ` Alex Bennée 0 siblings, 1 reply; 6+ messages in thread From: Kenneth Adam Miller @ 2021-09-22 12:32 UTC (permalink / raw) To: Alex Bennée; +Cc: QEMU Developers [-- Attachment #1: Type: text/plain, Size: 1106 bytes --] Well, maybe I'm understanding that wrong. I am talking the cache that the tcg keeps of translated code. If I got that variable wrong then please let me know. But I want to make sure that that is large enough to keep from having to run TCG again. How can I do that? On Wed, Sep 22, 2021, 6:54 AM Alex Bennée <alex.bennee@linaro.org> wrote: > > Kenneth Adam Miller <kennethadammiller@gmail.com> writes: > > > Hello all, > > > > I just want to ask this one question: if I change the qemu tcg cache > > size (TB_JMP_CACHE_SIZE), will that force any errors at run time? > > Hopefully not - for both user-mode and softmmu we take some care to > ensure tb_jmp_cache_hash_func and tb_jmp_cache_hash_page return > appropriately masked values for the table lookup. > > What has not been done since Emilio's work in 6f1653180f (tb-hash: > improve tb_jmp_cache hash function in user mode) is a deeper look at the > hit rate and bounce rate of the softmmu jump table hashing. Any > suggested changes will need some benchmarking to show what difference it > makes. > > -- > Alex Bennée > [-- Attachment #2: Type: text/html, Size: 1515 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Change TCG cache size? 2021-09-22 12:32 ` Kenneth Adam Miller @ 2021-09-22 17:32 ` Alex Bennée 2021-09-22 17:50 ` Kenneth Adam Miller 0 siblings, 1 reply; 6+ messages in thread From: Alex Bennée @ 2021-09-22 17:32 UTC (permalink / raw) To: Kenneth Adam Miller; +Cc: QEMU Developers Kenneth Adam Miller <kennethadammiller@gmail.com> writes: > Well, maybe I'm understanding that wrong. I am talking the cache that the tcg keeps of translated code. If I got that variable wrong then > please let me know. TB_JMP_CACHE_SIZE is used to keep a lookup of address to translated blocks (TBs). This is used to find the next TB on a computed jump without doing a full lookup in the QHT hash. The total size of the translation cache is tb-size in MBs (as in -accel tcg,tb-size=1024). We have some heuristics to guess at a size (see DEFAULT_CODE_GEN_BUFFER_SIZE in tcg/region.c) but you are free to specify your own. > But I want to make sure that that is large enough to keep from having > to run TCG again. How can I do that? Specifying a large tb-size will reduce the churn caused by running out of translation cache but you will never be able to eliminate it entirely. There are a number of places where tb_flush() has to get called and that will require stuff to get re-translated. Also the translator partitions the regions up per-CPU (for softmmu) so if one vCPU is responsible for all code generation it will run out sooner. You can observe the total number of flushes via the HMP and "info jit". What is the behaviour your seeing? What workload is it? > > On Wed, Sep 22, 2021, 6:54 AM Alex Bennée <alex.bennee@linaro.org> wrote: > > Kenneth Adam Miller <kennethadammiller@gmail.com> writes: > > > Hello all, > > > > I just want to ask this one question: if I change the qemu tcg cache > > size (TB_JMP_CACHE_SIZE), will that force any errors at run time? > > Hopefully not - for both user-mode and softmmu we take some care to > ensure tb_jmp_cache_hash_func and tb_jmp_cache_hash_page return > appropriately masked values for the table lookup. > > What has not been done since Emilio's work in 6f1653180f (tb-hash: > improve tb_jmp_cache hash function in user mode) is a deeper look at the > hit rate and bounce rate of the softmmu jump table hashing. Any > suggested changes will need some benchmarking to show what difference it > makes. > > -- > Alex Bennée -- Alex Bennée ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Change TCG cache size? 2021-09-22 17:32 ` Alex Bennée @ 2021-09-22 17:50 ` Kenneth Adam Miller 2021-09-22 19:52 ` Kenneth Adam Miller 0 siblings, 1 reply; 6+ messages in thread From: Kenneth Adam Miller @ 2021-09-22 17:50 UTC (permalink / raw) To: Alex Bennée; +Cc: QEMU Developers [-- Attachment #1: Type: text/plain, Size: 2758 bytes --] It's just the overhead of running a cross architecture emulation. For Arm to x86_64, the overhead is very high. I was wondering if there is some command line argument that I was missing in order to reduce this. I read somewhere that the tcg cache is defaulted to some value, and wanted to check in to make sure that that wasn't it. I can't see it right now, I was just looking into it. On Wed, Sep 22, 2021 at 1:39 PM Alex Bennée <alex.bennee@linaro.org> wrote: > > Kenneth Adam Miller <kennethadammiller@gmail.com> writes: > > > Well, maybe I'm understanding that wrong. I am talking the cache that > the tcg keeps of translated code. If I got that variable wrong then > > please let me know. > > TB_JMP_CACHE_SIZE is used to keep a lookup of address to translated > blocks (TBs). This is used to find the next TB on a computed jump > without doing a full lookup in the QHT hash. > > The total size of the translation cache is tb-size in MBs (as in -accel > tcg,tb-size=1024). We have some heuristics to guess at a size (see > DEFAULT_CODE_GEN_BUFFER_SIZE in tcg/region.c) but you are free to > specify your own. > > > But I want to make sure that that is large enough to keep from having > > to run TCG again. How can I do that? > > Specifying a large tb-size will reduce the churn caused by running out > of translation cache but you will never be able to eliminate it > entirely. There are a number of places where tb_flush() has to get > called and that will require stuff to get re-translated. Also the > translator partitions the regions up per-CPU (for softmmu) so if one > vCPU is responsible for all code generation it will run out sooner. > > You can observe the total number of flushes via the HMP and "info jit". > What is the behaviour your seeing? What workload is it? > > > > > On Wed, Sep 22, 2021, 6:54 AM Alex Bennée <alex.bennee@linaro.org> > wrote: > > > > Kenneth Adam Miller <kennethadammiller@gmail.com> writes: > > > > > Hello all, > > > > > > I just want to ask this one question: if I change the qemu tcg cache > > > size (TB_JMP_CACHE_SIZE), will that force any errors at run time? > > > > Hopefully not - for both user-mode and softmmu we take some care to > > ensure tb_jmp_cache_hash_func and tb_jmp_cache_hash_page return > > appropriately masked values for the table lookup. > > > > What has not been done since Emilio's work in 6f1653180f (tb-hash: > > improve tb_jmp_cache hash function in user mode) is a deeper look at the > > hit rate and bounce rate of the softmmu jump table hashing. Any > > suggested changes will need some benchmarking to show what difference it > > makes. > > > > -- > > Alex Bennée > > > -- > Alex Bennée > [-- Attachment #2: Type: text/html, Size: 3491 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Change TCG cache size? 2021-09-22 17:50 ` Kenneth Adam Miller @ 2021-09-22 19:52 ` Kenneth Adam Miller 0 siblings, 0 replies; 6+ messages in thread From: Kenneth Adam Miller @ 2021-09-22 19:52 UTC (permalink / raw) To: Alex Bennée; +Cc: QEMU Developers [-- Attachment #1: Type: text/plain, Size: 3038 bytes --] Also, that command flag was *really really buried, and I couldn't find it at all in the help. On Wed, Sep 22, 2021 at 1:50 PM Kenneth Adam Miller < kennethadammiller@gmail.com> wrote: > It's just the overhead of running a cross architecture emulation. For Arm > to x86_64, the overhead is very high. I was wondering if there is some > command line argument that I was missing in order to reduce this. I read > somewhere that the tcg cache is defaulted to some value, and wanted to > check in to make sure that that wasn't it. > > I can't see it right now, I was just looking into it. > > On Wed, Sep 22, 2021 at 1:39 PM Alex Bennée <alex.bennee@linaro.org> > wrote: > >> >> Kenneth Adam Miller <kennethadammiller@gmail.com> writes: >> >> > Well, maybe I'm understanding that wrong. I am talking the cache that >> the tcg keeps of translated code. If I got that variable wrong then >> > please let me know. >> >> TB_JMP_CACHE_SIZE is used to keep a lookup of address to translated >> blocks (TBs). This is used to find the next TB on a computed jump >> without doing a full lookup in the QHT hash. >> >> The total size of the translation cache is tb-size in MBs (as in -accel >> tcg,tb-size=1024). We have some heuristics to guess at a size (see >> DEFAULT_CODE_GEN_BUFFER_SIZE in tcg/region.c) but you are free to >> specify your own. >> >> > But I want to make sure that that is large enough to keep from having >> > to run TCG again. How can I do that? >> >> Specifying a large tb-size will reduce the churn caused by running out >> of translation cache but you will never be able to eliminate it >> entirely. There are a number of places where tb_flush() has to get >> called and that will require stuff to get re-translated. Also the >> translator partitions the regions up per-CPU (for softmmu) so if one >> vCPU is responsible for all code generation it will run out sooner. >> >> You can observe the total number of flushes via the HMP and "info jit". >> What is the behaviour your seeing? What workload is it? >> >> > >> > On Wed, Sep 22, 2021, 6:54 AM Alex Bennée <alex.bennee@linaro.org> >> wrote: >> > >> > Kenneth Adam Miller <kennethadammiller@gmail.com> writes: >> > >> > > Hello all, >> > > >> > > I just want to ask this one question: if I change the qemu tcg cache >> > > size (TB_JMP_CACHE_SIZE), will that force any errors at run time? >> > >> > Hopefully not - for both user-mode and softmmu we take some care to >> > ensure tb_jmp_cache_hash_func and tb_jmp_cache_hash_page return >> > appropriately masked values for the table lookup. >> > >> > What has not been done since Emilio's work in 6f1653180f (tb-hash: >> > improve tb_jmp_cache hash function in user mode) is a deeper look at >> the >> > hit rate and bounce rate of the softmmu jump table hashing. Any >> > suggested changes will need some benchmarking to show what difference >> it >> > makes. >> > >> > -- >> > Alex Bennée >> >> >> -- >> Alex Bennée >> > [-- Attachment #2: Type: text/html, Size: 3983 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-09-22 19:54 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-09-21 21:07 Change TCG cache size? Kenneth Adam Miller 2021-09-22 10:47 ` Alex Bennée 2021-09-22 12:32 ` Kenneth Adam Miller 2021-09-22 17:32 ` Alex Bennée 2021-09-22 17:50 ` Kenneth Adam Miller 2021-09-22 19:52 ` Kenneth Adam Miller
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).