Not good performance on Xen

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Not good performance on Xen

Berillions
Hi,

I create a Windows 10 VM (HVM) with Xen. I passed on my VM this device :
- AMD Rx480 - 8 Go

About CPU and Memory, i passed all my cores (16 cores) from my Ryzen (1700X). My system has 16Gb of Memory RAM and i passed 10Gb for the Guest, the host have 4Gb only (dom0_mem=4000M,max:4000M).

I installed "Rise of Tomb Raider" and i launch a benchmark on Win10 on Xen and Win10 BareMetal (installed really on my Desktop). With the same graphic options, the performance decrease on Xen compared to BareMetal.

Bare Metal Benchmark :
https://cdn.discordapp.com/attachments/352637536935346187/371588269231570944/RottR_Benchmark_BM.jpg

Xen Benchmark :
https://media.discordapp.net/attachments/352637536935346187/371588325015814155/RottR_Benchmark_Xen.jpg?width=400&height=225

The average FPS is not bad on Xen, i lost only 4FPS. But if you see the "minimal FPS", there are a big differences :
31FPS for Xen, 51 for BareMetal for the 1st test
7FPS for Xen, 21FPS pour BareMetal for the 2nd and 3rd test)

I remarked something between Xen and Bare Metal.
- During the test #2 (Syria) i have slowdown on Xen. But not on BM.
- During the test #3 (Geothermal Valley), there are missing texture on Xen and not on BM

To have early the same perf, no slowdown and missing texture, i must to set 8 vcpus to the Guest

What can be the causes of these performance losses?

Thanks,
Maxime

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Kun Cheng
Hi Maxime,

Maybe it's related to vcpu scheduling and load balancing, as you mentioned 8 vcpus provided better performance.

Could you try to:

1) pin those 10 guest vpcus to 6-15 (in vm configuration, set cpus="6-15")

2) pin dom0's vcpus to 0- 3.
add 'dom0_max_vcpus=4 dom0_vcpus_pin'  to your kernel cmd line;
use 'xl vcpu-pin' to manually pin dom0's vcpus after bootting into dom0. 
(if dom0 needs more cores then you can just change the number accordingly)

Best,
Kenneth

On Wed, Oct 25, 2017 at 7:53 AM Berillions <[hidden email]> wrote:
Hi,

I create a Windows 10 VM (HVM) with Xen. I passed on my VM this device :
- AMD Rx480 - 8 Go

About CPU and Memory, i passed all my cores (16 cores) from my Ryzen (1700X). My system has 16Gb of Memory RAM and i passed 10Gb for the Guest, the host have 4Gb only (dom0_mem=4000M,max:4000M).

I installed "Rise of Tomb Raider" and i launch a benchmark on Win10 on Xen and Win10 BareMetal (installed really on my Desktop). With the same graphic options, the performance decrease on Xen compared to BareMetal.

Bare Metal Benchmark :
https://cdn.discordapp.com/attachments/352637536935346187/371588269231570944/RottR_Benchmark_BM.jpg

Xen Benchmark :
https://media.discordapp.net/attachments/352637536935346187/371588325015814155/RottR_Benchmark_Xen.jpg?width=400&height=225

The average FPS is not bad on Xen, i lost only 4FPS. But if you see the "minimal FPS", there are a big differences :
31FPS for Xen, 51 for BareMetal for the 1st test
7FPS for Xen, 21FPS pour BareMetal for the 2nd and 3rd test)

I remarked something between Xen and Bare Metal.
- During the test #2 (Syria) i have slowdown on Xen. But not on BM.
- During the test #3 (Geothermal Valley), there are missing texture on Xen and not on BM

To have early the same perf, no slowdown and missing texture, i must to set 8 vcpus to the Guest

What can be the causes of these performance losses?

Thanks,
Maxime
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Dario Faggioli
In reply to this post by Berillions
On Mon, 2017-10-23 at 17:18 +0200, Berillions wrote:
> Hi,
>
Hi,

> I remarked something between Xen and Bare Metal.
> - During the test #2 (Syria) i have slowdown on Xen. But not on BM.
> - During the test #3 (Geothermal Valley), there are missing texture
> on Xen and not on BM
>
> To have early the same perf, no slowdown and missing texture, i must
> to set 8 vcpus to the Guest
>
Ah, interesting! So if the guest has 16 vCPUs, perf is bad, if it has 8
perf is good, is this correct?

Could you, perhaps, execute, on the host, while the benchmark is
running, the following command:

# xl vcpu-list

in both cases (i.e., one time when the guest has 16 vCPUs, and another
when it has 8)?

Also, still while running the benchmark, can you run `xentop' (still on
the host), let it run for a few seconds, check how much CPU % both the
guest and the host are consuming, and repot it here?

Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Adam Goryachev-3
In reply to this post by Berillions



On 24/10/17 02:18, Berillions wrote:
Hi,

I create a Windows 10 VM (HVM) with Xen. I passed on my VM this device :
- AMD Rx480 - 8 Go

About CPU and Memory, i passed all my cores (16 cores) from my Ryzen (1700X). My system has 16Gb of Memory RAM and i passed 10Gb for the Guest, the host have 4Gb only (dom0_mem=4000M,max:4000M).

I installed "Rise of Tomb Raider" and i launch a benchmark on Win10 on Xen and Win10 BareMetal (installed really on my Desktop). With the same graphic options, the performance decrease on Xen compared to BareMetal.

Bare Metal Benchmark :
https://cdn.discordapp.com/attachments/352637536935346187/371588269231570944/RottR_Benchmark_BM.jpg

Xen Benchmark :
https://media.discordapp.net/attachments/352637536935346187/371588325015814155/RottR_Benchmark_Xen.jpg?width=400&height=225

The average FPS is not bad on Xen, i lost only 4FPS. But if you see the "minimal FPS", there are a big differences :
31FPS for Xen, 51 for BareMetal for the 1st test
7FPS for Xen, 21FPS pour BareMetal for the 2nd and 3rd test)

I remarked something between Xen and Bare Metal.
- During the test #2 (Syria) i have slowdown on Xen. But not on BM.
- During the test #3 (Geothermal Valley), there are missing texture on Xen and not on BM

To have early the same perf, no slowdown and missing texture, i must to set 8 vcpus to the Guest

What can be the causes of these performance losses?

I think you will definitely want to reserve at least 2 cores for the dom0, just like you reserved 6GB RAM.

In addition, as others have mentioned, cpu pinning will probably also help, but first step is to not share cores between the dom0 and the domU, as the dom0 also needs to do some work to allow the domU to work.

Regards,
Adam

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Berillions
In reply to this post by Dario Faggioli
Hello guys :-) ,

Sorry for the delay, i was a little busy.

@Kun Cheng :
I did your suggestion but i don't know how to correctly use "xl vcpu-pin" with this configuration. If i don't use this command and launch the benchmark, the slowdown is less important but still exist and about the missing texture, this issue is gone.

@Dario Faggioli
I did your test (during the benchmark) like you asked me and i share your my result for the test with 8 and 16 vcpus. I use pastebin, it's more clearly.
- 8 cpus test : https://pastebin.com/JK48QRGz
- 16 cous test : https://pastebin.com/FFiRBCYW

Cheers,
Maxime

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Kun Cheng
2017-10-27 2:52 GMT+08:00 Berillions <[hidden email]>:
> Hello guys :-) ,
>
> Sorry for the delay, i was a little busy.
>
> @Kun Cheng :
> I did your suggestion but i don't know how to correctly use "xl vcpu-pin"
> with this configuration. If i don't use this command and launch the
> benchmark, the slowdown is less important but still exist and about the
> missing texture, this issue is gone.

Glad to hear it provided some help.

Now you have pinned the VCPUs of the guest to core 6-15 and limit Dom0 to
use 6 VCPUs, right? Because from your test data I saw 16 VCPUs for
dom0.

Could you try pin dom0's vcpus after entering Dom0 with cmds (root
privilege maybe required):

xl vcpu-pin 0 all 0-5 0-5

(change the number and range accordingly if you Dom0's VCPU
configuration is other than 6)

It sets Dom0's all(i.e. 6) VCPUs to core 0-5 with both hard and soft
CPU affinity (which means those VCPUs won't be migrated to other cores
during load balancing).

You could do the same to your guest VM if you wish. Remember don't let
your guest VM use core 0-5 in this case.

Maybe you can append -f to that cmd.

>
> @Dario Faggioli
> I did your test (during the benchmark) like you asked me and i share your my
> result for the test with 8 and 16 vcpus. I use pastebin, it's more clearly.
> - 8 cpus test : https://pastebin.com/JK48QRGz
> - 16 cous test : https://pastebin.com/FFiRBCYW
>
> Cheers,
> Maxime

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Berillions
Hi Kun,

I have this error when i launch your command for 4 Dom0's VCPU (0-3) :
babaorheum@debian-desktop:~/Bureau$ sudo xl vcpu-pin 0 all 0-3 0-3
libxl: error: libxl.c:4970:libxl__set_vcpuaffinity: setting vcpu affinity: Invalid argument
libxl: warning: libxl.c:5033:libxl_set_vcpuaffinity_all: failed to set affinity for 0
libxl: error: libxl.c:4970:libxl__set_vcpuaffinity: setting vcpu affinity: Invalid argument
libxl: warning: libxl.c:5033:libxl_set_vcpuaffinity_all: failed to set affinity for 1
libxl: error: libxl.c:4970:libxl__set_vcpuaffinity: setting vcpu affinity: Invalid argument
libxl: warning: libxl.c:5033:libxl_set_vcpuaffinity_all: failed to set affinity for 2
libxl: error: libxl.c:4970:libxl__set_vcpuaffinity: setting vcpu affinity: Invalid argument
libxl: warning: libxl.c:5033:libxl_set_vcpuaffinity_all: failed to set affinity for 3
Could not set affinity.


2017-10-27 4:48 GMT+02:00 Kun Cheng <[hidden email]>:
2017-10-27 2:52 GMT+08:00 Berillions <[hidden email]>:
> Hello guys :-) ,
>
> Sorry for the delay, i was a little busy.
>
> @Kun Cheng :
> I did your suggestion but i don't know how to correctly use "xl vcpu-pin"
> with this configuration. If i don't use this command and launch the
> benchmark, the slowdown is less important but still exist and about the
> missing texture, this issue is gone.

Glad to hear it provided some help.

Now you have pinned the VCPUs of the guest to core 6-15 and limit Dom0 to
use 6 VCPUs, right? Because from your test data I saw 16 VCPUs for
dom0.

Could you try pin dom0's vcpus after entering Dom0 with cmds (root
privilege maybe required):

xl vcpu-pin 0 all 0-5 0-5

(change the number and range accordingly if you Dom0's VCPU
configuration is other than 6)

It sets Dom0's all(i.e. 6) VCPUs to core 0-5 with both hard and soft
CPU affinity (which means those VCPUs won't be migrated to other cores
during load balancing).

You could do the same to your guest VM if you wish. Remember don't let
your guest VM use core 0-5 in this case.

Maybe you can append -f to that cmd.

>
> @Dario Faggioli
> I did your test (during the benchmark) like you asked me and i share your my
> result for the test with 8 and 16 vcpus. I use pastebin, it's more clearly.
> - 8 cpus test : https://pastebin.com/JK48QRGz
> - 16 cous test : https://pastebin.com/FFiRBCYW
>
> Cheers,
> Maxime


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Dario Faggioli
In reply to this post by Kun Cheng
On Fri, 2017-10-27 at 10:48 +0800, Kun Cheng wrote:

> Could you try pin dom0's vcpus after entering Dom0 with cmds (root
> privilege maybe required):
>
> xl vcpu-pin 0 all 0-5 0-5
>
> (change the number and range accordingly if you Dom0's VCPU
> configuration is other than 6)
>
> It sets Dom0's all(i.e. 6) VCPUs to core 0-5 with both hard and soft
> CPU affinity (which means those VCPUs won't be migrated to other
> cores
> during load balancing).
>
If you don't want dom0's vCPUs to be run anywhere outside of pCPUs 0-5,
setting hard affinity is enough, like this:

# xl vcpu-pin 0 all 0-5

or:

# xl vcpu-pin 0 all 0-5 all

Setting both hard and soft affinity to the same set of pCPUs is
useless, and, if anything, only causes more overhead, so just don't do
that.

Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Berillions
Thanks Dario,

I will try your suggestion after work. I hope it will works, had error message with the suggestion from Kun.

And about my test and result, what do you think ?

Maxime

Envoyé de mon iPhone

> Le 27 oct. 2017 à 09:30, Dario Faggioli <[hidden email]> a écrit :
>
>> On Fri, 2017-10-27 at 10:48 +0800, Kun Cheng wrote:
>> Could you try pin dom0's vcpus after entering Dom0 with cmds (root
>> privilege maybe required):
>>
>> xl vcpu-pin 0 all 0-5 0-5
>>
>> (change the number and range accordingly if you Dom0's VCPU
>> configuration is other than 6)
>>
>> It sets Dom0's all(i.e. 6) VCPUs to core 0-5 with both hard and soft
>> CPU affinity (which means those VCPUs won't be migrated to other
>> cores
>> during load balancing).
>>
> If you don't want dom0's vCPUs to be run anywhere outside of pCPUs 0-5,
> setting hard affinity is enough, like this:
>
> # xl vcpu-pin 0 all 0-5
>
> or:
>
> # xl vcpu-pin 0 all 0-5 all
>
> Setting both hard and soft affinity to the same set of pCPUs is
> useless, and, if anything, only causes more overhead, so just don't do
> that.
>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Dario Faggioli
In reply to this post by Berillions
On Fri, 2017-10-27 at 06:30 +0200, Berillions wrote:

> Hi Kun,
>
> I have this error when i launch your command for 4 Dom0's VCPU (0-3)
> :
> > babaorheum@debian-desktop:~/Bureau$ sudo xl vcpu-pin 0 all 0-3 0-3
> > libxl: error: libxl.c:4970:libxl__set_vcpuaffinity: setting vcpu
> > affinity: Invalid argument
> > libxl: warning: libxl.c:5033:libxl_set_vcpuaffinity_all: failed to
> > set affinity for 0
>
This is a bit weird actually.

What Xen version is this, again?

How did you install it (distribution packages, compiled it yourself,
etc?)

Are you perhaps using the 'dom0_vcpus_pin' boot parameter? (It did not
seem like that from the output of the commands you pasted in the replay
to my other email, but this behavior seems to suggest it...)

Can you show us the output of:

# xl info

Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Berillions
@Dario :
I have this error message when i set 'dom0_max_vcpus=4 dom0_vcpus_pin' to the boot parameter.

And i use Xen 4.8.1 from Debian Stable repository.

Envoyé de mon iPhone

> Le 27 oct. 2017 à 09:58, Dario Faggioli <[hidden email]> a écrit :
>
>> On Fri, 2017-10-27 at 06:30 +0200, Berillions wrote:
>> Hi Kun,
>>
>> I have this error when i launch your command for 4 Dom0's VCPU (0-3)
>> :
>>> babaorheum@debian-desktop:~/Bureau$ sudo xl vcpu-pin 0 all 0-3 0-3
>>> libxl: error: libxl.c:4970:libxl__set_vcpuaffinity: setting vcpu
>>> affinity: Invalid argument
>>> libxl: warning: libxl.c:5033:libxl_set_vcpuaffinity_all: failed to
>>> set affinity for 0
>>
> This is a bit weird actually.
>
> What Xen version is this, again?
>
> How did you install it (distribution packages, compiled it yourself,
> etc?)
>
> Are you perhaps using the 'dom0_vcpus_pin' boot parameter? (It did not
> seem like that from the output of the commands you pasted in the replay
> to my other email, but this behavior seems to suggest it...)
>
> Can you show us the output of:
>
> # xl info
>
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Dario Faggioli
On Fri, 2017-10-27 at 10:42 +0200, LOMBARD Maxime wrote:
> @Dario :
> I have this error message when i set 'dom0_max_vcpus=4
> dom0_vcpus_pin' to the boot parameter.
>
Right.

Get rid of 'dom0_vcpus_pin'.

Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Berillions
I tried with your suggestion Dario and even if i launch "xl vcpu-pin 0 all 0-3", i have slowndown during the benchmark for Rise of the Tomb Raider and Assassin's Creed : Origins.
I share you my "xl info" :
https://pastebin.com/T2ydFuyi

2017-10-27 10:52 GMT+02:00 Dario Faggioli <[hidden email]>:
On Fri, 2017-10-27 at 10:42 +0200, LOMBARD Maxime wrote:
> @Dario :
> I have this error message when i set 'dom0_max_vcpus=4
> dom0_vcpus_pin' to the boot parameter.
>
Right.

Get rid of 'dom0_vcpus_pin'.

Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Not good performance on Xen

Dario Faggioli
On Fri, 2017-10-27 at 17:02 +0200, Berillions wrote:
> > I tried with your suggestion Dario and even if i launch "xl vcpu-
> pin
> > 0 all 0-3", i have slowndown during the benchmark for Rise of the
> > Tomb Raider and Assassin's Creed : Origins.
>
Weel, yes, that wasn't supposed to solve the problem (not alone, at
least). Point is, people suggested you to try pinning.

I'm not sure I agree, but it's probably worth a try. And with that
option set, you can't change the pinning of dom0, which is why I asked
you to get rid of it. It's a very awckward and rarely useful option anyway, so I always recommend leaving it alone, except for some very very very specific case.

Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users

signature.asc (849 bytes) Download Attachment