DomU not starting under pvhv2

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

DomU not starting under pvhv2

Peter-3
Hi.

Running Xen 4.10.0

A VM is not starting with type = 'pvh'.  The VM starts, but exits prior
to any data being read off the domU disk image.

The same VMs works if the type is changed to 'pv'.

Other VMs on other hosts (with the exact same setup) work with pvh fine.

The main difference between hosts where type=pvh does not work and ones
where it does is that the pvh does not work on older CPUs:
  Intel(R) Core(TM)2 Duo CPU     E4600  @ 2.40GHz
  Intel(R) Xeon(R) CPU           E3110  @ 3.00GHz
  Intel(R) Xeon(R) CPU           E5310  @ 1.60GHz
  Intel(R) Xeon(R) CPU           E5320  @ 1.86GHz
  Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
  Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz


Newer CPUs seem to be fine (start VMs with type = pvh):
  Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
  Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
  Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
  Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
  Intel(R) Xeon(R) CPU E3-1240 v5 @ 3.50GHz
  Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
  Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
  Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
  Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
  Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
  Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
  Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
  Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
  Intel(R) Xeon(R) CPU           X3440  @ 2.53GHz

Dom0 command line is: dom0_max_vcpus=2 dom0_vcpus_pin loglvl=info
vcpu_migration_delay=1000

I see (all?) the older CPUs don't have VT-d.  However some of the CPUs
where pvh is working do not report hvm_directio in xl info virt_caps
(e.g. the E5506 servers) though they do have VT-d available per
ark.intel.com.

And ideas on what may be stopping type=pvh VMs starting on those hosts?  
And how we could get it working?

Else we'll likely be dumping some hardware....

I did a diff of xl info; xl dmesg and xl create startup messages.  
Between one of the non-pvh-able hosts (the - line) and a host where pvh
works (the + line):
diff host of xl -v create and xlinfo:
-hw_caps                :
bfebfbff:0004e3bd:20100800:00000001:00000000:00000000:00000000:00000000
-virt_caps              : hvm
+hw_caps                :
bfebfbff:009ee3fd:2c100800:00000001:00000000:00000000:00000000:00000100
+virt_caps              : hvm hvm_directio

+libxl: detail: libxl_dom.c:204:numa_place_domain: NUMA placement
candidate with 1 nodes, 8 cpus and 16819 KB free selected

- (XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
+ (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs

- (XEN) I/O virtualisation disabled
+ (XEN) Intel VT-d Snoop Control enabled.
+ (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
+ (XEN) Intel VT-d Queued Invalidation enabled.
+ (XEN) Intel VT-d Interrupt Remapping enabled.
+ (XEN) Intel VT-d Posted Interrupt not enabled.
+ (XEN) Intel VT-d Shared EPT tables not enabled.
+ (XEN) I/O virtualisation enabled

-(XEN)  -> Using new ACK method
+(XEN)  -> Using old ACK method

   (XEN) VMX: Supported advanced features:
+ (XEN)  - APIC MMIO access virtualisation
+ (XEN)  - Extended Page Tables (EPT)
+ (XEN)  - Virtual-Processor Identifiers (VPID)
+ (XEN)  - Virtual NMI
+ (XEN)  - Unrestricted Guest


-(XEN) HVM: Hardware Assisted Paging (HAP) not detected
+(XEN) HVM: Hardware Assisted Paging (HAP) detected

+(XEN) Found masked UR signaling on 0000:00:00.0
+(XEN) Masked UR signaling on 0000:00:01.0
+(XEN) Masked UR signaling on 0000:00:03.0
+(XEN) Masked UR signaling on 0000:00:05.0
+(XEN) Masked UR signaling on 0000:00:07.0
+(XEN) Masked VT-d error signaling on 0000:00:14.0


Regards, Peter

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: DomU not starting under pvhv2

Christopher Myers
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512



On Mon, 2018-01-08 at 22:01 +1300, Peter wrote:

> Hi.
>
> Running Xen 4.10.0
>
> A VM is not starting with type = 'pvh'.  The VM starts, but exits prior
> to any data being read off the domU disk image.
>
> The same VMs works if the type is changed to 'pv'.
>
> Other VMs on other hosts (with the exact same setup) work with pvh fine.
>
> The main difference between hosts where type=pvh does not work and ones
> where it does is that the pvh does not work on older CPUs:
>   Intel(R) Core(TM)2 Duo CPU     E4600  @ 2.40GHz
>   Intel(R) Xeon(R) CPU           E3110  @ 3.00GHz
>   Intel(R) Xeon(R) CPU           E5310  @ 1.60GHz
>   Intel(R) Xeon(R) CPU           E5320  @ 1.86GHz
>   Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
>   Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz
>
>
> Newer CPUs seem to be fine (start VMs with type = pvh):
>   Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
>   Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
>   Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
>   Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
>   Intel(R) Xeon(R) CPU E3-1240 v5 @ 3.50GHz
>   Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
>   Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
>   Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
>   Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
>   Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
>   Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
>   Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
>   Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
>   Intel(R) Xeon(R) CPU           X3440  @ 2.53GHz
>
> Dom0 command line is: dom0_max_vcpus=2 dom0_vcpus_pin loglvl=info
> vcpu_migration_delay=1000
>
> I see (all?) the older CPUs don't have VT-d.  However some of the CPUs
> where pvh is working do not report hvm_directio in xl info virt_caps
> (e.g. the E5506 servers) though they do have VT-d available per
> ark.intel.com.


Out of curiosity, just because the CPU supports VT-d doesn't mean that
everything else in the system does or that it's enabled in the BIOS. My
work computer, for example, supports it, but I had to explicitly enable
it first. So just something to double-check.


>
And ideas on what may be stopping type=pvh VMs starting on those hosts?  

> And how we could get it working?
>
> Else we'll likely be dumping some hardware....
>
> I did a diff of xl info; xl dmesg and xl create startup messages.  
> Between one of the non-pvh-able hosts (the - line) and a host where pvh
> works (the + line):
> diff host of xl -v create and xlinfo:
> -hw_caps                :
> bfebfbff:0004e3bd:20100800:00000001:00000000:00000000:00000000:00000000
> -virt_caps              : hvm
> +hw_caps                :
> bfebfbff:009ee3fd:2c100800:00000001:00000000:00000000:00000000:00000100
> +virt_caps              : hvm hvm_directio
>
> +libxl: detail: libxl_dom.c:204:numa_place_domain: NUMA placement
> candidate with 1 nodes, 8 cpus and 16819 KB free selected
>
> - (XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
> + (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
>
> - (XEN) I/O virtualisation disabled
> + (XEN) Intel VT-d Snoop Control enabled.
> + (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> + (XEN) Intel VT-d Queued Invalidation enabled.
> + (XEN) Intel VT-d Interrupt Remapping enabled.
> + (XEN) Intel VT-d Posted Interrupt not enabled.
> + (XEN) Intel VT-d Shared EPT tables not enabled.
> + (XEN) I/O virtualisation enabled
>
> -(XEN)  -> Using new ACK method
> +(XEN)  -> Using old ACK method
>
>    (XEN) VMX: Supported advanced features:
> + (XEN)  - APIC MMIO access virtualisation
> + (XEN)  - Extended Page Tables (EPT)
> + (XEN)  - Virtual-Processor Identifiers (VPID)
> + (XEN)  - Virtual NMI
> + (XEN)  - Unrestricted Guest
>
>
> -(XEN) HVM: Hardware Assisted Paging (HAP) not detected
> +(XEN) HVM: Hardware Assisted Paging (HAP) detected
>
> +(XEN) Found masked UR signaling on 0000:00:00.0
> +(XEN) Masked UR signaling on 0000:00:01.0
> +(XEN) Masked UR signaling on 0000:00:03.0
> +(XEN) Masked UR signaling on 0000:00:05.0
> +(XEN) Masked UR signaling on 0000:00:07.0
> +(XEN) Masked VT-d error signaling on 0000:00:14.0
>
>
> Regards, Peter
>
> _______________________________________________
> Xen-users mailing list
> [hidden email]
>


> https://lists.xenproject.org/mailman/listinfo/xen-users
-----BEGIN PGP SIGNATURE-----

iQEyBAEBCgAdFiEE7GM/Dul8WSWn72odQ1nEo4DFCIUFAlpTkEsACgkQQ1nEo4DF
CIWHCQf4mB1U1V+J0j1C1Dr7Zz3QG7dc1rb8iAKEjDQCTMP4I5vke9jf7fDUNFaF
JI3C3SEzY1XeedRsywedqlkZKkk+RK0W5B0rUkpEGbjopGj4VuIB7G8qntWUnTHT
5kUmPAtQwjRHPypZIHFK7uhqI/3WgsnUy85du76q+3MzXE4NFx+hQ0wMbhCWrcyK
pUoWh+dFybsJPRAEGHv4FSUwVOdzNDocKbqPDd11wLG/Lw3tqguRCbHUhOwycUlP
vIKD21EtZ59Iq5oyqoGkIZQMIKA7JRrtSMvmwHSS2gp4p+4tYpK32A9UWIJj1jZD
ieX6l7d1cwnxp89D2MwtIYf7pdnj
=RvxY
-----END PGP SIGNATURE-----
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: DomU not starting under pvhv2

Peter-3
In reply to this post by Peter-3

On 9/01/18 4:37 AM, Christopher Myers wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
>
>
> On Mon, 2018-01-08 at 22:01 +1300, Peter wrote:
>>
>> I see (all?) the older CPUs don't have VT-d.  However some of the CPUs
>> where pvh is working do not report hvm_directio in xl info virt_caps
>> (e.g. the E5506 servers) though they do have VT-d available per
>> ark.intel.com.
>
>
> Out of curiosity, just because the CPU supports VT-d doesn't mean that
> everything else in the system does or that it's enabled in the BIOS. My
> work computer, for example, supports it, but I had to explicitly enable
> it first. So just something to double-check.
>
>

Note: my issue is that the VM cannot start under pvh.

Is VT-d / hardware assisted IO required for pvh?

I don't need hardware assisted IO.  I just want to be able to start the
VM under pvh rather than pv.

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users