pvgrub2(-like?) booting methods for PVHv2 guests

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

pvgrub2(-like?) booting methods for PVHv2 guests

andy smith-10
Hi,

What are the current options for booting PVHv2 guests in Xen 4.10.0?

As far as I can see there is only direct kernel booting, i.e.
supplying "kernel = /path/to/vmlinux".

In PV land there is pvgrub which is very convenient for being able
to use kernel and initramfs, but I am not seeing support for that in
upstream grub2. I see that Juergen Gross post a patch series that
claims to add this feature:

    <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>

…but this doesn't seem to have been applied yet. Is it an ongoing
idea or has that method been abandoned?

Cheers,
Andy

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
On 23/01/18 05:46, Andy Smith wrote:

> Hi,
>
> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>
> As far as I can see there is only direct kernel booting, i.e.
> supplying "kernel = /path/to/vmlinux".
>
> In PV land there is pvgrub which is very convenient for being able
> to use kernel and initramfs, but I am not seeing support for that in
> upstream grub2. I see that Juergen Gross post a patch series that
> claims to add this feature:
>
>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>
> …but this doesn't seem to have been applied yet. Is it an ongoing
> idea or has that method been abandoned?

Still waiting for review.


Juergen

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
On 01/23/2018 06:41 AM, Juergen Gross wrote:

> On 23/01/18 05:46, Andy Smith wrote:
>> Hi,
>>
>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>
>> As far as I can see there is only direct kernel booting, i.e.
>> supplying "kernel = /path/to/vmlinux".
>>
>> In PV land there is pvgrub which is very convenient for being able
>> to use kernel and initramfs, but I am not seeing support for that in
>> upstream grub2. I see that Juergen Gross post a patch series that
>> claims to add this feature:
>>
>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>
>> …but this doesn't seem to have been applied yet. Is it an ongoing
>> idea or has that method been abandoned?
>
> Still waiting for review.
Since I wanted to try to test this, I scraped the patches off the
grub-devel list and did the following:

1. clone https://git.savannah.gnu.org/git/grub.git
2. apply patches on top of master, plus an extra one with a puzzle piece
to make it actually compile (see attachment)

3. build:
./autogen.sh
./configure --target=i386 --with-platform=xenpvh
make

4. use mkstandalone to create a grub image to boot
./grub-mkstandalone -d grub-core --locale-directory=/ -o
~/grub-i386-xenpvh-fire-ze-missile -O i386-xenpvh
~/build/pvgrub2/pvgrub2/xvda-fire-ze-missile.cfg

contents of the config file that I want to have inserted into it (same
file I use for booting PV now):

root='(xen/xvda)'
insmod xzio
insmod gzio
insmod btrfs
insmod ext2
echo 'Loading Linux ...'
linux /vmlinuz root=/dev/xvda ro elevator=noop
echo 'Loading initial ramdisk ...'
initrd /initrd.img
boot

5. Try to use it already, to see what happens. (Xen 4.10.0)

-# cat yolo-pvgrub2

type="pvh"
kernel = "/yolo/grub-i386-xenpvh-fire-ze-missile"
name = "yolo-pvgrub2"
memory = 1024
vcpus = 10
vif = [
    "mac=02:00:0a:8c:d9:0d,bridge=ovs0.217",
]
disk = [
    '/dev/test/yolo,raw,xvda,rw',
]

-# xen create -c yolo-pvgrub2
Parsing config from yolo-pvgrub2
libxl: info: libxl_create.c:109:libxl__domain_build_info_setdefault:
qemu-xen is unavailable, using qemu-xen-traditional instead: No such
file or directory
S3 disabled
S4 disabled
CONV disabled

                     GNU GRUB  version 2.03

   Minimal BASH-like line editing is supported. For the first word, TAB
   lists possible command completions. Anywhere else TAB lists possible
   device or file completions.


grub>

Ok, so for some reason it seems to ignore the config? But I can type
things manually:

grub> insmod xzio
grub> insmod gzio
grub> insmod btrfs
grub> insmod ext2
grub> search.file /vmlinuz
 xen/xvda

Ok, so it can see the disk and fine the symlink to the kernel image. Now
let's try the rest of it:

grub> root='(xen/xvda)'
grub> linux /vmlinuz root=/dev/xvda ro elevator=noop
grub> initrd /initrd.img
grub> boot

And now it just hangs, no output any more.

6. Apply linux patches to the domU kernel and Xen

domU Linux kernel:
- Branch off from linux 4.14.14
- add https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git as
remote and cherry-pick the 4 patches mentioned in
  https://lists.gnu.org/archive/html/grub-devel/2017-12/msg00017.html

Xen: pick this one on top of 4.10.0:
 https://lists.xen.org/archives/html/xen-devel/2017-11/msg01807.html
 4a5733771e6f33918eba07b584cccce564a67ac1

7. Test again after dom0 reboot into the new xen and put the new domU
kernel on it.
 -> same result: grub command prompt, and no activity after 'boot'.

Extra things:

8. Start domU after copying kernel/initrd to the dom0:

type="pvh"
kernel = "/yolo/vmlinuz-4.14.14-yolo"
ramdisk = "/yolo/initrd.img-4.14.14-yolo"
root = "/dev/xvda ro elevator=noop console=hvc0"

This boots, dmesg: http://paste.debian.net/plainh/7120cef2

9. Just for fun, start domU with previous kernel/initrd without the patches:

type="pvh"
kernel = "/yolo/vmlinuz-4.14.0-3-amd64"
ramdisk = "/yolo/initrd.img-4.14.0-3-amd64"
root = "/dev/xvda ro elevator=noop console=hvc0"

This boots and has 1 vcpu and no ACPI lines in dmesg, which is expected
breakage, I believe, reading xen-devel.

So, what am I missing?

Hans

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users

grub-pvh.tgz (46K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
On 24/01/18 17:37, Hans van Kranenburg wrote:

> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>> On 23/01/18 05:46, Andy Smith wrote:
>>> Hi,
>>>
>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>
>>> As far as I can see there is only direct kernel booting, i.e.
>>> supplying "kernel = /path/to/vmlinux".
>>>
>>> In PV land there is pvgrub which is very convenient for being able
>>> to use kernel and initramfs, but I am not seeing support for that in
>>> upstream grub2. I see that Juergen Gross post a patch series that
>>> claims to add this feature:
>>>
>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>
>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>> idea or has that method been abandoned?
>>
>> Still waiting for review.
>
> Since I wanted to try to test this, I scraped the patches off the
> grub-devel list and did the following:
>
> 1. clone https://git.savannah.gnu.org/git/grub.git
> 2. apply patches on top of master, plus an extra one with a puzzle piece
> to make it actually compile (see attachment)
>
> 3. build:
> ./autogen.sh
> ./configure --target=i386 --with-platform=xenpvh
> make
>
> 4. use mkstandalone to create a grub image to boot
> ./grub-mkstandalone -d grub-core --locale-directory=/ -o
> ~/grub-i386-xenpvh-fire-ze-missile -O i386-xenpvh
> ~/build/pvgrub2/pvgrub2/xvda-fire-ze-missile.cfg
>
> contents of the config file that I want to have inserted into it (same
> file I use for booting PV now):
>
> root='(xen/xvda)'
> insmod xzio
> insmod gzio
> insmod btrfs
> insmod ext2
> echo 'Loading Linux ...'
> linux /vmlinuz root=/dev/xvda ro elevator=noop
> echo 'Loading initial ramdisk ...'
> initrd /initrd.img
> boot
>
> 5. Try to use it already, to see what happens. (Xen 4.10.0)
>
> -# cat yolo-pvgrub2
>
> type="pvh"
> kernel = "/yolo/grub-i386-xenpvh-fire-ze-missile"
> name = "yolo-pvgrub2"
> memory = 1024
> vcpus = 10
> vif = [
>     "mac=02:00:0a:8c:d9:0d,bridge=ovs0.217",
> ]
> disk = [
>     '/dev/test/yolo,raw,xvda,rw',
> ]
>
> -# xen create -c yolo-pvgrub2
> Parsing config from yolo-pvgrub2
> libxl: info: libxl_create.c:109:libxl__domain_build_info_setdefault:
> qemu-xen is unavailable, using qemu-xen-traditional instead: No such
> file or directory
> S3 disabled
> S4 disabled
> CONV disabled
>
>                      GNU GRUB  version 2.03
>
>    Minimal BASH-like line editing is supported. For the first word, TAB
>    lists possible command completions. Anywhere else TAB lists possible
>    device or file completions.
>
>
> grub>
>
> Ok, so for some reason it seems to ignore the config? But I can type
> things manually:
>
> grub> insmod xzio
> grub> insmod gzio
> grub> insmod btrfs
> grub> insmod ext2
> grub> search.file /vmlinuz
>  xen/xvda
>
> Ok, so it can see the disk and fine the symlink to the kernel image. Now
> let's try the rest of it:
>
> grub> root='(xen/xvda)'
> grub> linux /vmlinuz root=/dev/xvda ro elevator=noop
> grub> initrd /initrd.img
> grub> boot
>
> And now it just hangs, no output any more.
>
> 6. Apply linux patches to the domU kernel and Xen
>
> domU Linux kernel:
> - Branch off from linux 4.14.14
> - add https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git as
> remote and cherry-pick the 4 patches mentioned in
>   https://lists.gnu.org/archive/html/grub-devel/2017-12/msg00017.html
>
> Xen: pick this one on top of 4.10.0:
>  https://lists.xen.org/archives/html/xen-devel/2017-11/msg01807.html
>  4a5733771e6f33918eba07b584cccce564a67ac1
>
> 7. Test again after dom0 reboot into the new xen and put the new domU
> kernel on it.
>  -> same result: grub command prompt, and no activity after 'boot'.
>
> Extra things:
>
> 8. Start domU after copying kernel/initrd to the dom0:
>
> type="pvh"
> kernel = "/yolo/vmlinuz-4.14.14-yolo"
> ramdisk = "/yolo/initrd.img-4.14.14-yolo"
> root = "/dev/xvda ro elevator=noop console=hvc0"
>
> This boots, dmesg: http://paste.debian.net/plainh/7120cef2
>
> 9. Just for fun, start domU with previous kernel/initrd without the patches:
>
> type="pvh"
> kernel = "/yolo/vmlinuz-4.14.0-3-amd64"
> ramdisk = "/yolo/initrd.img-4.14.0-3-amd64"
> root = "/dev/xvda ro elevator=noop console=hvc0"
>
> This boots and has 1 vcpu and no ACPI lines in dmesg, which is expected
> breakage, I believe, reading xen-devel.
>
> So, what am I missing?
I used the attached script to build it (derived from the pvgrub build
script used in SUSE's grub-xen RPM).


Juergen


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
On 01/25/2018 07:01 AM, Juergen Gross wrote:

> On 24/01/18 17:37, Hans van Kranenburg wrote:
>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>> On 23/01/18 05:46, Andy Smith wrote:
>>>> Hi,
>>>>
>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>
>>>> As far as I can see there is only direct kernel booting, i.e.
>>>> supplying "kernel = /path/to/vmlinux".
>>>>
>>>> In PV land there is pvgrub which is very convenient for being able
>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>> claims to add this feature:
>>>>
>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>
>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>> idea or has that method been abandoned?
>>>
>>> Still waiting for review.
>>
>> Since I wanted to try to test this, I scraped the patches off the
>> grub-devel list and did the following:
>>
>> 1. clone https://git.savannah.gnu.org/git/grub.git
>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>> to make it actually compile (see attachment)
>>
>> 3. build:
>> ./autogen.sh
>> ./configure --target=i386 --with-platform=xenpvh
>> make
>>
>> 4. use mkstandalone to create a grub image to boot
>> ./grub-mkstandalone -d grub-core --locale-directory=/ -o
>> ~/grub-i386-xenpvh-fire-ze-missile -O i386-xenpvh
>> ~/build/pvgrub2/pvgrub2/xvda-fire-ze-missile.cfg
>>
>> contents of the config file that I want to have inserted into it (same
>> file I use for booting PV now):
>>
>> root='(xen/xvda)'
>> insmod xzio
>> insmod gzio
>> insmod btrfs
>> insmod ext2
>> echo 'Loading Linux ...'
>> linux /vmlinuz root=/dev/xvda ro elevator=noop
>> echo 'Loading initial ramdisk ...'
>> initrd /initrd.img
>> boot
>>
>> 5. Try to use it already, to see what happens. (Xen 4.10.0)
>>
>> -# cat yolo-pvgrub2
>>
>> type="pvh"
>> kernel = "/yolo/grub-i386-xenpvh-fire-ze-missile"
>> name = "yolo-pvgrub2"
>> memory = 1024
>> vcpus = 10
>> vif = [
>>     "mac=02:00:0a:8c:d9:0d,bridge=ovs0.217",
>> ]
>> disk = [
>>     '/dev/test/yolo,raw,xvda,rw',
>> ]
>>
>> -# xen create -c yolo-pvgrub2
>> Parsing config from yolo-pvgrub2
>> libxl: info: libxl_create.c:109:libxl__domain_build_info_setdefault:
>> qemu-xen is unavailable, using qemu-xen-traditional instead: No such
>> file or directory
>> S3 disabled
>> S4 disabled
>> CONV disabled
>>
>>                      GNU GRUB  version 2.03
>>
>>    Minimal BASH-like line editing is supported. For the first word, TAB
>>    lists possible command completions. Anywhere else TAB lists possible
>>    device or file completions.
>>
>>
>> grub>
>>
>> Ok, so for some reason it seems to ignore the config? But I can type
>> things manually:
>>
>> grub> insmod xzio
>> grub> insmod gzio
>> grub> insmod btrfs
>> grub> insmod ext2
>> grub> search.file /vmlinuz
>>  xen/xvda
>>
>> Ok, so it can see the disk and fine the symlink to the kernel image. Now
>> let's try the rest of it:
>>
>> grub> root='(xen/xvda)'
>> grub> linux /vmlinuz root=/dev/xvda ro elevator=noop
>> grub> initrd /initrd.img
>> grub> boot
>>
>> And now it just hangs, no output any more.
>>
>> 6. Apply linux patches to the domU kernel and Xen
>>
>> domU Linux kernel:
>> - Branch off from linux 4.14.14
>> - add https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git as
>> remote and cherry-pick the 4 patches mentioned in
>>   https://lists.gnu.org/archive/html/grub-devel/2017-12/msg00017.html
>>
>> Xen: pick this one on top of 4.10.0:
>>  https://lists.xen.org/archives/html/xen-devel/2017-11/msg01807.html
>>  4a5733771e6f33918eba07b584cccce564a67ac1
>>
>> 7. Test again after dom0 reboot into the new xen and put the new domU
>> kernel on it.
>>  -> same result: grub command prompt, and no activity after 'boot'.
>>
>> Extra things:
>>
>> 8. Start domU after copying kernel/initrd to the dom0:
>>
>> type="pvh"
>> kernel = "/yolo/vmlinuz-4.14.14-yolo"
>> ramdisk = "/yolo/initrd.img-4.14.14-yolo"
>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>
>> This boots, dmesg: http://paste.debian.net/plainh/7120cef2
>>
>> 9. Just for fun, start domU with previous kernel/initrd without the patches:
>>
>> type="pvh"
>> kernel = "/yolo/vmlinuz-4.14.0-3-amd64"
>> ramdisk = "/yolo/initrd.img-4.14.0-3-amd64"
>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>
>> This boots and has 1 vcpu and no ACPI lines in dmesg, which is expected
>> breakage, I believe, reading xen-devel.
>>
>> So, what am I missing?
>
> I used the attached script to build it (derived from the pvgrub build
> script used in SUSE's grub-xen RPM).

Thanks. This gets me a step further.

Using the "graft point syntax" thingie
("/boot/grub/grub.cfg=./grub.cfg") helped getting the config actually
executed instead of getting the prompt.

Next I got:

Welcome to GRUB!

Loading Linux ...
Loading initial ramdisk ...

...and then it stops.

So I modified my grub config a bit from the one shown above, adding
explicit console=hvc0:

root='(xen/xvda)'
insmod xzio
insmod gzio
insmod btrfs
insmod ext2
echo 'Loading Linux ...'
linux /vmlinuz root=/dev/xvda ro elevator=noop console=hvc0
echo 'Loading initial ramdisk ...'
initrd /initrd.img
echo 'There we go! ...'
boot

And now I get console output and things happen. Only it can't find the disk.

[...]
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/xvda does not exist.  Dropping to a shell!
(initramfs)

Full log at http://paste.debian.net/plainh/426bed60

Now I have to find out why's that. But first time for lunch. ;)

Hans

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
On 25/01/18 12:39, Hans van Kranenburg wrote:

> On 01/25/2018 07:01 AM, Juergen Gross wrote:
>> On 24/01/18 17:37, Hans van Kranenburg wrote:
>>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>>> On 23/01/18 05:46, Andy Smith wrote:
>>>>> Hi,
>>>>>
>>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>>
>>>>> As far as I can see there is only direct kernel booting, i.e.
>>>>> supplying "kernel = /path/to/vmlinux".
>>>>>
>>>>> In PV land there is pvgrub which is very convenient for being able
>>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>>> claims to add this feature:
>>>>>
>>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>>
>>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>>> idea or has that method been abandoned?
>>>>
>>>> Still waiting for review.
>>>
>>> Since I wanted to try to test this, I scraped the patches off the
>>> grub-devel list and did the following:
>>>
>>> 1. clone https://git.savannah.gnu.org/git/grub.git
>>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>>> to make it actually compile (see attachment)
>>>
>>> 3. build:
>>> ./autogen.sh
>>> ./configure --target=i386 --with-platform=xenpvh
>>> make
>>>
>>> 4. use mkstandalone to create a grub image to boot
>>> ./grub-mkstandalone -d grub-core --locale-directory=/ -o
>>> ~/grub-i386-xenpvh-fire-ze-missile -O i386-xenpvh
>>> ~/build/pvgrub2/pvgrub2/xvda-fire-ze-missile.cfg
>>>
>>> contents of the config file that I want to have inserted into it (same
>>> file I use for booting PV now):
>>>
>>> root='(xen/xvda)'
>>> insmod xzio
>>> insmod gzio
>>> insmod btrfs
>>> insmod ext2
>>> echo 'Loading Linux ...'
>>> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>> echo 'Loading initial ramdisk ...'
>>> initrd /initrd.img
>>> boot
>>>
>>> 5. Try to use it already, to see what happens. (Xen 4.10.0)
>>>
>>> -# cat yolo-pvgrub2
>>>
>>> type="pvh"
>>> kernel = "/yolo/grub-i386-xenpvh-fire-ze-missile"
>>> name = "yolo-pvgrub2"
>>> memory = 1024
>>> vcpus = 10
>>> vif = [
>>>     "mac=02:00:0a:8c:d9:0d,bridge=ovs0.217",
>>> ]
>>> disk = [
>>>     '/dev/test/yolo,raw,xvda,rw',
>>> ]
>>>
>>> -# xen create -c yolo-pvgrub2
>>> Parsing config from yolo-pvgrub2
>>> libxl: info: libxl_create.c:109:libxl__domain_build_info_setdefault:
>>> qemu-xen is unavailable, using qemu-xen-traditional instead: No such
>>> file or directory
>>> S3 disabled
>>> S4 disabled
>>> CONV disabled
>>>
>>>                      GNU GRUB  version 2.03
>>>
>>>    Minimal BASH-like line editing is supported. For the first word, TAB
>>>    lists possible command completions. Anywhere else TAB lists possible
>>>    device or file completions.
>>>
>>>
>>> grub>
>>>
>>> Ok, so for some reason it seems to ignore the config? But I can type
>>> things manually:
>>>
>>> grub> insmod xzio
>>> grub> insmod gzio
>>> grub> insmod btrfs
>>> grub> insmod ext2
>>> grub> search.file /vmlinuz
>>>  xen/xvda
>>>
>>> Ok, so it can see the disk and fine the symlink to the kernel image. Now
>>> let's try the rest of it:
>>>
>>> grub> root='(xen/xvda)'
>>> grub> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>> grub> initrd /initrd.img
>>> grub> boot
>>>
>>> And now it just hangs, no output any more.
>>>
>>> 6. Apply linux patches to the domU kernel and Xen
>>>
>>> domU Linux kernel:
>>> - Branch off from linux 4.14.14
>>> - add https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git as
>>> remote and cherry-pick the 4 patches mentioned in
>>>   https://lists.gnu.org/archive/html/grub-devel/2017-12/msg00017.html
>>>
>>> Xen: pick this one on top of 4.10.0:
>>>  https://lists.xen.org/archives/html/xen-devel/2017-11/msg01807.html
>>>  4a5733771e6f33918eba07b584cccce564a67ac1
>>>
>>> 7. Test again after dom0 reboot into the new xen and put the new domU
>>> kernel on it.
>>>  -> same result: grub command prompt, and no activity after 'boot'.
>>>
>>> Extra things:
>>>
>>> 8. Start domU after copying kernel/initrd to the dom0:
>>>
>>> type="pvh"
>>> kernel = "/yolo/vmlinuz-4.14.14-yolo"
>>> ramdisk = "/yolo/initrd.img-4.14.14-yolo"
>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>
>>> This boots, dmesg: http://paste.debian.net/plainh/7120cef2
>>>
>>> 9. Just for fun, start domU with previous kernel/initrd without the patches:
>>>
>>> type="pvh"
>>> kernel = "/yolo/vmlinuz-4.14.0-3-amd64"
>>> ramdisk = "/yolo/initrd.img-4.14.0-3-amd64"
>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>
>>> This boots and has 1 vcpu and no ACPI lines in dmesg, which is expected
>>> breakage, I believe, reading xen-devel.
>>>
>>> So, what am I missing?
>>
>> I used the attached script to build it (derived from the pvgrub build
>> script used in SUSE's grub-xen RPM).
>
> Thanks. This gets me a step further.
>
> Using the "graft point syntax" thingie
> ("/boot/grub/grub.cfg=./grub.cfg") helped getting the config actually
> executed instead of getting the prompt.
>
> Next I got:
>
> Welcome to GRUB!
>
> Loading Linux ...
> Loading initial ramdisk ...
>
> ...and then it stops.
>
> So I modified my grub config a bit from the one shown above, adding
> explicit console=hvc0:
>
> root='(xen/xvda)'
> insmod xzio
> insmod gzio
> insmod btrfs
> insmod ext2
> echo 'Loading Linux ...'
> linux /vmlinuz root=/dev/xvda ro elevator=noop console=hvc0
> echo 'Loading initial ramdisk ...'
> initrd /initrd.img
> echo 'There we go! ...'
> boot
>
> And now I get console output and things happen. Only it can't find the disk.
>
> [...]
> Begin: Running /scripts/local-block ... done.
> Begin: Running /scripts/local-block ... done.
> done.
> Gave up waiting for root file system device.  Common problems:
>  - Boot args (cat /proc/cmdline)
>    - Check rootdelay= (did the system wait long enough?)
>  - Missing modules (cat /proc/modules; ls /dev)
> ALERT!  /dev/xvda does not exist.  Dropping to a shell!
> (initramfs)
>
> Full log at http://paste.debian.net/plainh/426bed60
>
> Now I have to find out why's that. But first time for lunch. ;)

Missing xen-blkfront driver in initrd?


Juergen


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

andy smith-10
In reply to this post by Hans van Kranenburg-2
Hi Hans,

On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
> And now I get console output and things happen. Only it can't find the disk.

I was trying similar thing (4.10 and PVH) and also ended up with a
guest with no block devices. I reported this on grub-devel:

    <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>

as I was thinking this was not a Xen problem since same thing boots
okay outside grub with direct kernel boot.

Juergen did reply and said I needed this kernel patch in the guest:

    <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>

But I think you have this don't you?

I didn't investigate further as I need to run distribution packaged
kernels in the guests.

Cheers,
Andy

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
On 25/01/18 13:19, Andy Smith wrote:

> Hi Hans,
>
> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>> And now I get console output and things happen. Only it can't find the disk.
>
> I was trying similar thing (4.10 and PVH) and also ended up with a
> guest with no block devices. I reported this on grub-devel:
>
>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>
> as I was thinking this was not a Xen problem since same thing boots
> okay outside grub with direct kernel boot.
>
> Juergen did reply and said I needed this kernel patch in the guest:
>
>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>
> But I think you have this don't you?

As the ACPI tables are found, I'd say yes. :-)


Juergen

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
In reply to this post by Juergen Gross-3
On 25/01/2018 13:01, Juergen Gross wrote:

> On 25/01/18 12:39, Hans van Kranenburg wrote:
>> On 01/25/2018 07:01 AM, Juergen Gross wrote:
>>> On 24/01/18 17:37, Hans van Kranenburg wrote:
>>>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>>>> On 23/01/18 05:46, Andy Smith wrote:
>>>>>> Hi,
>>>>>>
>>>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>>>
>>>>>> As far as I can see there is only direct kernel booting, i.e.
>>>>>> supplying "kernel = /path/to/vmlinux".
>>>>>>
>>>>>> In PV land there is pvgrub which is very convenient for being able
>>>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>>>> claims to add this feature:
>>>>>>
>>>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>>>
>>>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>>>> idea or has that method been abandoned?
>>>>>
>>>>> Still waiting for review.
>>>>
>>>> Since I wanted to try to test this, I scraped the patches off the
>>>> grub-devel list and did the following:
>>>>
>>>> 1. clone https://git.savannah.gnu.org/git/grub.git
>>>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>>>> to make it actually compile (see attachment)
>>>>
>>>> 3. build:
>>>> ./autogen.sh
>>>> ./configure --target=i386 --with-platform=xenpvh
>>>> make
>>>>
>>>> 4. use mkstandalone to create a grub image to boot
>>>> ./grub-mkstandalone -d grub-core --locale-directory=/ -o
>>>> ~/grub-i386-xenpvh-fire-ze-missile -O i386-xenpvh
>>>> ~/build/pvgrub2/pvgrub2/xvda-fire-ze-missile.cfg
>>>>
>>>> contents of the config file that I want to have inserted into it (same
>>>> file I use for booting PV now):
>>>>
>>>> root='(xen/xvda)'
>>>> insmod xzio
>>>> insmod gzio
>>>> insmod btrfs
>>>> insmod ext2
>>>> echo 'Loading Linux ...'
>>>> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>>> echo 'Loading initial ramdisk ...'
>>>> initrd /initrd.img
>>>> boot
>>>>
>>>> 5. Try to use it already, to see what happens. (Xen 4.10.0)
>>>>
>>>> -# cat yolo-pvgrub2
>>>>
>>>> type="pvh"
>>>> kernel = "/yolo/grub-i386-xenpvh-fire-ze-missile"
>>>> name = "yolo-pvgrub2"
>>>> memory = 1024
>>>> vcpus = 10
>>>> vif = [
>>>>     "mac=02:00:0a:8c:d9:0d,bridge=ovs0.217",
>>>> ]
>>>> disk = [
>>>>     '/dev/test/yolo,raw,xvda,rw',
>>>> ]
>>>>
>>>> -# xen create -c yolo-pvgrub2
>>>> Parsing config from yolo-pvgrub2
>>>> libxl: info: libxl_create.c:109:libxl__domain_build_info_setdefault:
>>>> qemu-xen is unavailable, using qemu-xen-traditional instead: No such
>>>> file or directory
>>>> S3 disabled
>>>> S4 disabled
>>>> CONV disabled
>>>>
>>>>                      GNU GRUB  version 2.03
>>>>
>>>>    Minimal BASH-like line editing is supported. For the first word, TAB
>>>>    lists possible command completions. Anywhere else TAB lists possible
>>>>    device or file completions.
>>>>
>>>>
>>>> grub>
>>>>
>>>> Ok, so for some reason it seems to ignore the config? But I can type
>>>> things manually:
>>>>
>>>> grub> insmod xzio
>>>> grub> insmod gzio
>>>> grub> insmod btrfs
>>>> grub> insmod ext2
>>>> grub> search.file /vmlinuz
>>>>  xen/xvda
>>>>
>>>> Ok, so it can see the disk and fine the symlink to the kernel image. Now
>>>> let's try the rest of it:
>>>>
>>>> grub> root='(xen/xvda)'
>>>> grub> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>>> grub> initrd /initrd.img
>>>> grub> boot
>>>>
>>>> And now it just hangs, no output any more.
>>>>
>>>> 6. Apply linux patches to the domU kernel and Xen
>>>>
>>>> domU Linux kernel:
>>>> - Branch off from linux 4.14.14
>>>> - add https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git as
>>>> remote and cherry-pick the 4 patches mentioned in
>>>>   https://lists.gnu.org/archive/html/grub-devel/2017-12/msg00017.html
>>>>
>>>> Xen: pick this one on top of 4.10.0:
>>>>  https://lists.xen.org/archives/html/xen-devel/2017-11/msg01807.html
>>>>  4a5733771e6f33918eba07b584cccce564a67ac1
>>>>
>>>> 7. Test again after dom0 reboot into the new xen and put the new domU
>>>> kernel on it.
>>>>  -> same result: grub command prompt, and no activity after 'boot'.
>>>>
>>>> Extra things:
>>>>
>>>> 8. Start domU after copying kernel/initrd to the dom0:
>>>>
>>>> type="pvh"
>>>> kernel = "/yolo/vmlinuz-4.14.14-yolo"
>>>> ramdisk = "/yolo/initrd.img-4.14.14-yolo"
>>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>>
>>>> This boots, dmesg: http://paste.debian.net/plainh/7120cef2
>>>>
>>>> 9. Just for fun, start domU with previous kernel/initrd without the patches:
>>>>
>>>> type="pvh"
>>>> kernel = "/yolo/vmlinuz-4.14.0-3-amd64"
>>>> ramdisk = "/yolo/initrd.img-4.14.0-3-amd64"
>>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>>
>>>> This boots and has 1 vcpu and no ACPI lines in dmesg, which is expected
>>>> breakage, I believe, reading xen-devel.
>>>>
>>>> So, what am I missing?
>>>
>>> I used the attached script to build it (derived from the pvgrub build
>>> script used in SUSE's grub-xen RPM).
>>
>> Thanks. This gets me a step further.
>>
>> Using the "graft point syntax" thingie
>> ("/boot/grub/grub.cfg=./grub.cfg") helped getting the config actually
>> executed instead of getting the prompt.
>>
>> Next I got:
>>
>> Welcome to GRUB!
>>
>> Loading Linux ...
>> Loading initial ramdisk ...
>>
>> ...and then it stops.
>>
>> So I modified my grub config a bit from the one shown above, adding
>> explicit console=hvc0:
>>
>> root='(xen/xvda)'
>> insmod xzio
>> insmod gzio
>> insmod btrfs
>> insmod ext2
>> echo 'Loading Linux ...'
>> linux /vmlinuz root=/dev/xvda ro elevator=noop console=hvc0
>> echo 'Loading initial ramdisk ...'
>> initrd /initrd.img
>> echo 'There we go! ...'
>> boot
>>
>> And now I get console output and things happen. Only it can't find the disk.
>>
>> [...]
>> Begin: Running /scripts/local-block ... done.
>> Begin: Running /scripts/local-block ... done.
>> done.
>> Gave up waiting for root file system device.  Common problems:
>>  - Boot args (cat /proc/cmdline)
>>    - Check rootdelay= (did the system wait long enough?)
>>  - Missing modules (cat /proc/modules; ls /dev)
>> ALERT!  /dev/xvda does not exist.  Dropping to a shell!
>> (initramfs)
>>
>> Full log at http://paste.debian.net/plainh/426bed60
>>
>> Now I have to find out why's that. But first time for lunch. ;)
>
> Missing xen-blkfront driver in initrd?

Exactly the same kernel/initrd boots successfully when directly loading
them from dom0 without grub in between, see boot output I pastebinned
earlier.

K

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
In reply to this post by Juergen Gross-3
On 25/01/2018 13:29, Juergen Gross wrote:

> On 25/01/18 13:19, Andy Smith wrote:
>> Hi Hans,
>>
>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>> And now I get console output and things happen. Only it can't find the disk.
>>
>> I was trying similar thing (4.10 and PVH) and also ended up with a
>> guest with no block devices. I reported this on grub-devel:
>>
>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>
>> as I was thinking this was not a Xen problem since same thing boots
>> okay outside grub with direct kernel boot.
>>
>> Juergen did reply and said I needed this kernel patch in the guest:
>>
>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>
>> But I think you have this don't you?

Yes, see my earlier mail with all the steps that I did, step 6.

> As the ACPI tables are found, I'd say yes. :-)

dmesg output is pretty different when I boot directly with the kernel
and initrd copied on the dom0.

Remember, it's the same kernel/initrd, and without grub in between it
boots with all vcpus network and disk.

With grub in between, this at least does look suspicious:

[    0.032110] PCI: System does not support PCI

And yes, there are also no successful netfront lines.

Without grub: http://paste.debian.net/plainh/7120cef2
With grub: http://paste.debian.net/plainh/426bed60

K

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:

> On 25/01/2018 13:29, Juergen Gross wrote:
>> On 25/01/18 13:19, Andy Smith wrote:
>>> Hi Hans,
>>>
>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>> And now I get console output and things happen. Only it can't find the disk.
>>>
>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>> guest with no block devices. I reported this on grub-devel:
>>>
>>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>
>>> as I was thinking this was not a Xen problem since same thing boots
>>> okay outside grub with direct kernel boot.
>>>
>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>
>>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>
>>> But I think you have this don't you?
>
> Yes, see my earlier mail with all the steps that I did, step 6.
>
>> As the ACPI tables are found, I'd say yes. :-)
>
> dmesg output is pretty different when I boot directly with the kernel
> and initrd copied on the dom0.
>
> Remember, it's the same kernel/initrd, and without grub in between it
> boots with all vcpus network and disk.
>
> With grub in between, this at least does look suspicious:
>
> [    0.032110] PCI: System does not support PCI

Eh, that's in both, stay awake Hans.

> And yes, there are also no successful netfront lines.
>
> Without grub: http://paste.debian.net/plainh/7120cef2
> With grub: http://paste.debian.net/plainh/426bed60

Or the diff between them, which shows what changes when inserting grub
in between:

http://paste.debian.net/plainh/52b2d618

I must admit I don't know too much (yet) about all those changed lines,
but this next is also a very interesting change?...

-Booting paravirtualized kernel on Xen PVH
+Booting paravirtualized kernel on Xen HVM

K

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
On 25/01/18 15:12, Hans van Kranenburg wrote:

> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>> On 25/01/2018 13:29, Juergen Gross wrote:
>>> On 25/01/18 13:19, Andy Smith wrote:
>>>> Hi Hans,
>>>>
>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>> And now I get console output and things happen. Only it can't find the disk.
>>>>
>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>> guest with no block devices. I reported this on grub-devel:
>>>>
>>>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>
>>>> as I was thinking this was not a Xen problem since same thing boots
>>>> okay outside grub with direct kernel boot.
>>>>
>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>
>>>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>
>>>> But I think you have this don't you?
>>
>> Yes, see my earlier mail with all the steps that I did, step 6.
>>
>>> As the ACPI tables are found, I'd say yes. :-)
>>
>> dmesg output is pretty different when I boot directly with the kernel
>> and initrd copied on the dom0.
>>
>> Remember, it's the same kernel/initrd, and without grub in between it
>> boots with all vcpus network and disk.
>>
>> With grub in between, this at least does look suspicious:
>>
>> [    0.032110] PCI: System does not support PCI
>
> Eh, that's in both, stay awake Hans.
>
>> And yes, there are also no successful netfront lines.
>>
>> Without grub: http://paste.debian.net/plainh/7120cef2
>> With grub: http://paste.debian.net/plainh/426bed60
>
> Or the diff between them, which shows what changes when inserting grub
> in between:
>
> http://paste.debian.net/plainh/52b2d618
>
> I must admit I don't know too much (yet) about all those changed lines,
> but this next is also a very interesting change?...
>
> -Booting paravirtualized kernel on Xen PVH
> +Booting paravirtualized kernel on Xen HVM

Aah, yes, this should be the reason for the problems.

I addressed the ACPI problem first. What is missing now is to set PVH
mode when booting via grub.

So please stay tuned...


Juergen


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
On 01/25/2018 03:31 PM, Juergen Gross wrote:

> On 25/01/18 15:12, Hans van Kranenburg wrote:
>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>> Hi Hans,
>>>>>
>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>>> And now I get console output and things happen. Only it can't find the disk.
>>>>>
>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>
>>>>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>
>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>> okay outside grub with direct kernel boot.
>>>>>
>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>
>>>>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>
>>>>> But I think you have this don't you?
>>>
>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>
>>>> As the ACPI tables are found, I'd say yes. :-)
>>>
>>> dmesg output is pretty different when I boot directly with the kernel
>>> and initrd copied on the dom0.
>>>
>>> Remember, it's the same kernel/initrd, and without grub in between it
>>> boots with all vcpus network and disk.
>>>
>>> With grub in between, this at least does look suspicious:
>>>
>>> [    0.032110] PCI: System does not support PCI
>>
>> Eh, that's in both, stay awake Hans.
>>
>>> And yes, there are also no successful netfront lines.
>>>
>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>> With grub: http://paste.debian.net/plainh/426bed60
>>
>> Or the diff between them, which shows what changes when inserting grub
>> in between:
>>
>> http://paste.debian.net/plainh/52b2d618
>>
>> I must admit I don't know too much (yet) about all those changed lines,
>> but this next is also a very interesting change?...
>>
>> -Booting paravirtualized kernel on Xen PVH
>> +Booting paravirtualized kernel on Xen HVM
>
> Aah, yes, this should be the reason for the problems.

Bingo!

> I addressed the ACPI problem first. What is missing now is to set PVH
> mode when booting via grub.
>
> So please stay tuned...

Sure. At least I got as far as currently possible now, not bad. :-)

In the meantime I'll continue in my Xen 4.10 / PVHv2 test environment
with just throwing one kernel version on every domU and directly
starting it hardcoded.

Thanks,
Hans

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
In reply to this post by Juergen Gross-3
By the way, Juergen,

On 01/25/2018 07:01 AM, Juergen Gross wrote:

> On 24/01/18 17:37, Hans van Kranenburg wrote:
>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>> On 23/01/18 05:46, Andy Smith wrote:
>>>> Hi,
>>>>
>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>
>>>> As far as I can see there is only direct kernel booting, i.e.
>>>> supplying "kernel = /path/to/vmlinux".
>>>>
>>>> In PV land there is pvgrub which is very convenient for being able
>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>> claims to add this feature:
>>>>
>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>
>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>> idea or has that method been abandoned?
>>>
>>> Still waiting for review.
>>
>> Since I wanted to try to test this, I scraped the patches off the
>> grub-devel list and did the following:
>>
>> 1. clone https://git.savannah.gnu.org/git/grub.git
>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>> to make it actually compile (see attachment)
>>
>> [...]
I needed to add this one also to make it compile. Didn't you run into this?

-$ git show
commit a3589cc5b5675f302ac430a6fdc0eed4e24ab2ba
Author: Hans van Kranenburg <[hidden email]>
Date:   Tue Jan 23 20:03:42 2018 +0100

    grub-module-verifier: Ignore all_video for xenpvh

    This solves the build failing with "Error: no symbol table and no
    .moddeps section"

    Also see:
    - 6371e9c10433578bb236a8284ddb9ce9e201eb59
    - https://savannah.gnu.org/bugs/?49012

diff --git a/util/grub-module-verifier.c b/util/grub-module-verifier.c
index 9179285a5..8c713e015 100644
--- a/util/grub-module-verifier.c
+++ b/util/grub-module-verifier.c
@@ -128,6 +128,7 @@ struct platform_whitelist {

 static struct platform_whitelist whitelists[] = {
   {"i386", "xen", (const char *[]) {"all_video", 0}},
+  {"i386", "xenpvh", (const char *[]) {"all_video", 0}},
   {"x86_64", "xen", (const char *[]) {"all_video", 0}},
   {"sparc64", "ieee1275", (const char *[]) {"all_video", 0}},


Hans

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
In reply to this post by Hans van Kranenburg-2
On 25/01/18 15:39, Hans van Kranenburg wrote:

> On 01/25/2018 03:31 PM, Juergen Gross wrote:
>> On 25/01/18 15:12, Hans van Kranenburg wrote:
>>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>>> Hi Hans,
>>>>>>
>>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>>>> And now I get console output and things happen. Only it can't find the disk.
>>>>>>
>>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>>
>>>>>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>>
>>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>>> okay outside grub with direct kernel boot.
>>>>>>
>>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>>
>>>>>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>>
>>>>>> But I think you have this don't you?
>>>>
>>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>>
>>>>> As the ACPI tables are found, I'd say yes. :-)
>>>>
>>>> dmesg output is pretty different when I boot directly with the kernel
>>>> and initrd copied on the dom0.
>>>>
>>>> Remember, it's the same kernel/initrd, and without grub in between it
>>>> boots with all vcpus network and disk.
>>>>
>>>> With grub in between, this at least does look suspicious:
>>>>
>>>> [    0.032110] PCI: System does not support PCI
>>>
>>> Eh, that's in both, stay awake Hans.
>>>
>>>> And yes, there are also no successful netfront lines.
>>>>
>>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>>> With grub: http://paste.debian.net/plainh/426bed60
>>>
>>> Or the diff between them, which shows what changes when inserting grub
>>> in between:
>>>
>>> http://paste.debian.net/plainh/52b2d618
>>>
>>> I must admit I don't know too much (yet) about all those changed lines,
>>> but this next is also a very interesting change?...
>>>
>>> -Booting paravirtualized kernel on Xen PVH
>>> +Booting paravirtualized kernel on Xen HVM
>>
>> Aah, yes, this should be the reason for the problems.
>
> Bingo!
>
>> I addressed the ACPI problem first. What is missing now is to set PVH
>> mode when booting via grub.
>>
>> So please stay tuned...
>
> Sure. At least I got as far as currently possible now, not bad. :-)

Thanks for doing those tests! I appreciate it very much.


Juergen

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
In reply to this post by Hans van Kranenburg-2
On 25/01/18 15:45, Hans van Kranenburg wrote:

> By the way, Juergen,
>
> On 01/25/2018 07:01 AM, Juergen Gross wrote:
>> On 24/01/18 17:37, Hans van Kranenburg wrote:
>>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>>> On 23/01/18 05:46, Andy Smith wrote:
>>>>> Hi,
>>>>>
>>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>>
>>>>> As far as I can see there is only direct kernel booting, i.e.
>>>>> supplying "kernel = /path/to/vmlinux".
>>>>>
>>>>> In PV land there is pvgrub which is very convenient for being able
>>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>>> claims to add this feature:
>>>>>
>>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>>
>>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>>> idea or has that method been abandoned?
>>>>
>>>> Still waiting for review.
>>>
>>> Since I wanted to try to test this, I scraped the patches off the
>>> grub-devel list and did the following:
>>>
>>> 1. clone https://git.savannah.gnu.org/git/grub.git
>>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>>> to make it actually compile (see attachment)
>>>
>>> [...]
> I needed to add this one also to make it compile. Didn't you run into this?

No. Your change looks sensible to me. Do you mind me adding that patch
to my series? If no, could you send it to me with your Signed-off-by: ?


Juergen

>
> -$ git show
> commit a3589cc5b5675f302ac430a6fdc0eed4e24ab2ba
> Author: Hans van Kranenburg <[hidden email]>
> Date:   Tue Jan 23 20:03:42 2018 +0100
>
>     grub-module-verifier: Ignore all_video for xenpvh
>
>     This solves the build failing with "Error: no symbol table and no
>     .moddeps section"
>
>     Also see:
>     - 6371e9c10433578bb236a8284ddb9ce9e201eb59
>     - https://savannah.gnu.org/bugs/?49012
>
> diff --git a/util/grub-module-verifier.c b/util/grub-module-verifier.c
> index 9179285a5..8c713e015 100644
> --- a/util/grub-module-verifier.c
> +++ b/util/grub-module-verifier.c
> @@ -128,6 +128,7 @@ struct platform_whitelist {
>
>  static struct platform_whitelist whitelists[] = {
>    {"i386", "xen", (const char *[]) {"all_video", 0}},
> +  {"i386", "xenpvh", (const char *[]) {"all_video", 0}},
>    {"x86_64", "xen", (const char *[]) {"all_video", 0}},
>    {"sparc64", "ieee1275", (const char *[]) {"all_video", 0}},
>
>
> Hans
>


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
On 01/25/2018 03:53 PM, Juergen Gross wrote:

> On 25/01/18 15:45, Hans van Kranenburg wrote:
>> By the way, Juergen,
>>
>> On 01/25/2018 07:01 AM, Juergen Gross wrote:
>>> On 24/01/18 17:37, Hans van Kranenburg wrote:
>>>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>>>> On 23/01/18 05:46, Andy Smith wrote:
>>>>>> Hi,
>>>>>>
>>>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>>>
>>>>>> As far as I can see there is only direct kernel booting, i.e.
>>>>>> supplying "kernel = /path/to/vmlinux".
>>>>>>
>>>>>> In PV land there is pvgrub which is very convenient for being able
>>>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>>>> claims to add this feature:
>>>>>>
>>>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>>>
>>>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>>>> idea or has that method been abandoned?
>>>>>
>>>>> Still waiting for review.
>>>>
>>>> Since I wanted to try to test this, I scraped the patches off the
>>>> grub-devel list and did the following:
>>>>
>>>> 1. clone https://git.savannah.gnu.org/git/grub.git
>>>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>>>> to make it actually compile (see attachment)
>>>>
>>>> [...]
>> I needed to add this one also to make it compile. Didn't you run into this?
>
> No. Your change looks sensible to me. Do you mind me adding that patch
> to my series? If no, could you send it to me with your Signed-off-by: ?

Sure, just did, with a bit reworded commit message.

K

> Juergen
>
>>
>> -$ git show
>> commit a3589cc5b5675f302ac430a6fdc0eed4e24ab2ba
>> Author: Hans van Kranenburg <[hidden email]>
>> Date:   Tue Jan 23 20:03:42 2018 +0100
>>
>>     grub-module-verifier: Ignore all_video for xenpvh
>>
>>     This solves the build failing with "Error: no symbol table and no
>>     .moddeps section"
>>
>>     Also see:
>>     - 6371e9c10433578bb236a8284ddb9ce9e201eb59
>>     - https://savannah.gnu.org/bugs/?49012
>>
>> diff --git a/util/grub-module-verifier.c b/util/grub-module-verifier.c
>> index 9179285a5..8c713e015 100644
>> --- a/util/grub-module-verifier.c
>> +++ b/util/grub-module-verifier.c
>> @@ -128,6 +128,7 @@ struct platform_whitelist {
>>
>>  static struct platform_whitelist whitelists[] = {
>>    {"i386", "xen", (const char *[]) {"all_video", 0}},
>> +  {"i386", "xenpvh", (const char *[]) {"all_video", 0}},
>>    {"x86_64", "xen", (const char *[]) {"all_video", 0}},
>>    {"sparc64", "ieee1275", (const char *[]) {"all_video", 0}},
>>
>>
>> Hans
>>
>
>
> _______________________________________________
> Xen-users mailing list
> [hidden email]
> https://lists.xenproject.org/mailman/listinfo/xen-users
>


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
In reply to this post by Juergen Gross-3
On 01/25/2018 03:31 PM, Juergen Gross wrote:

> On 25/01/18 15:12, Hans van Kranenburg wrote:
>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>> Hi Hans,
>>>>>
>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>>> And now I get console output and things happen. Only it can't find the disk.
>>>>>
>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>
>>>>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>
>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>> okay outside grub with direct kernel boot.
>>>>>
>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>
>>>>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>
>>>>> But I think you have this don't you?
>>>
>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>
>>>> As the ACPI tables are found, I'd say yes. :-)
>>>
>>> dmesg output is pretty different when I boot directly with the kernel
>>> and initrd copied on the dom0.
>>>
>>> Remember, it's the same kernel/initrd, and without grub in between it
>>> boots with all vcpus network and disk.
>>>
>>> With grub in between, this at least does look suspicious:
>>>
>>> [    0.032110] PCI: System does not support PCI
>>
>> Eh, that's in both, stay awake Hans.
>>
>>> And yes, there are also no successful netfront lines.
>>>
>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>> With grub: http://paste.debian.net/plainh/426bed60
>>
>> Or the diff between them, which shows what changes when inserting grub
>> in between:
>>
>> http://paste.debian.net/plainh/52b2d618
>>
>> I must admit I don't know too much (yet) about all those changed lines,
>> but this next is also a very interesting change?...
>>
>> -Booting paravirtualized kernel on Xen PVH
>> +Booting paravirtualized kernel on Xen HVM
>
> Aah, yes, this should be the reason for the problems.
>
> I addressed the ACPI problem first. What is missing now is to set PVH
> mode when booting via grub.
>
> So please stay tuned...

For my info... Is this something like flipping a bit somewhere, or will
it involve a whole lot more complicated wizardry?

Real question is of course if there's a PoC solution to apply that I can
use now? ;-] I'm throwing custom build kernels at my test environment,
and if I could save the time (and mistakes) of manually editing the
guest files and instead hit the pvgrub2+PVHv2 path many times to test...

Hans

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Juergen Gross-3
On 05/02/18 20:49, Hans van Kranenburg wrote:

> On 01/25/2018 03:31 PM, Juergen Gross wrote:
>> On 25/01/18 15:12, Hans van Kranenburg wrote:
>>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>>> Hi Hans,
>>>>>>
>>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>>>> And now I get console output and things happen. Only it can't find the disk.
>>>>>>
>>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>>
>>>>>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>>
>>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>>> okay outside grub with direct kernel boot.
>>>>>>
>>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>>
>>>>>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>>
>>>>>> But I think you have this don't you?
>>>>
>>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>>
>>>>> As the ACPI tables are found, I'd say yes. :-)
>>>>
>>>> dmesg output is pretty different when I boot directly with the kernel
>>>> and initrd copied on the dom0.
>>>>
>>>> Remember, it's the same kernel/initrd, and without grub in between it
>>>> boots with all vcpus network and disk.
>>>>
>>>> With grub in between, this at least does look suspicious:
>>>>
>>>> [    0.032110] PCI: System does not support PCI
>>>
>>> Eh, that's in both, stay awake Hans.
>>>
>>>> And yes, there are also no successful netfront lines.
>>>>
>>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>>> With grub: http://paste.debian.net/plainh/426bed60
>>>
>>> Or the diff between them, which shows what changes when inserting grub
>>> in between:
>>>
>>> http://paste.debian.net/plainh/52b2d618
>>>
>>> I must admit I don't know too much (yet) about all those changed lines,
>>> but this next is also a very interesting change?...
>>>
>>> -Booting paravirtualized kernel on Xen PVH
>>> +Booting paravirtualized kernel on Xen HVM
>>
>> Aah, yes, this should be the reason for the problems.
>>
>> I addressed the ACPI problem first. What is missing now is to set PVH
>> mode when booting via grub.
>>
>> So please stay tuned...
>
> For my info... Is this something like flipping a bit somewhere, or will
> it involve a whole lot more complicated wizardry?
>
> Real question is of course if there's a PoC solution to apply that I can
> use now? ;-] I'm throwing custom build kernels at my test environment,
> and if I could save the time (and mistakes) of manually editing the
> guest files and instead hit the pvgrub2+PVHv2 path many times to test...

So I looked into this briefly and discovered that my memory really needs
the backup I have on my hard disk: you want commit
418492ba40b2c2bbdaf1a169aac5b1673bde8189 which was for 4.15.


Juergen

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: pvgrub2(-like?) booting methods for PVHv2 guests

Hans van Kranenburg-2
On 02/06/2018 09:35 AM, Juergen Gross wrote:

> On 05/02/18 20:49, Hans van Kranenburg wrote:
>> On 01/25/2018 03:31 PM, Juergen Gross wrote:
>>> On 25/01/18 15:12, Hans van Kranenburg wrote:
>>>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>>>> Hi Hans,
>>>>>>>
>>>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>>>>> And now I get console output and things happen. Only it can't find the disk.
>>>>>>>
>>>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>>>
>>>>>>>     <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>>>
>>>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>>>> okay outside grub with direct kernel boot.
>>>>>>>
>>>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>>>
>>>>>>>     <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>>>
>>>>>>> But I think you have this don't you?
>>>>>
>>>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>>>
>>>>>> As the ACPI tables are found, I'd say yes. :-)
>>>>>
>>>>> dmesg output is pretty different when I boot directly with the kernel
>>>>> and initrd copied on the dom0.
>>>>>
>>>>> Remember, it's the same kernel/initrd, and without grub in between it
>>>>> boots with all vcpus network and disk.
>>>>>
>>>>> With grub in between, this at least does look suspicious:
>>>>>
>>>>> [    0.032110] PCI: System does not support PCI
>>>>
>>>> Eh, that's in both, stay awake Hans.
>>>>
>>>>> And yes, there are also no successful netfront lines.
>>>>>
>>>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>>>> With grub: http://paste.debian.net/plainh/426bed60
>>>>
>>>> Or the diff between them, which shows what changes when inserting grub
>>>> in between:
>>>>
>>>> http://paste.debian.net/plainh/52b2d618
>>>>
>>>> I must admit I don't know too much (yet) about all those changed lines,
>>>> but this next is also a very interesting change?...
>>>>
>>>> -Booting paravirtualized kernel on Xen PVH
>>>> +Booting paravirtualized kernel on Xen HVM
>>>
>>> Aah, yes, this should be the reason for the problems.
>>>
>>> I addressed the ACPI problem first. What is missing now is to set PVH
>>> mode when booting via grub.
>>>
>>> So please stay tuned...
>>
>> For my info... Is this something like flipping a bit somewhere, or will
>> it involve a whole lot more complicated wizardry?
>>
>> Real question is of course if there's a PoC solution to apply that I can
>> use now? ;-] I'm throwing custom build kernels at my test environment,
>> and if I could save the time (and mistakes) of manually editing the
>> guest files and instead hit the pvgrub2+PVHv2 path many times to test...
>
> So I looked into this briefly and discovered that my memory really needs
> the backup I have on my hard disk: you want commit
> 418492ba40b2c2bbdaf1a169aac5b1673bde8189 which was for 4.15.

Aha. Well, I can as well better jump to 4.15 now, since picking that
patch on 4.14.17 shows I need more other patches it depends on, related
to struct x86_legacy_features and x86_hyper_init reorganization.

So, I just built a 4.15.1 kernel (using gcc 7.2.0), which definitely has
this one in it:
  418492ba40b2 x86/virt/xen: Use guest_late_init to detect Xen PVH guest

I can boot via pvgrub2 (the Xen that's running now is current
stable-4.10 plus the RSDP for PVH guest near 4GB patch).

But, it still tells me...
  Booting paravirtualized kernel on Xen HVM

...and then hangs somewhere halfway.

Full boot log here: http://paste.debian.net/plainh/ae3ea14b

Hans

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
12