VGA passthrough on unstable

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

VGA passthrough on unstable

Liwei Xie
(Pardon me if I broke any rules or this post doesn't belong to this list)

I forward patched yesterday's copy of the unstable repo in an attempt to
get VGA passthrough working for a Nvidia GTX460 on a HVM DomU.
(Surprisingly, it wasn't hard at all to get the patches in - getting
them to work is another ongoing story)

The patches were based on this post:
http://wiki.xensource.com/xenwiki/XenVGAPassthrough

I dumped the card's BIOS using nvtool as per the post's instructions.
The only major change was that the BIOS ROM setup code was shifted
into its own file and some adjustment was needed for that. I'll create
a new patchset once I get this fully working. I also hope to push this
into being accepted by the devs for future releases by allowing the
user to configure this option and provide his own BIOS dump (instead
of hardcoding everything).

My card doesn't seem to support FLR, since xend complains about being
unable to reset the device.

With or without the
qemu-claim-cycle-for-secondary-gfx-passthrough.patch, the secondary
graphics card does not show anything at all (maybe because it is
behind a PCI-E bridge?), and xl list seems to show the guest consuming
all CPU resources.

However, using the primary graphics card, again with or without the
secondary passthrough patch, it actually managed to partially work
booting up the Windows 7 install. It manages to reach the pulsating
windows logo before BSOD-ing with 0x0000000A (IRQ_NOT_LESS_OR_EQUAL).
Meanwhile, the logs show a lot of:

   pt_pci_read_config: Error: Failed to read register with invalid
access size alignment. [00:05.0][Offset:0eh][Length:4]
   pt_pci_read_config: Error: Failed to read register with invalid
access size alignment. [00:06.0][Offset:0eh][Length:4]

Reduced the amount of allocated memory to 4096MB will introduce video
corruption in the pulsating logo, followed by the same BSOD.

Reducing the amount of allocated memory to below 2048MB or 1024MB will
cause a 0x000000A5 BSOD (BIOS ACPI not compliant).

Attempting to boot the same setup (with 8192MB of memory) with vCPU
set to 8 produced slightly different behaviour. Qemu seems to crash
and reboot a few seconds after the pulsating windows logo appears
(earlier than before the BSOD appeared before). At this point, it
should be noted that the 8 vCPU and 8192MB configuration worked with
4.0. I couldn't test it in the patched unstable because VNC will only
produce a white screen (wrong VGA BIOS executed?).

Attempting to boot Ubuntu also produced similar results, except the
log now shows errors similar to (I did not copy out the log before
they were overwritten):

   Error: Failed to write register with invalid access size

The boot up seems to fail at trying to read from the emulated SATA
drive though, something about interrupt lost, and keeps on trying
again and again forever.

With the above tests, I also unscientifically fiddled around with the
pci_power_mgmt, pci_msitranslate, hap, hpet, pae, apic, acpi and
viridian toggleable settings.

I made no functional changes to the patches however, so maybe there
was something that I had to change in order to customise it for my
card. It'd be great if someone points that out to me if it is true.

I cannot use my USB keyboard and mouse at all in all my tests with
unstable, USB controller is passed in. USB works on 4.0 in the windows
installer, but I haven't tested them before the installer boots, so it
may be possible that passthrough is broken with my setup (does the
BIOS initialise USB peripherals for use during boot?).

So how should I proceed on from here?

Setup details as follows:

EVGA P55 Classified
Intel i7 860
8GB Memory
2x Palit Nvidia GTX460 (Primary and secondary)

Debian Squeeze
Dom0 is 2.6.32+29 (From repo)

PCI devices (Only those bound to xen_pciback are listed):
(It should be noted that except for the primary GFX, all other devices
are behind a NF200 PCI-E bridge)
0b:00.0 - Secondary GFX
0b:00.1 - Secondary GFX audio
01:00.0 - Primary GFX
01:00.1 - Primary GFX audio
0e:00.0 - Multiport network card
04:00.0 - Singleport network card
00:1a.0 - USB2 root
00:1b.0 - HD audio device
00:1d.0 - USB2 root

PCI devices combinations tested (in each case, the audio is
passthroughed with the GFX):
(OT: Why doesn't multi-device BDF binding work on xen_pciback?)
Primary GFX only
Secondary GFX only
Primary GFX + Secondary GFX
Primary GFX + others
Secondary GFX + others
Primary GFX + Secondary GFX + others

Attached: Log files produced by xend and qemu and config files
(Sorry that only one set of logs are available, wasn't thinking
properly when executing rm *.log)

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel

~qemu-dm-W7Test.log (11K) Download Attachment
~W7Test.cfg (1K) Download Attachment
~xend-config.sxp (15K) Download Attachment
~xl-W7Test.log (160 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Pasi Kärkkäinen
On Thu, May 05, 2011 at 07:33:46PM +0800, Liwei wrote:

> (Pardon me if I broke any rules or this post doesn't belong to this list)
>
> I forward patched yesterday's copy of the unstable repo in an attempt to
> get VGA passthrough working for a Nvidia GTX460 on a HVM DomU.
> (Surprisingly, it wasn't hard at all to get the patches in - getting
> them to work is another ongoing story)
>
> The patches were based on this post:
> http://wiki.xensource.com/xenwiki/XenVGAPassthrough
>
> I dumped the card's BIOS using nvtool as per the post's instructions.
> The only major change was that the BIOS ROM setup code was shifted
> into its own file and some adjustment was needed for that. I'll create
> a new patchset once I get this fully working. I also hope to push this
> into being accepted by the devs for future releases by allowing the
> user to configure this option and provide his own BIOS dump (instead
> of hardcoding everything).
>

Being able to specify which vgabios file to load would be great..
Feel free to send patches!


> My card doesn't seem to support FLR, since xend complains about being
> unable to reset the device.
>
> With or without the
> qemu-claim-cycle-for-secondary-gfx-passthrough.patch, the secondary
> graphics card does not show anything at all (maybe because it is
> behind a PCI-E bridge?), and xl list seems to show the guest consuming
> all CPU resources.
>
> However, using the primary graphics card, again with or without the
> secondary passthrough patch, it actually managed to partially work
> booting up the Windows 7 install. It manages to reach the pulsating
> windows logo before BSOD-ing with 0x0000000A (IRQ_NOT_LESS_OR_EQUAL).
> Meanwhile, the logs show a lot of:
>
>    pt_pci_read_config: Error: Failed to read register with invalid
> access size alignment. [00:05.0][Offset:0eh][Length:4]
>    pt_pci_read_config: Error: Failed to read register with invalid
> access size alignment. [00:06.0][Offset:0eh][Length:4]
>

Hmm..

So did you apply the vBar == pBar patches ?
Did you modify them to fit your hardware?

-- Pasi


> Reduced the amount of allocated memory to 4096MB will introduce video
> corruption in the pulsating logo, followed by the same BSOD.
>
> Reducing the amount of allocated memory to below 2048MB or 1024MB will
> cause a 0x000000A5 BSOD (BIOS ACPI not compliant).
>
> Attempting to boot the same setup (with 8192MB of memory) with vCPU
> set to 8 produced slightly different behaviour. Qemu seems to crash
> and reboot a few seconds after the pulsating windows logo appears
> (earlier than before the BSOD appeared before). At this point, it
> should be noted that the 8 vCPU and 8192MB configuration worked with
> 4.0. I couldn't test it in the patched unstable because VNC will only
> produce a white screen (wrong VGA BIOS executed?).
>
> Attempting to boot Ubuntu also produced similar results, except the
> log now shows errors similar to (I did not copy out the log before
> they were overwritten):
>
>    Error: Failed to write register with invalid access size
>
> The boot up seems to fail at trying to read from the emulated SATA
> drive though, something about interrupt lost, and keeps on trying
> again and again forever.
>
> With the above tests, I also unscientifically fiddled around with the
> pci_power_mgmt, pci_msitranslate, hap, hpet, pae, apic, acpi and
> viridian toggleable settings.
>
> I made no functional changes to the patches however, so maybe there
> was something that I had to change in order to customise it for my
> card. It'd be great if someone points that out to me if it is true.
>
> I cannot use my USB keyboard and mouse at all in all my tests with
> unstable, USB controller is passed in. USB works on 4.0 in the windows
> installer, but I haven't tested them before the installer boots, so it
> may be possible that passthrough is broken with my setup (does the
> BIOS initialise USB peripherals for use during boot?).
>
> So how should I proceed on from here?
>
> Setup details as follows:
>
> EVGA P55 Classified
> Intel i7 860
> 8GB Memory
> 2x Palit Nvidia GTX460 (Primary and secondary)
>
> Debian Squeeze
> Dom0 is 2.6.32+29 (From repo)
>
> PCI devices (Only those bound to xen_pciback are listed):
> (It should be noted that except for the primary GFX, all other devices
> are behind a NF200 PCI-E bridge)
> 0b:00.0 - Secondary GFX
> 0b:00.1 - Secondary GFX audio
> 01:00.0 - Primary GFX
> 01:00.1 - Primary GFX audio
> 0e:00.0 - Multiport network card
> 04:00.0 - Singleport network card
> 00:1a.0 - USB2 root
> 00:1b.0 - HD audio device
> 00:1d.0 - USB2 root
>
> PCI devices combinations tested (in each case, the audio is
> passthroughed with the GFX):
> (OT: Why doesn't multi-device BDF binding work on xen_pciback?)
> Primary GFX only
> Secondary GFX only
> Primary GFX + Secondary GFX
> Primary GFX + others
> Secondary GFX + others
> Primary GFX + Secondary GFX + others
>
> Attached: Log files produced by xend and qemu and config files
> (Sorry that only one set of logs are available, wasn't thinking
> properly when executing rm *.log)





> _______________________________________________
> Xen-devel mailing list
> [hidden email]
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Liwei Xie
On 5 May 2011 23:20, Pasi Kärkkäinen <[hidden email]> wrote:
>
> Being able to specify which vgabios file to load would be great..
> Feel free to send patches!

I dumped the BIOS of my Palit Nvidia GTX480 card with the nvdump tool.
Not sure if its legal to post it online though. Will send the patches
when there's substantial progress.

>
>
>> However, using the primary graphics card, again with or without the
>> secondary passthrough patch, it actually managed to partially work
>> booting up the Windows 7 install. It manages to reach the pulsating
>> windows logo before BSOD-ing with 0x0000000A (IRQ_NOT_LESS_OR_EQUAL).
>> Meanwhile, the logs show a lot of:
>>
>>    pt_pci_read_config: Error: Failed to read register with invalid
>> access size alignment. [00:05.0][Offset:0eh][Length:4]
>>    pt_pci_read_config: Error: Failed to read register with invalid
>> access size alignment. [00:06.0][Offset:0eh][Length:4]
>>
>
> Hmm..
>
> So did you apply the vBar == pBar patches ?
> Did you modify them to fit your hardware?
>
> -- Pasi

http://markmail.org/message/7gb7djbmlaxruaai

The patch that I use is based on Weidong's pBAR==vBAR patch. From what
I hear it automatically remaps any vBAR access? I'm not familiar with
low level hardware architecture, so I'd need some prodding to
understand what's going on in the patched code.

So where do I modify them to fit my hardware?

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Pasi Kärkkäinen
On Fri, May 06, 2011 at 12:56:30AM +0800, Liwei wrote:
> On 5 May 2011 23:20, Pasi Kärkkäinen <[hidden email]> wrote:
> >
> > Being able to specify which vgabios file to load would be great..
> > Feel free to send patches!
>
> I dumped the BIOS of my Palit Nvidia GTX480 card with the nvdump tool.
> Not sure if its legal to post it online though. Will send the patches
> when there's substantial progress.
>

Yeah I didn't mean you to post the actual BIOS image file,
only the Xen patches that support loading the BIOS image from specified file.
(so that people don't have to hardcode the BIOS image to custom binaries).


> >
> >
> >> However, using the primary graphics card, again with or without the
> >> secondary passthrough patch, it actually managed to partially work
> >> booting up the Windows 7 install. It manages to reach the pulsating
> >> windows logo before BSOD-ing with 0x0000000A (IRQ_NOT_LESS_OR_EQUAL).
> >> Meanwhile, the logs show a lot of:
> >>
> >>    pt_pci_read_config: Error: Failed to read register with invalid
> >> access size alignment. [00:05.0][Offset:0eh][Length:4]
> >>    pt_pci_read_config: Error: Failed to read register with invalid
> >> access size alignment. [00:06.0][Offset:0eh][Length:4]
> >>
> >
> > Hmm..
> >
> > So did you apply the vBar == pBar patches ?
> > Did you modify them to fit your hardware?
> >
> > -- Pasi
>
> http://markmail.org/message/7gb7djbmlaxruaai
>
> The patch that I use is based on Weidong's pBAR==vBAR patch. From what
> I hear it automatically remaps any vBAR access? I'm not familiar with
> low level hardware architecture, so I'd need some prodding to
> understand what's going on in the patched code.
>
> So where do I modify them to fit my hardware?

I think other people who tried those patches (and got them working)
had to modify the vbar/pbar ranges to match their hardware..

btw the error message above mentions alignment.. did you try the
alignment option for xen-pciback?

-- Pasi


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Liwei Xie
On 6 May 2011 01:06, Pasi Kärkkäinen <[hidden email]> wrote:
>
> Yeah I didn't mean you to post the actual BIOS image file,
> only the Xen patches that support loading the BIOS image from specified file.
> (so that people don't have to hardcode the BIOS image to custom binaries).
>

That I have not started and will probably need some work integrating a
hex converter (or maybe require the user to supply the converted
binary) and finding out how xen/qemu handles the configuration files.

>
> I think other people who tried those patches (and got them working)
> had to modify the vbar/pbar ranges to match their hardware..
>
> btw the error message above mentions alignment.. did you try the
> alignment option for xen-pciback?
>
> -- Pasi
>
>

Where do I modify the ranges? Is it the bunch of DWordMemory macros in
dsdt.asl? And how do I determine the range that I must use?

I did a quick test by appending pci=resource_alignment=[all the
pciback devices] to my boot command, it didn't seem to trigger any
changes in my bootlog nor the guest booting, is that the right option?

Come to think of it, why are there errors for BDF 00:05.0 and 00:06.0
when only 01:00.0 and 01:00.1 is passed in?

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Pasi Kärkkäinen
On Fri, May 06, 2011 at 01:59:29AM +0800, Liwei wrote:

> On 6 May 2011 01:06, Pasi Kärkkäinen <[hidden email]> wrote:
> >
> > Yeah I didn't mean you to post the actual BIOS image file,
> > only the Xen patches that support loading the BIOS image from specified file.
> > (so that people don't have to hardcode the BIOS image to custom binaries).
> >
>
> That I have not started and will probably need some work integrating a
> hex converter (or maybe require the user to supply the converted
> binary) and finding out how xen/qemu handles the configuration files.
>

Ok.

IMHO it's fine to require some pre-processing by the user,
since it's pretty manual step anyway to get the image captured from the card..

> >
> > I think other people who tried those patches (and got them working)
> > had to modify the vbar/pbar ranges to match their hardware..
> >
> > btw the error message above mentions alignment.. did you try the
> > alignment option for xen-pciback?
> >
> > -- Pasi
> >
> >
>
> Where do I modify the ranges? Is it the bunch of DWordMemory macros in
> dsdt.asl? And how do I determine the range that I must use?
>

I'm not sure, unfortunately. I haven't looked at the patches.

> I did a quick test by appending pci=resource_alignment=[all the
> pciback devices] to my boot command, it didn't seem to trigger any
> changes in my bootlog nor the guest booting, is that the right option?
>

So I assume you've read:
http://wiki.xen.org/xenwiki/XenPCIpassthrough

Which dom0 kernel are you using? is pciback a module, or built-in?

> Come to think of it, why are there errors for BDF 00:05.0 and 00:06.0
> when only 01:00.0 and 01:00.1 is passed in?
>

What pciback mode are you using?

-- Pasi


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Liwei Xie
On 6 May 2011 02:12, Pasi Kärkkäinen <[hidden email]> wrote:
>
> IMHO it's fine to require some pre-processing by the user,
> since it's pretty manual step anyway to get the image captured from the card..
>

Was thinking of that too, that'll simplify the patch to just the
configuration and binary loading.

>
> So I assume you've read:
> http://wiki.xen.org/xenwiki/XenPCIpassthrough
>
> Which dom0 kernel are you using? is pciback a module, or built-in?
>

Yeah, I've read that and tried both the pci=resource_alignment and
reassigndev options. Unless it doesn't produce any obvious log entries
in dmesg, I don't think they did anything.

That'd be 2.6.32-5-xen-amd64 from the debian squeeze repository, using
the builtin xen_pciback.

>> Come to think of it, why are there errors for BDF 00:05.0 and 00:06.0
>> when only 01:00.0 and 01:00.1 is passed in?
>>
>
> What pciback mode are you using?
>
> -- Pasi
>

I have xen-pciback.permissive in my boot argument, so I assume that
means its permissive?

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Pasi Kärkkäinen
On Fri, May 06, 2011 at 02:28:29AM +0800, Liwei wrote:
> On 6 May 2011 02:12, Pasi Kärkkäinen <[hidden email]> wrote:
> >
> > IMHO it's fine to require some pre-processing by the user,
> > since it's pretty manual step anyway to get the image captured from the card..
> >
>
> Was thinking of that too, that'll simplify the patch to just the
> configuration and binary loading.
>

Indeed.

> >
> > So I assume you've read:
> > http://wiki.xen.org/xenwiki/XenPCIpassthrough
> >
> > Which dom0 kernel are you using? is pciback a module, or built-in?
> >
>
> Yeah, I've read that and tried both the pci=resource_alignment and
> reassigndev options. Unless it doesn't produce any obvious log entries
> in dmesg, I don't think they did anything.
>
> That'd be 2.6.32-5-xen-amd64 from the debian squeeze repository, using
> the builtin xen_pciback.
>

Ok. So pci=resource_alignment= should do it..


> >> Come to think of it, why are there errors for BDF 00:05.0 and 00:06.0
> >> when only 01:00.0 and 01:00.1 is passed in?
> >>
> >
> > What pciback mode are you using?
> >
> > -- Pasi
> >
>
> I have xen-pciback.permissive in my boot argument, so I assume that
> means its permissive?

Uhm, I meant the CONFIG_XEN_PCIDEV_BACKEND_PASS,
CONFIG_XEN_PCIDEV_BACKEND_VPCI etc in kernel .config..

If it's VPCI then the PCI ID's in the VM are different from dom0..
if it's PASS then PCI IDs will be the same.

-- Pasi



_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Liwei Xie
On 6 May 2011 02:33, Pasi Kärkkäinen <[hidden email]> wrote:
>
> Ok. So pci=resource_alignment= should do it..
>

Should it produce anything in the logs?

>
> Uhm, I meant the CONFIG_XEN_PCIDEV_BACKEND_PASS,
> CONFIG_XEN_PCIDEV_BACKEND_VPCI etc in kernel .config..
>
> If it's VPCI then the PCI ID's in the VM are different from dom0..
> if it's PASS then PCI IDs will be the same.
>
> -- Pasi
>

I have this, so I assume that means that the IDs are different? So
those statements are referring to the graphics card and its audio
device. With the pci=resource_alignment option, this is still
happening, what gives?

CONFIG_XEN_PCIDEV_BACKEND_VPCI=y
# CONFIG_XEN_PCIDEV_BACKEND_PASS is not set
# CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set
# CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Pasi Kärkkäinen
On Fri, May 06, 2011 at 02:42:33AM +0800, Liwei wrote:
> On 6 May 2011 02:33, Pasi Kärkkäinen <[hidden email]> wrote:
> >
> > Ok. So pci=resource_alignment= should do it..
> >
>
> Should it produce anything in the logs?
>

Not sure.

> >
> > Uhm, I meant the CONFIG_XEN_PCIDEV_BACKEND_PASS,
> > CONFIG_XEN_PCIDEV_BACKEND_VPCI etc in kernel .config..
> >
> > If it's VPCI then the PCI ID's in the VM are different from dom0..
> > if it's PASS then PCI IDs will be the same.
> >
> > -- Pasi
> >
>
> I have this, so I assume that means that the IDs are different? So
> those statements are referring to the graphics card and its audio
> device. With the pci=resource_alignment option, this is still
> happening, what gives?
>

pci=resource_alignment has nothing to do with changing PCI IDs.

> CONFIG_XEN_PCIDEV_BACKEND_VPCI=y
> # CONFIG_XEN_PCIDEV_BACKEND_PASS is not set
> # CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set
> # CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set

Ok, so maybe you should rebuild a kernel with PASS instead of VPCI ..
(to get matching PCI IDs in the VM).

-- Pasi


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Liwei Xie
On 6 May 2011 02:49, Pasi Kärkkäinen <[hidden email]> wrote:
> Ok, so maybe you should rebuild a kernel with PASS instead of VPCI ..
> (to get matching PCI IDs in the VM).

Back with a newly compiled 2.6.32.39 from Jeremy's 2.6.32.x branch.
(Was stuck for days trying to get Konrad's 2.6.39 branch to work as
dom0 only to realise that its not supported yet) Also upgraded xen to
yesterday's revision.

Nothing seems to have changed. I'm still getting:
pt_pci_read_config: Error: Failed to read register with invalid access
size alignment. [00:05.0][Offset:0eh][Length:4]
pt_pci_read_config: Error: Failed to read register with invalid access
size alignment. [00:06.0][Offset:0eh][Length:4]

The PCI IDs did not change even though I've switched PCIDEV to pass:
# cat /boot/config-2.6.32.39 |grep PCIDEV
CONFIG_XEN_PCIDEV_FRONTEND=y
CONFIG_XEN_PCIDEV_BACKEND=y
# CONFIG_XEN_PCIDEV_BACKEND_VPCI is not set
CONFIG_XEN_PCIDEV_BACKEND_PASS=y
# CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set
# CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set
# CONFIG_XEN_PCIDEV_BE_DEBUG is not set

With the following config, I managed to prevent the vm from
continuously rebooting, but it just stays at the pulsating windows
logo doing nothing:
W7Test                                       9  4087     1     -b----      37.9

builder='hvm'
memory = 4096
vcpus = 1
shadow_memory = 32
name = "W7Test"
vif = [ 'type=ioemu, bridge=xenbr0' ]
viridian = 1
acpi = 1
apic = 1
pae = 1
hpet = 1
hap = 1
disk = [ 'phy:/dev/mapper/VMStore-W7Test,hdb,w', 'file:/w7.iso,hda:cdrom,r' ]
boot="cd"
sdl=0
vnc=1
vncconsole=1
vncpasswd=''
serial='pty'
usbdevice='tablet'
gfx_passthru = 1
pci = [ '01:00.0', '01:00.1' ]
pci_msitranslate = 1

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

JavMV
In reply to this post by Pasi Kärkkäinen
Hi all,
Im trying to get VGA passthrough working for Nvidia GTX460 (like u), but i can't apply any patches without errors... im following these steps: "http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00441.html" and using those patches, but I am only able to apply the first and the third patch succesfully while the others seems to not fit with the actual version of xen-unstable code
(Some Hunks say succeed and some say failed, for example the 4th patch (hvmloader.patch) returns
   # patch -p0 < hvmloader.patch
  patching file tools/firmware/hvmloader/hvmloader.c
  Hunk #1 succeeded at 114 with fuzz 2 (offset -1 lines).
  Hunk #2 FAILED at 220.
  Hunk #3 succeeded at 436 (offset -257 lines).
  1 out of 3 hunks FAILED -- saving rejects to file tools/firmware/hvmloader/hvmloader.c.rej

the path "makefile" neither works.. )
 
Also i have tried to use the patches that Mr Teo En Ming attach at the end of this post "http://lists.xensource.com/archives/html/xen-devel/2009-08/msg01176.html" but I've got similarly errors..

Could you post how do you proceed to make this patches work? Did u make your own patches?
I really need to make this work..

Thank you for your time.


Pasi Kärkkäinen wrote
On Thu, May 05, 2011 at 07:33:46PM +0800, Liwei wrote:
> (Pardon me if I broke any rules or this post doesn't belong to this list)
>
> I forward patched yesterday's copy of the unstable repo in an attempt to
> get VGA passthrough working for a Nvidia GTX460 on a HVM DomU.
> (Surprisingly, it wasn't hard at all to get the patches in - getting
> them to work is another ongoing story)
>
> The patches were based on this post:
> http://wiki.xensource.com/xenwiki/XenVGAPassthrough
>
> I dumped the card's BIOS using nvtool as per the post's instructions.
> The only major change was that the BIOS ROM setup code was shifted
> into its own file and some adjustment was needed for that. I'll create
> a new patchset once I get this fully working. I also hope to push this
> into being accepted by the devs for future releases by allowing the
> user to configure this option and provide his own BIOS dump (instead
> of hardcoding everything).
>

Being able to specify which vgabios file to load would be great..
Feel free to send patches!


> My card doesn't seem to support FLR, since xend complains about being
> unable to reset the device.
>
> With or without the
> qemu-claim-cycle-for-secondary-gfx-passthrough.patch, the secondary
> graphics card does not show anything at all (maybe because it is
> behind a PCI-E bridge?), and xl list seems to show the guest consuming
> all CPU resources.
>
> However, using the primary graphics card, again with or without the
> secondary passthrough patch, it actually managed to partially work
> booting up the Windows 7 install. It manages to reach the pulsating
> windows logo before BSOD-ing with 0x0000000A (IRQ_NOT_LESS_OR_EQUAL).
> Meanwhile, the logs show a lot of:
>
>    pt_pci_read_config: Error: Failed to read register with invalid
> access size alignment. [00:05.0][Offset:0eh][Length:4]
>    pt_pci_read_config: Error: Failed to read register with invalid
> access size alignment. [00:06.0][Offset:0eh][Length:4]
>

Hmm..

So did you apply the vBar == pBar patches ?
Did you modify them to fit your hardware?

-- Pasi


> Reduced the amount of allocated memory to 4096MB will introduce video
> corruption in the pulsating logo, followed by the same BSOD.
>
> Reducing the amount of allocated memory to below 2048MB or 1024MB will
> cause a 0x000000A5 BSOD (BIOS ACPI not compliant).
>
> Attempting to boot the same setup (with 8192MB of memory) with vCPU
> set to 8 produced slightly different behaviour. Qemu seems to crash
> and reboot a few seconds after the pulsating windows logo appears
> (earlier than before the BSOD appeared before). At this point, it
> should be noted that the 8 vCPU and 8192MB configuration worked with
> 4.0. I couldn't test it in the patched unstable because VNC will only
> produce a white screen (wrong VGA BIOS executed?).
>
> Attempting to boot Ubuntu also produced similar results, except the
> log now shows errors similar to (I did not copy out the log before
> they were overwritten):
>
>    Error: Failed to write register with invalid access size
>
> The boot up seems to fail at trying to read from the emulated SATA
> drive though, something about interrupt lost, and keeps on trying
> again and again forever.
>
> With the above tests, I also unscientifically fiddled around with the
> pci_power_mgmt, pci_msitranslate, hap, hpet, pae, apic, acpi and
> viridian toggleable settings.
>
> I made no functional changes to the patches however, so maybe there
> was something that I had to change in order to customise it for my
> card. It'd be great if someone points that out to me if it is true.
>
> I cannot use my USB keyboard and mouse at all in all my tests with
> unstable, USB controller is passed in. USB works on 4.0 in the windows
> installer, but I haven't tested them before the installer boots, so it
> may be possible that passthrough is broken with my setup (does the
> BIOS initialise USB peripherals for use during boot?).
>
> So how should I proceed on from here?
>
> Setup details as follows:
>
> EVGA P55 Classified
> Intel i7 860
> 8GB Memory
> 2x Palit Nvidia GTX460 (Primary and secondary)
>
> Debian Squeeze
> Dom0 is 2.6.32+29 (From repo)
>
> PCI devices (Only those bound to xen_pciback are listed):
> (It should be noted that except for the primary GFX, all other devices
> are behind a NF200 PCI-E bridge)
> 0b:00.0 - Secondary GFX
> 0b:00.1 - Secondary GFX audio
> 01:00.0 - Primary GFX
> 01:00.1 - Primary GFX audio
> 0e:00.0 - Multiport network card
> 04:00.0 - Singleport network card
> 00:1a.0 - USB2 root
> 00:1b.0 - HD audio device
> 00:1d.0 - USB2 root
>
> PCI devices combinations tested (in each case, the audio is
> passthroughed with the GFX):
> (OT: Why doesn't multi-device BDF binding work on xen_pciback?)
> Primary GFX only
> Secondary GFX only
> Primary GFX + Secondary GFX
> Primary GFX + others
> Secondary GFX + others
> Primary GFX + Secondary GFX + others
>
> Attached: Log files produced by xend and qemu and config files
> (Sorry that only one set of logs are available, wasn't thinking
> properly when executing rm *.log)





> _______________________________________________
> Xen-devel mailing list
> [hidden email]
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
JavMV
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

JavMV
Hi again,
I have made some progress, as I posted yesterday I am following these steps :http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00441.html
I have applied the passthrough patch manually cause there are some errors in the file, after that I have applied the dsdt patch succesfully. The patches for the Makefile and hvmloader.c do not work but I can compile the tools without errors so maybe i dont need it. In fact hvmloader.patch seems to be the xen-load-vbios-file.patch that Teo En Ming posted, but taking a look to the code it seems that this patch is already applied in xen-unstable-4.2, or am I wrong? (This is important... anyone knows if this is correct?)

Well now i have applied 2 patches and have compiled XEN without problems, the next steps are:
- Extract the firmware of my GTX460 nvidia card with nvflash tool as vgabios-pt.bin
- Copy it into tools/firmware/vgabios/
- I have the lines gfx_passthru = 1, and pci=['01:00.0'] on my Windows7VMachine.cfg
- Bind the graphics card to pci-stub:
    echo "10de 0e22" > /sys/bus/pci/drivers/pci-stub/new_id
    echo "0000:01:00.0" > /sys/bus/pci/devices/0000:01:00.0/driver/unbind    
    echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind
-Finally create the virtual machine

At this point I have to run VNC in other PC to see my Virtual Machine, but it doesn't work, i only see the qemu console...

In my qemu log file i can see this:

domid: 1
config qemu network with xen bridge for  tap1.0 eth0
Using file /home/xen/w7/diskw7.img in read-write mode
Using file /dev/cdrom in read-only mode
Watching /local/domain/0/device-model/1/logdirty/cmd
Watching /local/domain/0/device-model/1/command
Watching /local/domain/1/cpu
qemu_map_cache_init nr_buckets = 4000 size 327680
shared page at pfn feffd
buffered io page at pfn feffb
Guest uuid = 1a8ecd4e-3514-652f-615b-eaba86c6a43b
Time offset set 0
Register xen platform.
Done register platform.
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
xs_read(/local/domain/0/device-model/1/xen_extended_power_mgmt): read error
xs_read(): vncpasswd get error. /vm/1a8ecd4e-3514-652f-615b-eaba86c6a43b/vncpasswd.
medium change watch on `hdc' (index: 1): /dev/cdrom
I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
Log-dirty: no command yet.
vcpu-set: watch node error.
xs_read(/local/domain/1/log-throttling): read error
qemu: ignoring not-understood drive `/local/domain/1/log-throttling'
medium change watch on `/local/domain/1/log-throttling' - unknown device, ignored
dm-command: hot insert pass-through pci dev
register_real_device: Assigning real physical device 01:00.0 ...
register_real_device: Enable MSI translation via per device option
register_real_device: Disable power management
pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No such file or directory: 0x1:0x0.0x0
pt_register_regions: IO region registered (size=0x02000000 base_addr=0xec000000)
pt_register_regions: IO region registered (size=0x08000000 base_addr=0xe000000c)
pt_register_regions: IO region registered (size=0x04000000 base_addr=0xe800000c)
pt_register_regions: IO region registered (size=0x00000080 base_addr=0x00002001)
pt_register_regions: Expansion ROM registered (size=0x00080000 base_addr=0xee080002)
setup_vga_pt: vga bios checksum is adjusted!
gfx_claim_vga_cycle: bridge for bus 7, previous bridge control is 4
gfx_claim_vga_cycle: bus=0x0, dev=0x1e, func=0x0
gfx_claim_vga_cycle: bridge for bus 7, updated bridge control is 4
gfx_claim_vga_cycle: bridge for bus 1, previous bridge control is 18
gfx_claim_vga_cycle: bus=0x0, dev=0x1, func=0x0
gfx_claim_vga_cycle: bridge for bus 1, updated bridge control is 18
gfx_claim_vga_cycle: previous igd control is 32
gfx_claim_vga_cycle: updated igd control is 32
pt_msi_setup: msi mapped with pirq 37
pci_intx: intx=1
register_real_device: Real physical device 01:00.0 registered successfuly!
IRQ type = MSI-INTx
char device redirected to /dev/pts/0
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
xen be: console-0: xen be: console-0: initialise() failed
initialise() failed
pt_bar_reg_read: first read BARs of gfx
pt_bar_reg_read: first read BARs of gfx
pt_bar_reg_read: first read BARs of gfx
pt_bar_reg_read: first read BARs of gfx
pt_iomem_map: e_phys=ec000000 maddr=ec000000 type=0 len=33554432 index=0 first_map=1
pt_iomem_map: e_phys=e0000000 maddr=e0000000 type=8 len=134217728 index=1 first_map=1
pt_iomem_map: e_phys=e8000000 maddr=e8000000 type=8 len=67108864 index=3 first_map=1
pt_ioport_map: e_phys=2000 pio_base=2000 len=128 index=5 first_map=1
pt_ioport_map: e_phys=c200 pio_base=2000 len=128 index=5 first_map=0
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state.
platform_fixed_ioport: changed ro/rw state of ROM memory area. now is ro state.
pt_pci_read_config: Error: Failed to read register with invalid access size alignment. [00:05.0][Offset:0eh][Length:4]
pt_pci_read_config: Error: Failed to read register with invalid access size alignment. [00:05.0][Offset:0eh][Length:4]
pt_pci_read_config: Error: Failed to read register with invalid access size alignment. [00:05.0][Offset:0eh][Length:4]
pt_pci_read_config: Error: Failed to read register with invalid access size alignment. [00:05.0][Offset:0eh][Length:4]
pt_pci_read_config: Error: Failed to read register with invalid access size alignment. [00:05.0][Offset:0eh][Length:4]
pt_pci_read_config: Error: Failed to read register with invalid access size alignment. [00:05.0][Offset:0eh][Length:4]
pt_pci_read_config: Error: Failed to read register with invalid access size alignment. [00:05.0][Offset:0eh][Length:4]

pt_bar_reg_read: first read BARs of gfx
pt_bar_reg_read: first read BARs of gfx
pt_iomem_map: e_phys=ffffffff maddr=ec000000 type=0 len=33554432 index=0 first_map=0
pt_iomem_map: e_phys=ffffffff maddr=e0000000 type=8 len=134217728 index=1 first_map=0
pt_iomem_map: e_phys=ffffffff maddr=e8000000 type=8 len=67108864 index=3 first_map=0
pt_ioport_map: e_phys=ffff pio_base=2000 len=128 index=5 first_map=0
pt_iomem_map: e_phys=ec000000 maddr=ec000000 type=0 len=33554432 index=0 first_map=0
pt_iomem_map: e_phys=e0000000 maddr=e0000000 type=8 len=134217728 index=1 first_map=0
pt_iomem_map: e_phys=e8000000 maddr=e8000000 type=8 len=67108864 index=3 first_map=0
pt_ioport_map: e_phys=c200 pio_base=2000 len=128 index=5 first_map=0


How can i fix this?
Is the patch hvmloader already applied in xen-unstable4.2?
Is there any difference between using pci-stub instead of pciback?

With these 2 patches applied I cannot run the virtual machine neither with gfx_passthru = 0, but without those patches i was able to do it and was able to see the Nvidia Card inside Windows but with error code43 and a yellow exclamation symbol...

Any help would be welcome, thanks.


JavMV
Reply | Threaded
Open this post in threaded view
|

Re: VGA passthrough on unstable

Konrad Rzeszutek Wilk
> How can i fix this?
> Is the patch hvmloader already applied in xen-unstable4.2?
> Is there any difference between using pci-stub instead of pciback?

Not for HVM guests.
>
> With these 2 patches applied I cannot run the virtual machine neither with
> gfx_passthru = 0, but without those patches i was able to do it and was able
> to see the Nvidia Card inside Windows but with error code43 and a yellow
> exclamation symbol...

Hmm, Does the error code show up anything on Google?

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel