[PVE-User] MxGPU with AMD S7150

[Dominik Csapak]

On 01.03.19 13:37, Mark Adams wrote:
> Hi All,
> 
> I'm trying this out, based on the wiki post and the forum posts:
> 
> https://forum.proxmox.com/threads/amd-s7150-mxgpu-with-proxmox-ve-5-x.50464/
> 
> https://pve.proxmox.com/wiki/MxGPU_with_AMD_S7150_under_Proxmox_VE_5.x
> 
> However I'm having issues getting the gim driver working. Was just
> wondering if the Proxmox staff member that tested this out came across this
> particular issue, or if anyone else had any insights.

Hi, i am the one that tested this.

> 
> My hardware is an ASRock EPYCD8-2T motherboard (SR-IOV enabled in bios) and
> an AMD S7150. Proxmox is 5.3-11.
> 
> When running the modprobe of gim, it crashes out with the following:
> 
> [Fri Mar  1 12:31:49 2019] gim info:(enable_sriov:299) Enable SRIOV
> [Fri Mar  1 12:31:49 2019] gim info:(enable_sriov:300) Enable SRIOV vfs
> count = 16
> [Fri Mar  1 12:31:49 2019] pci 0000:61:02.0: [1002:692f] type 7f class
> 0xffffff
> [Fri Mar  1 12:31:49 2019] pci 0000:61:02.0: unknown header type 7f,
> ignoring device
> [Fri Mar  1 12:31:50 2019] gim error:(enable_sriov:311) Fail to enable
> sriov, status = fffffffb
> [Fri Mar  1 12:31:50 2019] gim error:(set_new_adapter:668) Failed to
> properly enable SRIOV
> [Fri Mar  1 12:31:50 2019] gim info:(gim_probe:91) AMD GIM probe: pf_count
> = 1
> 

mhmm i cannot really remember if that exact error message occured, but
you have to enable several things in the bios

AMD-Vi/VT-d
SR-IOV
ARI
and possibly above-4g-decoding

also make sure you enable the 'legacy' or non uefi oprom for
that card

on our supermicro board we could select the oprom for each pcie port 
separately

names may be different in your bios,
or some options may not exists at all

at last, a different pcie port may be necessary, depending on how
the mainboard is wired (with epyc all pcie ports should go to the
cpu, but i do not know about your specific board)

if all else fails, i would open an issue on github for the gim project
and ask there if anything is known

hope this helps