# GPU support for Ampere Altra?

**URL:** https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274
**Category:** General Discussion
**Created:** [April 18, 2023, 3:38am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274 "2023-04-18T03:38:16Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![geerlingguy](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/geerlingguy/32/62_2.png) [@geerlingguy](https://community.amperecomputing.com/u/geerlingguy)
#### Post date: [April 18, 2023, 3:38am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/1 "2023-04-18T03:38:16Z")

</div>

…specifically for the Ampere Altra Development Platform or Dev Kit.

Right now the [list of supported graphics cards](https://www.ipi.wiki/pages/comhpc-docs?page=NVIDIA_cards.html) for the Dev Kit / Dev Platform is a bit slim:

- GeForce® RTX™ 3070 Ti VENTUS 3X 8G OC
- PNY GeForce® RTX™ 3060 Ti 8GB
- GeForce® GTX 1050 2G OC
- ZOTAC GeForce® GTX 1060 3GB AMP Core Edition

I have a number of other cards I’d like to test out, but I was wondering if anyone else has tested any and found them to work (either Nvidia or AMD).

I just plugged in my RTX 8000 and tried the latest Nvidia Professional driver for aarch64, version `525.105.17`, but when it tries initializing the GPU, dmesg outputs:

```auto
[477.995897] NVRM: GPU 000d:01:00.0: RmInitAdapter failed! (0x24:0x65:1423)
[477.995979] NVRM: GPU 000d:01:00.0: rm_init_adapter failed, device minor number 0

```

I am able to get video output over DisplayPort, and Ubuntu 22.04 sees the card just fine, but it doesn’t seem to be able to use any card features, and `nvidia-smi` returns `No devices were found`.

I know cloud providers are pairing up Altra Max with cards like the A100, so there must be known-good combinations. Do I have to drop back to Ubuntu 20.04 to make things work?

I’ve been documenting my adventure in [this issue](https://github.com/geerlingguy/sbc-reviews/issues/19#issuecomment-1512386215).

---

<div class="post-metadata">

### Author: ![Manawyrm](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@Manawyrm](https://community.amperecomputing.com/u/Manawyrm)
#### Post date: [April 18, 2023, 9:54am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/2 "2023-04-18T09:54:32Z")

</div>

Hi Jeff!

from what I can tell, Ampere has some of the same memory mapping / caching issues (related to write combining, etc.) again, which you’re already very familiar with 😃

The good news is, there are some hacks/workarounds:

> <https://github.com/Tencent/TencentOS-kernel/commit/f454797b673c06c0eb1b77be20d8a475ad2fbf6f>
>
> Altra's PCIe controller may generate incorrect addresses when receiving
> writes f…rom the CPU with a discontiguous set of byte enables. Attempt to
> work around this by handing out Device-nGnRE maps instead of Normal
> Non-cacheable maps for PCIe memory areas.
> 
> Signed-off-by: D Scott Phillips \<scott@os.amperecomputing.com\>

> <https://github.com/Tencent/TencentOS-kernel/commit/adb335972fcb7a6b59bb8034498b1ffddfb37c97>
>
> A later patch will hand out Device memory in some cases to code
> which expects a …Normal memory type, as an errata workaround.
> Unaligned accesses to Device memory will fault though, so here we
> add a fixup handler to emulate faulting accesses, at a performance
> penalty.
> 
> Many of the instructions in the Loads and Stores group are supported,
> but these groups are not handled here:
> 
> \* Advanced SIMD load/store multiple structures
> \* Advanced SIMD load/store multiple structures (post-indexed)
> \* Advanced SIMD load/store single structure
> \* Advanced SIMD load/store single structure (post-indexed)
> \* Load/store memory tags
> \* Load/store exclusive
> \* LDAPR/STLR (unscaled immediate)
> \* Load register (literal) \[cannot Alignment fault\]
> \* Load/store register (unprivileged)
> \* Atomic memory operations
> \* Load/store register (pac)
> 
> Instruction implementations are translated from the Exploration tools'
> ASL specifications.
> 
> Signed-off-by: D Scott Phillips \<scott@os.amperecomputing.com\>

Those won’t be good for performance at all, but it could (should?) work.

What kernel version are you using? You might even want to just compile yourself a linux/master kernel.

There was some work on AMD GPU support on Arm64 in the recent 6.2 kernel release, so you might even give your AMD GPU a shot with a very recent kernel.

Unfortunately, I don’t have a workstation Ampere yet, only servers in the datacenter, so I can’t join into the fun yet 🙂

Please keep us updated 🙂

---

<div class="post-metadata">

### Author: ![q66](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/q66/32/780_2.png) [@q66](https://community.amperecomputing.com/u/q66)
#### Post date: [May 27, 2023, 2:59am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/3 "2023-05-27T02:59:56Z")

</div>

some good news: [AMD GPUs on the Altra devkit and other Altras - patches available now](https://community.amperecomputing.com/t/amd-gpus-on-the-altra-devkit-and-other-altras-patches-available-now/336)

---

<div class="post-metadata">

### Author: ![shadethegrey](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/shadethegrey/32/479_2.png) [@shadethegrey](https://community.amperecomputing.com/u/shadethegrey)
#### Post date: [March 9, 2024, 6:29am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/4 "2024-03-09T06:29:08Z")

</div>

I’m running an Nvidia T600 on the asrock rack motherboard in Fedora 39 with nvidia drivers.

---

<div class="post-metadata">

### Author: ![JoeSpeed](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/joespeed/32/16_2.png) [@JoeSpeed](https://community.amperecomputing.com/u/JoeSpeed)
#### Post date: [March 10, 2024, 12:25am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/5 "2024-03-10T00:25:51Z")

</div>

Nice! So now the ones I know of being used with Ampere are NVIDIA **T600** , RTX A4500, A6000, 4060, 4070, 4080, 4000 Ada SFF, 3070 Ti, 3060 GTX 1050, A100, A100X, A10, A10G, A16, A30. I think also L4, L40.

---

<div class="post-metadata">

### Author: ![anernest](https://avatars.discourse-cdn.com/v4/letter/a/7cd45c/32.png) [@anernest](https://community.amperecomputing.com/u/anernest)
#### Post date: [March 21, 2024, 8:01pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/6 "2024-03-21T20:01:20Z")

</div>

I just picked up an ASRock Rack ALTRAD8UD-1L2T and plugged an RTX 4090 into it with no luck yet. It goes into a bootloop prior to loading the OS. ASRock Rack Tech Support thinks it might be some BIOS settings for PCIe. I’ll keep trying when I get some down time.

---

<div class="post-metadata">

### Author: ![pimzand](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/pimzand/32/463_2.png) [@pimzand](https://community.amperecomputing.com/u/pimzand)
#### Post date: [March 31, 2024, 2:46pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/7 "2024-03-31T14:46:28Z")

</div>

The list of supported graphics cards are all Nvidia.  
Does this imply Nvidia closed source drivers are required?  
Or would the open source “nouveau” driver work?

---

<div class="post-metadata">

### Author: ![geerlingguy](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/geerlingguy/32/62_2.png) [@geerlingguy](https://community.amperecomputing.com/u/geerlingguy)
#### Post date: [April 1, 2024, 10:21pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/8 "2024-04-01T22:21:28Z")

</div>

4070 Ti here, haven’t tried a non-Ti version.

---

<div class="post-metadata">

### Author: ![dgilmore](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@dgilmore](https://community.amperecomputing.com/u/dgilmore)
#### Post date: [April 1, 2024, 11:50pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/9 "2024-04-01T23:50:17Z")

</div>

I am using a GeForce RTX 3060 with nouveau drivers

---

<div class="post-metadata">

### Author: ![TheComputerGuy](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/thecomputerguy/32/509_2.png) [@TheComputerGuy](https://community.amperecomputing.com/u/TheComputerGuy)
#### Post date: [April 13, 2024, 4:00am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/10 "2024-04-13T04:00:27Z")

</div>

I’m curious if the RX 6800 works with the amdgpu driver in Linux.

---

<div class="post-metadata">

### Author: ![pimzand](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/pimzand/32/463_2.png) [@pimzand](https://community.amperecomputing.com/u/pimzand)
#### Post date: [June 28, 2024, 10:12am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/11 "2024-06-28T10:12:27Z")

</div>

Hi all,

I’m really having a hard time to get _any_ GPU properly working on my ASRockRack board using Fedora 40.

@bexcran, @Civiloid , @shadethegrey , you have had more success. Any idea?  
What really stomps me is the “no signal” part. I have used three different cables, mini-DP to mini-DP, mini-DP to DP and mini-DP to HDMI.

I was really reluctant to get an NVidia T1000, because I want to use manufacturer supported open source drivers, but even that did not work.

| card | architecture | driver | kernel | pcie patches applied | pre-boot graphics using GOP | result |
| --- | --- | --- | --- | --- | --- | --- |
| NVidia GF 610 | Curie | nouveau | 6.9.4-200.fc40 | no | no | driver does not load, firmware missing |
| AMD Radeon WX3200 Pro | Polaris | amdgpu | 6.9.4-200.fc40 | no | manual from UEFI shell only | garbled screen, kernel errors |
| AMD Radeon WX3200 Pro | Polaris | amdgpu | 6.9.0 vanilla | yes | manual from UEFI shell only | works, but really sluggish, kernel errors |
| Sapphire Pulse Radeon RX 6400 | Navi 2 | amdgpu | 6.9.4-200.fc40 | no | yes, starting in grub | driver crashes |
| Sapphire Pulse Radeon RX 6400 | Navi 2 | amdgpu | 6.9.0 vanilla | yes | yes, starting in grub | driver crashes |
| Sparkle Intel Arc A310 Eco | DG2 | xe | 6.9.0 vanilla | yes | no | driver loads OK but no signal on monitor |
| Sparkle Intel Arc A310 Eco | DG2 | xe from intel drm-next | 6.10.0.-rc3 | yes | no | driver loads OK but no signal on monitor |
| NVidia T1000 | Turing | nouveau | 6.9.5-200.fc40 | no | no | driver loads OK but no signal on monitor |
| NVidia T1000 | Turing | nvidia from rpmfusion | 6.9.5-200.fc40 | no | no | akmod fails to build driver |

---

<div class="post-metadata">

### Author: ![sevo](https://avatars.discourse-cdn.com/v4/letter/s/ed655f/32.png) [@sevo](https://community.amperecomputing.com/u/sevo)
#### Post date: [June 28, 2024, 3:58pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/12 "2024-06-28T15:58:45Z")

</div>

I am using AlmaLinux 9.4 with an RTX 3060. I started with Fedora 40 but the newer gcc and kernel 6.9 were leading to compile errors for the nvidia 550 akmod from rpmfusion. I switched to AlmaLinux and the nvidia 550 drivers compile successfully (older kernel 5.14 and gcc is version 11). I started with the nvidia driver installer from their website but I have switched, not to rpmfusion, but to the nvidia driver repos from [https://negativo17.org/nvidia-driver/](https://negativo17.org/nvidia-driver/).

I haven’t circled back to Fedora 40 yet, but I would suggest trying the negativo17 repo for Fedora 40. They use akmod for Fedora 40 just like rpmfusion, but their Fedora 40 repo uses the 555 version of the nvidia driver, which I assume compiles successfully with the newer kernel and gcc 14 in Fedora 40. Their repo also will enable two important nvidia\_drm module options (via adding a file in /etc/modprobe.d) to get you a text console on your nvidia card, namely modeset=1 and fbdev=1. Without those two options, X will use your nvidia card but you won’t have text tty console on the card, and wayland compositors likely won’t work either.

I should mention that the negativo17 nvidia repo packages do not add nvidia modules to your initramfs, so the nvidia display won’t initialize (and start giving you console messages) until after the root fs is mounted, i.e. you won’t see console output quite as early as when the modules are loaded in the initramfs. You could add the modules to the initramfs yourself if desired.

An alternative option would be to download the nvidia installer for version 555. In that case, you will need to manually enable the two nvidia\_drm module options mentioned above if you want to see any text console, as the nvidia installer doesn’t enable those options (at least it didn’t when I used it to install version 550).

Then there’s the whole arm64 option ROM issue. My RX 6800 came out of box with an arm64 option ROM, but I didn’t want to maintain the PCIe erratum kernel patch to make the amdgpu driver work, so I switched to an nvidia card. It’s disappointing to hear that even with the patch you weren’t having good success with your AMD gpu’s. Of course, nvidia has no arm64 option ROM so their can never be a UEFI GOP to display boot menus, etc. Perhaps they will release one at some point…

Good luck!

---

<div class="post-metadata">

### Author: ![shadethegrey](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/shadethegrey/32/479_2.png) [@shadethegrey](https://community.amperecomputing.com/u/shadethegrey)
#### Post date: [June 28, 2024, 6:17pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/13 "2024-06-28T18:17:49Z")

</div>

I was on Fedora 39, but I used a mixture of the Nvidia driver directly from nvidia, and the nvidia xconfig command from the ampere script for ubuntu after I installed the nvidia driver. ( sudo nvidia-xconfig -a --cool-bits=31 --allow-empty-initial-configuration) If the the nvidia driver does not complete cleanly, it won’t work. I think I had to set something in grub when it booted or I wouldn’t get any display at all, but I cannot remember what I put, I think nomodeset, but I’m not sure. I haven’t used Fedora in a while, I switched to Ubuntu for the 32-bit libraries for gaming.

---

<div class="post-metadata">

### Author: ![sevo](https://avatars.discourse-cdn.com/v4/letter/s/ed655f/32.png) [@sevo](https://community.amperecomputing.com/u/sevo)
#### Post date: [June 28, 2024, 8:42pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/14 "2024-06-28T20:42:20Z")

</div>

The specific problem with Fedora 40 is compile errors with nvidia driver 550, regardless how it is packaged. And because there is no efi gop framebuffer, you need nvidia\_drm.modeset=1 and nvidia\_drm.fbdev=1 to get a framebuffer for the linux console.

Also add console=tty1 to kernel command line

---

<div class="post-metadata">

### Author: ![pimzand](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/pimzand/32/463_2.png) [@pimzand](https://community.amperecomputing.com/u/pimzand)
#### Post date: [June 28, 2024, 9:27pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/15 "2024-06-28T21:27:08Z")

</div>

Thanks for pointing out [Nvidia driver, CUDA tools and libraries – negativo17.org](https://negativo17.org/nvidia-driver/) . Indeed it installs and builds fine. I might have achieved the same results by switching to rpmfusion rawhide. I can’t try the effect right now, I am accessing my server remotely now and there is no gpu card in it now.

The modeset instructions might help. Perhaps for nouveau too. The reason that AMD cards gave me a picture at all may be caused by the fact that there is a GOP. For Intel and NVidia there is none, which makes my monitor go to deep sleep immediately, or switch to another active port. This might cause the driver to not detect the monitor.

---

<div class="post-metadata">

### Author: ![sevo](https://avatars.discourse-cdn.com/v4/letter/s/ed655f/32.png) [@sevo](https://community.amperecomputing.com/u/sevo)
#### Post date: [June 28, 2024, 9:58pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/16 "2024-06-28T21:58:34Z")

</div>

The initial framebuffer the linux kernel uses for the console with these cards is typically the efi gop framebuffer. The linux device driver is efifb. In the case of these cards with no arm64 option rom, there is no UEFI GOP driver to create that framebuffer, which gets passed to the kernel and used by the efifb driver. By telling the nvidia\_drm module fbdev=1, it will create a framebuffer that the linux console can use. Not sure what, if any, equivalent there is for the intel gpus.

Of course, the cards will output video if the X server starts and is configured to use the card, even if there is no linux console framebuffer, as the X server creates its own framebuffer. But when the X server quits, the monitor will again go blank. So in your nice table above, I wonder if some of the “driver loads but monitor is blank” is because there is no linux console framebuffer and X server has not been started using that card.

---

<div class="post-metadata">

### Author: ![pimzand](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/pimzand/32/463_2.png) [@pimzand](https://community.amperecomputing.com/u/pimzand)
#### Post date: [July 1, 2024, 11:53am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/17 "2024-07-01T11:53:01Z")

</div>

Finally got a picture from the T1000 on my 4K monitor, using the negativo17 repo, using nvidia-open kernel modules. There’s an rpmfusion nvidia beta (555) repo in copr that I tried first, but that does not support aarch64.

But I did not get a picture immediately. I switched from the default Wayland to Xorg, restarted gdm, then switched back to Wayland, again restarted gdm, and then finally I got a login screen.

I have yet to see whether this is going to be a standard routine after each boot.

It surprises me that there’s nothing in dmesg or journalctl that seems to log the detection of displays.

---

<div class="post-metadata">

### Author: ![bexcran](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/bexcran/32/188_2.png) [@bexcran](https://community.amperecomputing.com/u/bexcran)
#### Post date: [July 2, 2024, 3:45am UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/18 "2024-07-02T03:45:09Z")

</div>

> Of course, nvidia has no arm64 option ROM so their can never be a UEFI GOP to display boot menus, etc.

For ADLINK systems, I’ve added an x86 emulator driver to my firmware builds ([GitHub - bcran/edk2\_aadp: EDK2 UEFI for ADLINK Systems](https://github.com/bcran/edk2_aadp)) so the x86 optrom gets loaded and you get to use the display during boot.

---

<div class="post-metadata">

### Author: ![sevo](https://avatars.discourse-cdn.com/v4/letter/s/ed655f/32.png) [@sevo](https://community.amperecomputing.com/u/sevo)
#### Post date: [July 2, 2024, 5:27pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/19 "2024-07-02T17:27:12Z")

</div>

I might have to try building an aarch64 [MultiArchUefiPkg](https://github.com/intel/MultiArchUefiPkg/tree/d235d9fec2c8ae1b02b5ba63af925efb23ad1d90) and trying it out on the Asrock Ampere board.

---

<div class="post-metadata">

### Author: ![sevo](https://avatars.discourse-cdn.com/v4/letter/s/ed655f/32.png) [@sevo](https://community.amperecomputing.com/u/sevo)
#### Post date: [July 4, 2024, 9:40pm UTC](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274/20 "2024-07-04T21:40:33Z")

</div>

The emulator driver loads on the Asrock board with latest 2.06 firmware from the uefi shell, and the device it creates shows up as expected from the documentation. However, X64 binaries (eg. X64 build of the edk2 uefi shell) fail with an “unsupported” error message when attempting to execute them. The same emulator driver and X64 binaries work as expected when tested in a qemu kvm aarch64 virtual machine. I haven’t delved into what AMI might have done to prevent this from working… I assume it is that the uefi firmware from AMI doesn’t support the EFI protocol that the emulator driver produces, EDKII\_PECOFF\_IMAGE\_EMULATOR\_PROTOCOL but I haven’t done the work to prove that yet

[Next page](https://community.amperecomputing.com/t/gpu-support-for-ampere-altra/274.md?page=2)
