# Dogfooding Ampere (or Arm) **URL:** \ **Category:** General Discussion\ **Created:** [January 4, 2023, 3:26pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90 "2023-01-04T15:26:39Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dgilmore](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@dgilmore](https://community.amperecomputing.com/u/dgilmore)\ **Post date:** [January 4, 2023, 3:26pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/1 "2023-01-04T15:26:39Z")
I got myself one of the adlinktech 64 core Ampere-based boxes. I am dogfooding it as my primary desktop despite some niggling issues. Some of it is probably self-induced, as ADLINK only recommends using NVidia cards. I have been a Radeon user for years as for my needs, things just work. I have installed [XFX Speedster QICK 210 AMD Radeon™ RX 6500 XT Core Gaming Graphics Card with 4GB GDDR6, AMD RDNA™ 2](https://www.xfxforce.com/shop/xfx-speedster-qick-210-amd-radeon-tm-rx-6500-black) in mine,it is great in that it ships with an AArch64 optionrom so everything works in the firmware, and all necessary support in the amdgpu driver has landed in the 6.2 merge window. on module loading, I get the following traceback 32 times. ```auto [25.945356] WARNING: CPU: 0 PID: 18 at arch/arm64/mm/mmu.c:1156 vmemmap_populate+0x20/0x34 [25.953609] Modules linked in: amdgpu(+) bridge stp llc raid1 drm_ttm_helper ttm video gpu_sched crct10dif_ce drm_buddy polyval_ce nvme polyval_generic drm_display_helper ghash_ce ixgbe sbsa_gwdt nvme_core cec igb nvme_common onboard_usb_hub ast mdio xgene_hwmon gpio_dwapb scsi_dh_rdac scsi_dh_emc scsi_dh_alua ip6_tables ip_tables dm_multipath i2c_dev fuse [25.985151] CPU: 0 PID: 18 Comm: kworker/0:1 Tainted: G W ------- --- 6.2.0-0.rc2.20230103git69b41ac87e4a.19.fc37.aarch64 #1 [25.997829] Hardware name: ADLINK AVA Developer Platform/AVA Developer Platform, BIOS TianoCore 2.04.100.07 (SYS: 2.06.20220308) 09/08/2022 [26.010333] Workqueue: events work_for_cpu_fn [26.014679] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [26.021628] pc : vmemmap_populate+0x20/0x34 [26.025800] lr : __populate_section_memmap+0x1a4/0x1d8 [26.030927] sp : ffff800008423900 [26.034229] x29: ffff800008423900 x28: ffffc14d44231000 x27: ffff07ffaf9aefe0 [26.041353] x26: ffff080f315fc000 x25: fffffc0000000000 x24: fffffffffe000000 [26.048477] x23: ffffc14d60fd7000 x22: 0000000fffff8000 x21: 0000000000000000 [26.055601] x20: 0000000fffff8000 x19: fffffffffde00000 x18: 0000000000000014 [26.062725] x17: 00000000f974f0fe x16: ffffc14d605216e4 x15: ffffc14d43ab1b00 [26.069849] x14: ffffc14d43ab9fe8 x13: ffffc14d5f7d87cc x12: ffffc14d5f7d4784 [26.076973] x11: ffffc14d5f7272d4 x10: ffffc14d5f7e20fc x9 : ffffc14d607a89dc [26.084097] x8 : ffff080f315fc028 x7 : 0000000000000000 x6 : 0000000000000000 [26.091221] x5 : 0000000000000001 x4 : fffffe0000000000 x3 : 0000000000000000 [26.098344] x2 : 0000000000000000 x1 : fffffffffe000000 x0 : fffffffffde00000 [26.105468] Call trace: [26.107902] vmemmap_populate+0x20/0x34 [26.111726] __populate_section_memmap+0x1a4/0x1d8 [26.116506] sparse_add_section+0x138/0x1f4 [26.120678] __add_pages+0xd8/0x180 [26.124155] pagemap_range+0x324/0x41c [26.127893] memremap_pages+0x184/0x2b4 [26.131717] devm_memremap_pages+0x30/0x7c [26.135802] svm_migrate_init+0xd8/0x18c [amdgpu] [26.140993] kgd2kfd_device_init+0x39c/0x5e0 [amdgpu] [26.146525] amdgpu_amdkfd_device_init+0x13c/0x1d4 [amdgpu] [26.152576] amdgpu_device_ip_init+0x53c/0x588 [amdgpu] [26.158276] amdgpu_device_init+0x828/0xc60 [amdgpu] [26.163714] amdgpu_driver_load_kms+0x28/0x1a0 [amdgpu] [26.169412] amdgpu_pci_probe+0x1b0/0x420 [amdgpu] [26.174675] local_pci_probe+0x48/0xa0 [26.178416] work_for_cpu_fn+0x24/0x40 [26.178418] process_one_work+0x1ec/0x470 [26.178420] worker_thread+0x200/0x410 [26.178422] kthread+0xec/0x100 [26.178424] ret_from_fork+0x10/0x20 [26.178427] ---[end trace 0000000000000000]--- ``` This seems to be an issue with the system firmware, and I have an issue open with ADLINK. One area that needs improvement for better growth is for developers to have Arm based machines on their desks that they can touch and feel. This seems like an okay, if a bit expensive, start, though it is a rough experience. I get more crashes in the video stack than I should. I am curious about what kinds of dogfooding others are doing. ---
**Author:** ![dneary](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@dneary](https://community.amperecomputing.com/u/dneary)\ **Post date:** [January 5, 2023, 9:15pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/2 "2023-01-05T21:15:04Z")
Hi Dennis, apparently we have some known issues with AMD GPUs on Altra - I’ll share more details when I get them. Sorry you’re having some issues! Dave. ---
**Author:** ![dneary](https://avatars.discourse-cdn.com/v4/letter/d/47e85d/32.png) [@dneary](https://community.amperecomputing.com/u/dneary)\ **Post date:** [January 9, 2023, 9:42pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/3 "2023-01-09T21:42:05Z")
@dgilmore I checked with our product folks - and while we validate cards from lots of vendors, since this is an ADLink platform, whether or not they support particular GPUs is really up to them. Have you asked them whether/which AMD GPUs they support on the platform? Thanks! Dave ---
**Author:** ![dgilmore](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@dgilmore](https://community.amperecomputing.com/u/dgilmore)\ **Post date:** [January 9, 2023, 9:59pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/4 "2023-01-09T21:59:01Z")
@dneary ADLINK tell me no AMD GPU’s work and no drivers exist. They do claim to be working with Ampere on it. ---
**Author:** ![vikingforties](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/vikingforties/32/1192_2.png) [@vikingforties](https://community.amperecomputing.com/u/vikingforties)\ **Post date:** [February 28, 2023, 11:26am UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/5 "2023-02-28T11:26:31Z")
I have a couple of intentions along these lines. I’ll need to get hold of something quiet like the ADLINK AVA. As a daily driver at Ampere the big choice would be run Windows 11: > **[Ampere ALTRA - Industry leading ARM64 Server - Cloudbase Solutions](https://cloudbase.it/ampere-altra-industry-leading-arm64-server/)** > > AmpereComputing were kind enough to send us at our Cloudbase Solutions office the new version of their top-tier server product, which incorporates their latest Ampere ALTRA ARM64 Processor. The server version has a beefy dual-socket AMPERE ALTRA... and have happy apps or run the few Windows apps I need on Linux via browser (like Office 365) and WINE. I also do a bit of Folding@Home so it’ll be interesting to see how this performs with a big silent NVIDIA card and the some of the cores given over to GROMACS workloads. ---
**Author:** ![bexcran](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/bexcran/32/188_2.png) [@bexcran](https://community.amperecomputing.com/u/bexcran)\ **Post date:** [November 8, 2024, 11:21pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/6 "2024-11-08T23:21:53Z")
This problem still exists in Linux kernel 6.11.6 with an AMD Radeon RX 7900 XTX. ---
**Author:** ![bexcran](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/bexcran/32/188_2.png) [@bexcran](https://community.amperecomputing.com/u/bexcran)\ **Post date:** [November 9, 2024, 3:13am UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/7 "2024-11-09T03:13:23Z")
I built a kernel 6.12.0-rc6 and that crash went away (i.e. it’s been fixed). But there was a new problem with queues on the card getting full causing it to stall. Turns out it’s related to PSR, and disabling that allowed the card to start working. As the last comment on the Fedora thread says, add the following to the kernel parameter command line in GRUB: ```bash amdgpu.dcdebugmask=0x210 ``` > **[Constant freezes after suspend on Fedora 40 with AMD GPU (RX 6700 XT)](https://discussion.fedoraproject.org/t/constant-freezes-after-suspend-on-fedora-40-with-amd-gpu-rx-6700-xt/131956/2)** > > There have been reports of crashes after suspend because of gnome extensions. GNOME Shell crashes can be connected to the same issue. Do you have any extensions enabled? If so, you could try disabling them and performing some tests. Afterwards... ---
**Author:** ![GloriousEggroll](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/gloriouseggroll/32/757_2.png) [@GloriousEggroll](https://community.amperecomputing.com/u/GloriousEggroll)\ **Post date:** [November 10, 2024, 6:22am UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/8 "2024-11-10T06:22:28Z")
Huge thanks to @bexcran on the help with this over twitter. This commit backported to 6.11 allows amdgpu to work again and resolves OPs kernel error: > > > The memory hot-plug and resource management code needs to know the > largest addre…ss which can fit in the linear map, so set PHYSMEM\_END for > that purpose. > > This fixes a crash at boot when amdgpu tries to create > DEVICE\_PRIVATE\_MEMORY and is given a physical address by the resource > management code which is outside the range which can have a \`struct > page\` > > | Unable to handle kernel paging request at virtual address 000001ffa6000034 > | user pgtable: 4k pages, 48-bit VAs, pgdp=000008000287c000 > | \[000001ffa6000034\] pgd=0000000000000000, p4d=0000000000000000 > | Call trace: > | \_\_init\_zone\_device\_page.constprop.0+0x2c/0xa8 > | memmap\_init\_zone\_device+0xf0/0x210 > | pagemap\_range+0x1e0/0x410 > | memremap\_pages+0x18c/0x2e0 > | devm\_memremap\_pages+0x30/0x90 > | kgd2kfd\_init\_zone\_device+0xf0/0x200 \[amdgpu\] > | amdgpu\_device\_ip\_init+0x674/0x888 \[amdgpu\] > | amdgpu\_device\_init+0x7a4/0xea0 \[amdgpu\] > | amdgpu\_driver\_load\_kms+0x28/0x1c0 \[amdgpu\] > | amdgpu\_pci\_probe+0x1a0/0x560 \[amdgpu\] > > Signed-off-by: D Scott Phillips \ > Link: https://lore.kernel.org/r/20240903164532.3874988-1-scott@os.amperecomputing.com > Signed-off-by: Will Deacon \ ---
**Author:** ![GloriousEggroll](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/gloriouseggroll/32/757_2.png) [@GloriousEggroll](https://community.amperecomputing.com/u/GloriousEggroll)\ **Post date:** [November 10, 2024, 6:23am UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/9 "2024-11-10T06:23:05Z")
The original 2 patches from here are still needed as well: > **[linux-ampere-altra-erratum-pcie-65/v6.7 at main ·...](https://github.com/AmpereComputing/linux-ampere-altra-erratum-pcie-65/tree/main/v6.7)** > > Kernel patches with workaround for Altra erratum PCIE\_65 - AmpereComputing/linux-ampere-altra-erratum-pcie-65 for the PSR issue for now bex has found kernel option `amdgpu.dcdebugmask=0x210` to work, although these pending patches may fix that (untested): [https://lore.kernel.org/dri-devel/20240708202907.383917-1-hamza.mahfooz@amd.com/](https://lore.kernel.org/dri-devel/20240708202907.383917-1-hamza.mahfooz@amd.com/) I’ll report back once I’ve tested those^ ---
**Author:** ![GloriousEggroll](https://yyz1.discourse-cdn.com/flex027/user_avatar/community.amperecomputing.com/gloriouseggroll/32/757_2.png) [@GloriousEggroll](https://community.amperecomputing.com/u/GloriousEggroll)\ **Post date:** [November 12, 2024, 6:29pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/10 "2024-11-12T18:29:18Z")
well, suspend is broken on 6.11 on ampere so i cant really test those last patches atm, but i can confirm amdgpu works fine with the PHYSMEM\_END patch. ---
**Author:** ![dgilmore](https://avatars.discourse-cdn.com/v4/letter/d/da6949/32.png) [@dgilmore](https://community.amperecomputing.com/u/dgilmore)\ **Post date:** [November 19, 2024, 5:45pm UTC](https://community.amperecomputing.com/t/dogfooding-ampere-or-arm/90/11 "2024-11-19T17:45:00Z")
I applied the PHYSMEM\_END patch to 6.11 and found amdgpu to not be functional. 6.12 I found did not work with amdgpu or nouveau with my radeon or Nvidia cards. there was a lot of PCIe noise in 6.12. I am currently using adlinks bios and bmc.