Arm developer build guide for Ampere Altra

General Discussion
1

Here is guidance for building arm developer workstations, homelab servers, NAS, et al. using ASRock Rack ALTRAD8UD-1L2T Ampere Altra. We worked with ASRock Rack and Newegg to create these nice priced bundles for you. Patrick Kennedy says “This is the Ampere Arm motherboard you want.” and loves Noctua Ampere Altra coolers.

See what others have built and tell us about your build here and watch replay of our June 5 2024 webinar for prizes.

For buy instead of build ADLINK Ampere Altra Dev Platform, MIFCOM Ampere Altra Workstation, and many servers are click-to-buy.

Components
ASRock Rack Ampere Altra Memory QVL (qualified vendor list) are tested by ASRock Rack using ALTRAD8UD-1L2T.

Ampere Altra AVL is list of components (memory, storage, network cards, HBA controllers, et al.) that have been Ampere tested and supported on “Mt Jade” Ampere Altra reference platform. So these probably work fine on ALTRAD8UD-1L2T but keep your receipts.
@bexcran has thoughts here and itemized an example Ampere Altra desktop build.

Workstation and other enclosures
ASRock Rack Ampere Altra “deep mATX” should be fine with enclosures that are for “EATX/ATX/mATX”. A few mATX enclosures also work but measure carefully, consider cable routing, and keep your receipt. Pro tip: SFX PSUs are much smaller and enabled the board to fit in some mATX enclosures.

The ASRock Rack board is deeper than normal, 10.5” so look for something with an extra inch of depth. Note that one of the many standard MicroATX mounting holes is not present on this board so take care to not have that particular extra standoff damage the board, remove that standoff or put a protective pad on it.

CPU Coolers
This is a list of coolers (active, passive, water) designed for and tested on ASRock Rack Ampere “deep microATX” motherboard which is click-to-buy on Newegg. All of these are also believed to work on the ADLINK COM-HPC Altra module, Ampere Altra Dev Kit, Ampere Altra Dev Platform workstation.

Operating Systems

  • All of the Linux distros that come to mind and a few that don’t.
  • FreeBSD not yet on ASRock Rack because of some ACPI issue. We’re looking into this and you can help! FreeBSD is fine on the ADLINK Ampere Altra products.
  • Windows 11 24H2, Windows Server 2025 Insider Preview install and work fine. Main issue is lack of drivers but people are working on that. Not yet officially supported though Azure and their customers run huge Windows workloads in production on Azure Arm Ampere Altra instances all the live long day. And is supported with ASPEED Windows Display Driver 1.15.2 and later download here. For more details "Fastest Windows Arm PC" Ampere guide

Display and GPU Support

Faster Boot
As-is from the factory this board takes a long time to boot like 5-6 minutes so be patient. These changes will make it boot way faster:

  • -2 minutes: Use the BMC bypass jumper (mislabeled in the user manual as “BMC disable”) which makes it proceed with system booting without waiting for BMC to finish booting The “BMC bypass” jumper lets you apply power to the host. It won’t even power on until the BMC has booted if it’s not configured. Refer to the ASRock Rack Ampere user manual here: https://www.asrockrack.com/general/productdetail.asp?Model=ALTRAD8UD-1L2T#Manual To check if your ASRock Rack “BMC_DIS” jumper has been set to bypass waiting for BMC: Unplug the AC cord. Wait a few seconds. Plug it back in. See if pressing the power button straight away works to turn on the host. If it does then the BMC Bypass jumper is set. If you have to wait 1-3 minutes until the power button does anything, then it means the BMC bypass jumper is not set (which is how they come from the factory unless we convince ASRock Rack to change this). So says Rebecca Cran our firmware engineer who knows more about the ASRock Rack Ampere boards & firmware than anyone.
  • -1 minute: For Ubuntu disable DHCP on all but the network port you’re actually using Speeding up Ubuntu boot on Ampere-based Systems | Wiki.js. ASRock Rack board has many ethernet ports so the delays add up.
  • -1 minute: Disabling the network stack in the UEFI Setup which disables network boot (PXE/HTTP) and disables Redfish support during the UEFI phase. ASRock Rack ALTRAD8UD | Wiki.js

Tips and Tricks

IMG_0703
IMG_07031920×2560 364 KB

2 bonus points for @cypou, 1 per cat. Those are the rules.

10 Likes
3

Just to note that the NVIDIA T400 is based on the Turing architecture and so is supported by the NVIDIA Open GPU drivers.

There’s more technical information about the card at NVIDIA T400.

2 Likes
4

@JoeSpeed I wrote an article about my build (with the BOM, photos, etc.):

:arrow_right: Homelab: 2U server, short depth, with front IO, based on Ampere Altra and Asrock Rack ALTRAD8UD-1L2T motherboard

3 Likes
5

So something I discovered today, my RX 6800 from PowerColor has both x86 and aarch64 GOP drivers in the card. Still couldn’t get the card to fully power up and display the firmware. However, the fans do spin up for a moment and a display does activate but the card shuts down after a couple seconds. Maybe ASRock’s UEFI firmware for their board ignores GOP driver? Or it could be I need to enable or change a firmware option?

6

This is wonderful Nicolas ! I love the level of detail. Bravo - et merci!

I also like the surprised sounding “it looks quite professional!”

Dave

2 Likes
8

FYI, I just popped an AMD Radeon Pro W7700 in a Thelio Astra this week. It booted right up, zero issues, with the amdgpu driver in Ubuntu 24.04…

One wrinkle is this system’s OS was prepared by System76, so I can’t guarantee it didn’t have the patch pre-applied. But I did get some artifacting on some system text (e.g. Chrome tabs), but not on most.

4 Likes
9

I’m glad you got one to review; and it sounds promising that it worked, with minor issues, out of the box.

3 Likes
10

Has anyone tested any high density RAM module with the ALTRAD8UD-1L2T? I saw on the the Ampere Altra Family Approved Vendor List (AVL) – March 2024 a 256GB module from SK Hynix listed (HMAT14JXSLB189N). On the ALTRAD8UD-1L2T I saw a 128GB module from SMART (ST1637RD440465-SM).

Has anyone tried any of these, or any other of such density? Where are they available for sale?

3 Likes
11

Hey all, I just got a Mt. Collins 2U server. I also have 128gb/256gb LRDIMMs… but my system is running an older BIOS/BMC which I don’t think supports them. I’d be happy to test these larger dimms sizes in it, but I’d need to get these updated BIOS/firmware which seems to be a pain to get through Ampere directly (they keep telling me to email different teams and none have allowed me access yet). If anyone can help me get those I’d be more than happy to test larger dimm sizes.

1 Like
12

@bexcran @ppouliot Do either of you know how to help @zivester here? Zach, my understanding is that the “ask” you have is access to firmware updates so that you can try the LRDIMMs in a Mt. Collins - is that right?

1 Like
13

Correct. I have ~2021 Firmware/BIOS on these but I see a lot of weird errors booting Ubuntu and I know its definitely out of date. I have a couple 256gb LRDIMMs and 128gb LRDIMMs that I can test.

14

Unfortunately accessing updated BIOS and BMC firmware for the reference platforms (Mt Jade, Mt Collins, etc.) requires having a signed NDA with Ampere in order to login to Customer Connect - unless the company the systems were purchased from provides firmware and documentation via their site. And similar to many other companies, you generally have to have an existing relationship with Ampere to be allowed to sign the NDA (i.e. individuals not associated with a company are unlikely to be given access).

I don’t know about 128GB and 256GB DIMM support, but unless DDR4 and DDR5 are very different in terms of support, I suspect 256GB won’t be supported even in the latest SYS/TF-A binaries for Altra.

1 Like
15

I’ve just confirmed that DDR4 and DDR5 are very different in terms of support, and the Ampere Altra AVL (Ampere Altra family Device Documentation) does list a few 128GB DIMMs and one 256GB one - so you can certainly try the DIMMs you have and see if they work even if they’re not listed on the AVL.

3 Likes
16

I agree with Dave, that’s a very nice write up. I wish commercial products had that level of detail and great photos showing details. Thanks!
John

3 Likes
17

I must be on an Ampere blacklist because they won’t respond to my messages anymore about a trying to sign that NDA, sigh.

Anyway, I have another system in house a Gigabyte R152-P30. It supports 256gb and 128gb dimms on the latest BIOS. I only have 2x256gb 3200aa and 8x 128gb 2666v modules but I’m pretty sure the system would work just fine with the full 4TB it says it supports. Thanks Gigabyte for making firmware/bios publicly available!

128gb_dimms
128gb_dimms796×595 59.5 KB

256gb_dimms
256gb_dimms798×595 57.6 KB

1 Like
18

Hi @zivester , no blacklist - the issue is that the customer support section is for Ampere’s customers, the OEMs and ODMs who build systems - in principle, your first port of call for firmware updates should be the supplier you bought the system from (in this case, Gigabyte, if I understand correctly).

Dave.

1 Like
19

Ampere systems are hitting second hand markets (that’s where I purchased mine). The Mt. Collins systems are Ampere controlled AFAIK (w/ Foxconn as the board manufacturer). The person I purchased it from doesn’t have a contract w/ Ampere and thus cannot supply me any updates. If Ampere just shared the files on their website I’d just help myself and not bother them. That’s how a lot of the board suppliers work e.g. SuperMicro, Gigabyte, Asrock, etc. For a company trying to gain adoption it seems very anti consumer. I’m trying to explore Arm in the server environment, so it’s just pretty annoying to acquire something I can’t get access to files that already exist but are under lock and key. /rant

3 Likes
20

Thanks @zivester - I will forward your message to our customer support org - again, this is complicated for us primarily because we do not build the systems. As you say, Mt. Collins boards were built to our specs by Foxconn, so normally Foxconn is where you would go to get firmware updates. I realize that this is frustrating for you - I will see what our customer service folks respond to your message.

Thanks,
Dave.

2 Likes
21

Bit of a noob question. But I’m confused over the motherboard docs:

This motherboard provides three 8-pin ATX 12V power connectors. Under the lowest power supply, user can freely use ATX12V1, ATX12V2 or ATXV3 connector.

Does this mean i can get away with just connecting one ATX cable to the Motherboard for power? Or will I need more if I have a power hungry chip. (I have the M128-30. So that’s 250W). The motherboard docs aren’t very enlightening. What is “under the lowest power supply” ?

1 Like
22

With your CPU wattage, I would consider connecting to all three 12v connectors. Especially if you have 8 DIMMs, and several PCI cards and nvme’s. If you had a lower TDP CPU and minimal other cards, could get away with fewer 12v supplies… that’s what I think they are getting at.

1 Like