Performance Comparisons for Reccomendation Model Architectures

General Discussion
1

I am attempting to benchmark and compare different Recommendation Model Architectures to evaluate tradeoffs of the different types for a given HPC platform.

Model Types:

  • Embedding-Dominated
  • MLP-Dominated
  • Attention-Dominated

This is based on Stanford’s MLsys Deep Recommender Systems seminar presented by Carol-Jean Wu, who is an AI researcher at Facebook.

What I am looking for is an opensource repository that enables a user to select the specific model architecture to run against a dataset and compare performance on a specific platform. Can anyone point me in the right direction?

2 Likes
2

I would recommend you to evaluate Mixture of Expert (MoE) model architecture with CPU-offloading on 64KB kernel page size.

I’m working on it now.

CleanShot 2025-08-11 at 16.11.02@2x
CleanShot 2025-08-11 at 16.11.02@2x1770×1528 413 KB

3 Likes
3

I would have thought most recommendation engines are based on fairly simple heuristics - maybe some reinforced learning & nearest-neighbor vector searches and clustering. Is that “embedding-dominated”? I’m guessing that the reinforced learning around neural nets is MLP-dominated (MLP = multi-layer perceptron, correct?).

Anyway - very interested in the topic! I will be looking at the answers you get!

4

Yes that is correct @dneary . Facebook DLRM has a diagram outlining the architecture for their opensource Recommendation Model.

Facebook DLRM Github

1 Like