Can the Apple M1 Max (64 GB) run MLExpert?
MLExpert·34.7B (3.5B active)·Apple M1 Max (64 GB)·64 GB unified memory
Yes. The Apple M1 Max (64 GB) runs MLExpert fully in unified memory at Q8_0 (about 37 GB of 48 GB), at roughly 60 tokens/s.
- best quant
- Q8_0
- needs
- 37 GB
- usable
- 48 GB
- est. speed
- ~60 tok/s
Estimates for an 8K context; offload assumes 32 GB of system RAM.Try other contexts on the MLExpert page →
Every quantisation of MLExpert on the Apple M1 Max (64 GB)
| Quant | Memory | Fit | Speed |
|---|---|---|---|
| Q2_K | 15 GB | fits | ~194 tok/s |
| Q3_K_M | 18 GB | fits | ~130 tok/s |
| Q4_K_M | 23 GB | fits | ~105 tok/s |
| Q5_K_M | 28 GB | fits | ~90 tok/s |
| Q6_K | 31 GB | fits | ~77 tok/s |
| Q8_0 pick | 37 GB | fits | ~60 tok/s |
| FP16 / BF16 | 69 GB | too big | — |
Speed is a bandwidth-based estimate for generating text (400 GB/s); prompt processing and long contexts are slower.
Measured speed on the Apple M1 Max (64 GB)
No community reports for this pair yet.
Report your own speed →Run MLExpert on the Apple M1 Max (64 GB)
Using Q8_0, the best fit for Apple M1 Max (64 GB).
ollama run hf.co/bartowski/InternScience_MLExpert-GGUF:Q8_0 llama-server -hf bartowski/InternScience_MLExpert-GGUF:Q8_0brew install llama.cpp ships with Metal. Downloads the GGUF on first run.
lms get bartowski/InternScience_MLExpert-GGUF@Q8_0Then load it from the LM Studio app or with `lms load`.
from transformers import pipeline
pipe = pipeline("text-generation", model="InternScience/MLExpert", dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Explain self-hosting in one paragraph."}]
out = pipe(messages, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])Chat models that fit the Apple M1 Max (64 GB) comfortably
All that fit →| Model | Params | Best quant | Memory | Speed |
|---|---|---|---|---|
| 36BA3B | Q8_0 | ≈38 GBFits in memory | ~69 tok/s | |
| 31.2BA4.6B | Q8_0 | ≈39 GBFits in memory | ~45 tok/s | |
| 32.8B | Q8_0 | ≈37 GBFits in memory | ~6.3 tok/s | |
| 39.8BA1.3B | Q8_0 | ≈43 GBFits in memory | ~166 tok/s | |
| 20.9BA4.2B | Q8_0 | ≈13 GBFits in memory | ~49 tok/s | |
| 30.5BA3B | Q8_0 | ≈33 GBFits in memory | ~69 tok/s |
Other GPUs that run MLExpert
Frequently asked questions
- Can the Apple M1 Max (64 GB) run MLExpert?
- Yes. The Apple M1 Max (64 GB) runs MLExpert fully in unified memory at Q8_0 (about 37 GB of 48 GB), at roughly 60 tokens/s.
- How fast does MLExpert run on the Apple M1 Max (64 GB)?
- An estimate from memory bandwidth gives about 60 tokens/s at Q8_0.
- Which quantisation of MLExpert should I use on the Apple M1 Max (64 GB)?
- Q8_0: the largest one that fits. Expected quality: lossless compared with the full-precision weights.
Memory and speed are estimates from the model's size and the card's memory bandwidth; real numbers depend on the runtime, drivers and context length. Community figures are medians of user reports.