selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

InternScience logo

Can the Apple M4 Pro (48 GB) run MLExpert?

MLExpert·34.7B (3.5B active)·Apple M4 Pro (48 GB)·48 GB unified memory

Saved in this browser, used on every model page.
Yes

Yes. The Apple M4 Pro (48 GB) runs MLExpert fully in unified memory at Q6_K (about 31 GB of 36 GB), at roughly 53 tokens/s.

best quant
Q6_K
needs
31 GB
usable
36 GB
est. speed
~53 tok/s

Estimates for an 8K context; offload assumes 32 GB of system RAM.Try other contexts on the MLExpert page →

Every quantisation of MLExpert on the Apple M4 Pro (48 GB)

QuantQualityMemoryFitSpeed
Q2_K Heavy loss15 GBfits~132 tok/s
Q3_K_M Noticeable loss18 GBfits~89 tok/s
Q4_K_M Small loss23 GBfits~72 tok/s
Q5_K_M Near-lossless28 GBfits~61 tok/s
Q6_K pickNear-lossless31 GBfits~53 tok/s
Q8_0 Lossless37 GBtoo big—
FP16 / BF16 Lossless69 GBtoo big—

Speed is a bandwidth-based estimate for generating text (273 GB/s); prompt processing and long contexts are slower.

Measured speed on the Apple M4 Pro (48 GB)

No community reports for this pair yet.

Report your own speed →

Run MLExpert on the Apple M4 Pro (48 GB)

Using Q6_K, the best fit for Apple M4 Pro (48 GB).

Ollama · Q6_K straight from Hugging Facedocs
ollama run hf.co/bartowski/InternScience_MLExpert-GGUF:Q6_K
llama.cpp · OpenAI-compatible server with a web UIdocs
llama-server -hf bartowski/InternScience_MLExpert-GGUF:Q6_K

brew install llama.cpp ships with Metal. Downloads the GGUF on first run.

LM Studio · Download with the lms CLIdocs
lms get bartowski/InternScience_MLExpert-GGUF@Q6_K

Then load it from the LM Studio app or with `lms load`.

Transformers · Chat with the pipeline APIdocs
from transformers import pipeline

pipe = pipeline("text-generation", model="InternScience/MLExpert", dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Explain self-hosting in one paragraph."}]
out = pipe(messages, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])

Chat models that fit the Apple M4 Pro (48 GB) comfortably

All that fit →
ModelParamsBest quantMemorySpeedLicence
36BA3BQ6_K≈31 GBFits in memory~61 tok/sCommercial OK
31.2BA4.6BQ6_K≈32 GBFits in memory~40 tok/sCommercial OK
32.8BQ6_K≈29 GBFits in memory~5.6 tok/sCommercial OK
RecursiveSI logoflintRecursiveSI
39.8BA1.3BQ6_K≈33 GBFits in memory~146 tok/sCommercial OK
20.9BA4.2BQ8_0≈13 GBFits in memory~34 tok/sCommercial OK
30.5BA3BQ8_0≈33 GBFits in memory~47 tok/sCommercial OK

Other GPUs that run MLExpert

Frequently asked questions

Can the Apple M4 Pro (48 GB) run MLExpert?
Yes. The Apple M4 Pro (48 GB) runs MLExpert fully in unified memory at Q6_K (about 31 GB of 36 GB), at roughly 53 tokens/s.
How fast does MLExpert run on the Apple M4 Pro (48 GB)?
An estimate from memory bandwidth gives about 53 tokens/s at Q6_K.
Which quantisation of MLExpert should I use on the Apple M4 Pro (48 GB)?
Q6_K: the largest one that fits. Expected quality: near-lossless compared with the full-precision weights.

Memory and speed are estimates from the model's size and the card's memory bandwidth; real numbers depend on the runtime, drivers and context length. Community figures are medians of user reports.