Encyclopedia Autonomica

Encyclopedia Autonomica

Saturday Code Snippet

Running inference on Llama3.1 Instruct on a consumer-grade workstation.

Jan Daniel Semrau (MFin, CAIO)'s avatar
Jan Daniel Semrau (MFin, CAIO)
Jul 27, 2024
∙ Paid
Share

This short code snippet loads Meta’s new Llama3.1 Instruct model and performs inference on it.

For this exercise, I was particularly interested in mathematical reasoning. and prompted the model with the question “Why is 9.11 is larger than 9.9? Explain your reasoning step by step.”

Below is the code that worked on my workstation. If you have any question…

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 JDS
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture