Saturday Code Snippet

Running inference on Llama3.1 Instruct on a consumer-grade workstation.

This short code snippet loads Meta’s new Llama3.1 Instruct model and performs inference on it.

For this exercise, I was particularly interested in mathematical reasoning. and prompted the model with the question “Why is 9.11 is larger than 9.9? Explain your reasoning step by step.”

Below is the code that worked on my workstation. If you have any question…