Quantizing an 8B Function-Calling Model So It Actually Fits on My RTX 3060
Share

Or: how I learned that “smaller in BF16” isn’t always better than “bigger in INT4.”

 

 Or: how I learned that “smaller in BF16” isn’t always better than “bigger in INT4.”Continue reading on Medium » Read More Python on Medium 

#python

By ali

Leave a Reply