When it makes sense
Predictable high volume, strict data residency, or a narrow task a small fine-tuned model handles well. For occasional general-purpose use, an API is almost always cheaper.
Hardware reality
A quantised 7B-class model runs usefully on a single modern GPU and acceptably on CPU for batch work. Latency, not capability, is usually the constraint on modest hardware.
The hidden operational cost
You now own uptime, updates, scaling and evaluation. Budget the same attention you would give any other production service.
