What comes with an AI model on your laptop?

“Runs on a laptop” opens a useful question

A maintenance-note tool might process a document without sending its contents to a hosted model, provided its surrounding components support that route. That last condition is where I would start the buying decision. The model can run locally while conversion, reporting or a fallback still sends data elsewhere. Memory capacity matters, but the business also needs useful results from the whole task and someone able to maintain the installation.

Google’s 3 June 2026 announcement introduced Gemma 4 12B and described local operation with 16 GB of VRAM or unified memory. Its 1 July roundup brought that laptop-oriented direction into a wider account of its June releases. These are supplier descriptions, not measurements on the reader’s machine. The interesting development is the option they invite us to examine, while suitability still depends on the intended machine, runtime and complete document-processing task.

Take a maintenance-note extractor at a site with unreliable connectivity. Keeping sensitive documents offline would be valuable there. The same business still needs consistent structured output, a recoverable interrupted run and support when the person who installed it is away. Give local inference that complete job to perform. A laptop demonstration can answer whether a model launches; it leaves the rest of this operating choice to investigate.

I would write the decision as a comparison between complete operating routes: local inference, hosted inference and, where it makes sense, a limited local route with an explicit hosted fallback. The maintenance team should be able to identify the route each document will take before using the tool, especially when unreliable connectivity or sensitive material is the reason for considering local processing.

Price the completed task, not the absence of an API bill

Here is a deliberately invented monthly cost model. A hosted route costs £0.04 per accepted document task, with no fixed cost included in this simplified comparison. A local route has £100 monthly allocated cost and £0.01 variable cost per accepted task. The fixed amount might represent equipment allocation and support time, but it is a planning assumption, not a hardware quote or market price.

Under these assigned costs, 1,000 tasks produce a £40 hosted estimate and £110 local estimate; at 5,000, the totals become £200 and £150. Dividing the £100 fixed allocation by the £0.03 variable difference gives a crossover of about 3,334 monthly tasks. That calculation identifies where the simplified infrastructure estimates meet. Review work, actual hardware requirements and support effort could change the purchasing decision on either side.

Add one assigned quality difference: the local route needs two extra minutes of correction on one in twenty accepted tasks. At 5,000 tasks, that means 250 corrections and 500 review minutes. A trial has to find out whether that difference exists. If it does, keep those minutes beside the £50 infrastructure saving even before agreeing an hourly value. The missing API bill cannot stand in for an account of all the work the business still has to do.

The word accepted matters. If a route rejects difficult documents and the cost denominator includes only its successes, it can look inexpensive while transferring the hard work elsewhere. Count attempted, accepted, referred and failed tasks, then show the route for each. A useful local tool may deliberately handle a limited subset. Its value should come from that subset, not from quietly removing failures from the calculation.

Keep startup and steady operation separate in the timing sheet. The first document after loading a model can have a different waiting experience from the next document in an active session. Measure both where they matter to the workflow. A tool opened once each morning and a service used continuously all afternoon should not inherit the same latency assumption merely because they use the same model artefact.

Capacity is another constraint. A local installation may serve one person well and create a queue when several colleagues arrive together. Measure a representative burst, the largest supported document and the foreground applications that must coexist on the machine. Fitting weights into memory does not establish usable latency, available context or reliable performance under the workload the business actually has.

Inspect where the information travels

Local inference can keep model inputs on the machine when the application and surrounding components are configured accordingly. It does not make every connected feature local. Document conversion, error reporting, software updates, remote search and optional fallback can each create a separate network path. Draw those paths before writing a privacy claim in a procurement note.

Distinguish offline extraction from the separate feature that looks up a product on a supplier website. The lookup may be useful, but its model identifier or search terms leave the machine. Sending the full document to a hosted fallback changes the boundary much more. Make that escalation a visible, governed choice. A setting promising a better answer should not quietly decide that a document chosen for local processing may now be disclosed.

Storage matters too. Local source files, extracted text, temporary conversion files, logs and result histories may outlive the request. Decide who can access them, how they are protected and when they are removed. A local machine shared by several people can present a different risk from a well-managed hosted service. The location of inference is one architectural fact among several.

Define the data or task that must remain within the chosen boundary, then verify the full route against that requirement. If the maintenance tool must work offline, the acceptance exercise should disconnect the network and cover launch, document conversion, extraction and result retrieval. This tests a consequential property of the installation rather than assuming privacy or availability from the deployment category alone.

The model’s licence and distribution route belong in the review as well. Verify the terms and the actual artefact being installed, including any quantised conversion or bundled runtime. An announcement about an open release does not establish the provenance of every file someone finds with a similar name. Keep the installed model and runtime revisions in the operating record so the result can be investigated later.

Buy an operating responsibility you can maintain

A useful local pilot should leave the business with a repeatable task pack and a clear owner, not just a pleasing demonstration. Record the machine, available memory, model artefact, runtime settings, document limits and observed task outcomes. Compare the same material through the alternative route where permitted. Report actual timings as observations on that configuration, rather than implying they follow from the model name.

Include a difficult document, an unsupported format and an interrupted run in the pilot. Follow what happens to each draft and how the colleague resumes or refers it. Then try updating the installation and returning to its previous working configuration. Local control lets the organisation choose when to change, which is useful. It also means the business has to keep that change and recovery route working rather than hand the responsibility to a hosted operator.

A hybrid can be a sensible result. Perhaps local extraction handles a known document family offline, while unusual documents are referred to a person. Perhaps a hosted route remains the default and local operation protects one particular setting. Keep the routing rule understandable, and do not send a document to the hosted path if the reason for choosing local was a prohibition on that disclosure.

For this maintenance tool, I would choose the route that can do the required offline work at a quality and cost the team can maintain. Test the intended machine, complete document path and unsupported inputs, then keep the update and recovery record. Local, hosted or a bounded hybrid can each be a sensible result. The decision is easier to defend when it describes who does the remaining work and which data actually leaves.