You type a question into an AI app. Where does the work happen? It may happen on your computer, on a remote server or in a mixture of both. The interface can look much the same.
Here, “local AI” means the model runs on your own device. A model is the software component that produces the answer. If that particular task stays local, its input does not need to go to a remote model.
Local does not mean every feature is offline
An app may run its chat locally while using the internet for search, updates, file syncing or other features. It may also collect diagnostics. Check each feature's behaviour instead of relying on the word “local”.
For example, Microsoft's on-device AI documentation describes tools that work offline after their model files have been downloaded. The download itself still needs a connection.
Your computer does the work
A local model uses your device's memory, storage and processing power. Its speed and capabilities depend on the model and hardware. Microsoft describes these considerations in its local and cloud AI comparison.
A remote service may be a better fit for a task your device cannot handle well. Check the service's data practices and costs before choosing it.
Four questions to ask
- Does this task run on my device or a remote server?
- Which features need an internet connection?
- Are my prompts, files or diagnostics stored or sent anywhere?
- What permissions does the app need, and can I change them?
Local processing does not make an answer accurate or protect a computer that is otherwise insecure. Keep your device updated, control access and maintain backups.
Try one task with non-sensitive material. Check the documentation, available settings and any offline behaviour before trusting the app with private work.