AI Task in Home Assistant

Optional AI Automations

AI Task lets an automation send a bounded task to an AI model and receive text or structured data in return. It can produce a short camera summary, a readable weather message, or a simple classification.

It is a building block rather than an AI service of its own. Another integration must provide the AI Task entity and the model that performs the work.

When does AI Task fit?

Good use	Poor use
Write a short summary of non-critical data	Replace a smoke, gas, or leak sensor
Describe a door camera for a notification	Identify a person and unlock the door
Sort text into a few categories	Make an irreversible decision
Produce a friendly status message	Treat free text as an accurate measurement
Extract controlled structured fields	Send the entire home’s history without need

Use normal Home Assistant sensors and conditions for safety logic. Let AI Task word or enrich information where an error cannot trigger a dangerous action.

Understand the flow

Trigger → bounded input → AI Task entity → response → validation → notification or display

AI Task can provide:

normal text;
structured fields that you define;
analysis of supported files or camera images;
a generated image when the selected integration and model support it.

Capabilities vary between providers and models. Check the actual integration feature list rather than assuming every entity can process images or return the same structure.

Choose a local or cloud model

Choice	Advantage	Cost
Local model through a supported integration	Data may remain on your network	Requires hardware, model operations, and often more latency
Cloud model	Easy access and often strong image understanding	Data leaves the home and API use may cost money

A local Home Assistant installation does not make the AI task local. When the model runs at a provider, instructions and attachments are sent under that provider’s terms.

Create an AI Task entity

Choose a compatible AI integration

Read Home Assistant’s documentation for the provider. Confirm it offers an AI Task entity rather than only a conversation agent.
Add the integration from the UI

Go to Settings → Devices & services → Add integration. Follow the setup flow and keep API keys in Home Assistant’s credential storage, not in automation descriptions or YAML examples.
Create the AI Task sub-entry

Open the integration and add the AI Task configuration it provides. Choose a model with the needed features and a sensible usage limit.
Select the preferred AI Task

Go to Settings → System → AI tasks and choose the preferred entity. This makes actions and shared blueprints easier to reuse.
Begin with a text test

Run ai_task.generate_data from the developer tools or a temporary test automation. Use non-sensitive input and ask for a short, checkable answer.

First example: readable indoor climate status

Use existing temperature and humidity sensors as facts. AI Task should only write the message.

Create an automation with a fixed time trigger.
Add the AI Task: Generate data action.
Give it a stable task name such as Morning indoor climate.
Include the current sensor values in the instructions.
Ask for no more than three short sentences and no health advice.
Store the answer in a response variable.
Send it to a persistent notification or your phone.

The instruction might say:

Use only the values below. Write no more than three neutral sentences. Living room temperature: 21.2 °C. Humidity: 46%. Outdoor temperature: 12 °C.

The sensors remain the source of truth. If the AI task fails, the automation should stop or send a normal fallback message containing the raw values.

Camera summary without false security

A supported AI Task entity can accept a camera image as an attachment. A cautious door camera flow looks like this:

Use the doorbell or motion sensor as an ordinary trigger.
Send only the relevant camera image to AI Task.
Ask for a short description of visible objects and activity.
Prohibit guesses about identity, age, health, or intent.
Send the original image and text together in the notification.
Let the recipient decide; do not operate locks or alarms from the description.

A description such as “a person is standing at the door with a parcel” may be useful, but it may be wrong. Keep the original image available for checking.

Use structured responses

Structured output is safer than free text when the next step expects specific fields. Define only what you need:

Field	Type	Example
`short_summary`	Text	“Parcel at the front door”
`activity`	Choice	`none`, `person`, `animal`, `unknown`
`uncertainty`	Text	“Subject partly obscured”

Always validate:

that a response exists;
that each field has the expected type;
that choices are within the allowed values;
that failure leads to a safe fallback.

Privacy and cost

Before enabling a repeated task:

calculate how often it can run on a busy day;
limit image count and retention;
send only the entities and files the task needs;
read the provider’s processing and retention terms;
use spending limits and billing alerts;
state in the automation description which provider receives data.

A camera task every minute may be expensive and intrusive. Trigger it from a specific event and add a cooldown when several events can arrive close together.

Verify before regular use

Test at least:

Normal input with an expected response.
A missing sensor value.
An unavailable AI Task entity.
A slow or empty response.
An ambiguous camera image.
A day with several rapid triggers.

Inspect the automation trace, provider usage, and the message the user actually receives.

Troubleshooting

AI Task cannot be found as an integration

It is a building block. First add an AI provider that explicitly supplies an AI Task entity, then create its sub-entry.

The action cannot process images

Check the entity’s supported features and the chosen model. Do not switch models blindly; test one non-sensitive file first.

The response is empty or truncated

Shorten the instruction, reduce the input, and check provider response limits. Never place the API key in debug output.

The automation response varies

Tighten the instruction and use structured output. If identical input must produce the same critical decision every time, AI Task is probably the wrong tool.

Local voice control

See local Assist for a pipeline that does not require cloud AI.

AI security

Read security and permissions.