Home automation has transformed the way we interact with our living spaces. Home Assistant, combined with AI capabilities, offers powerful tools to enhance home automation. In this article, we’ll explore the “Skylar OTG Report” automation, which provides a quick status update on the house whenever I leave.

One of the main reasons I got into home automation was the worry of leaving the garage door open. Though it has only happened once, it left a lasting impression. This automation uses AI image analysis as a workaround when the Aladdin integration for Home Assistant was broken from January 2024 to June 2024. It takes a snapshot from my garage camera and sends it to OpenAI to analyze for people or open doors. This method has proven surprisingly accurate.

Even though the Aladdin integration is now fixed, I prefer the image analysis feature and plan to keep it. However, you can remove this part of the automation if you want to save on costs.

Understanding the YAML Configuration

The YAML configuration for the Skylar OTG Report automation is designed to monitor and report the status of your home when you leave. Here’s a breakdown of each section:

Trigger

The automation is triggered when the specified person (Skylar) leaves the designated home zone:

trigger:
  - platform: zone
    entity_id: person.skyflyt
    zone: zone.home
    event: leave
    enabled: true

This ensures that the automation activates as soon as Skylar exits the home zone.

Actions

Delay Action: Introduces a delay of 3 minutes before proceeding to the next actions. This is useful to ensure that any immediate return doesn’t trigger unnecessary actions.

action:
  - delay:
      hours: 0
      minutes: 3
      seconds: 0
      milliseconds: 0
    enabled: true

Snapshot from Garage Camera: Captures a snapshot from the garage camera and saves it to a specified location.

- service: camera.snapshot
  metadata: {}
  data:
    filename: /config/www/grge.jpg
  target:
    entity_id: camera.plexbox_garage_cam_53

AI Image Analysis: Uses an AI service to analyze the snapshot and ensure the garage doors are closed and there are no people in the garage. The AI response is stored in a variable for later use.

- service: extended_openai_conversation.query_image
  metadata: {}
  data:
    max_tokens: 300
    config_entry: 8fb173114f738c7b6
    prompt: >
      you are a vigilant security guard. Analyze the image and ensure the garage doors are closed and there are no people in it. Your message should be brief either saying Doors Closed, Garage Clear or describe the issue you see.
    images:
      url: https://myhomeassistant.ui.nabu.casa/local/grge.jpg
    model: gpt-4-vision-preview
  response_variable: securityguy

AI House Status Update: Uses another AI service to provide a humorous and concise update on the house’s status, including the security report from the garage camera analysis.

- service: conversation.process
  metadata: {}
  data:
    agent_id: 8fb173114f738c7b6abd
    text: >
      You are an intelligent smart home AI - respond as yourself. Your tone can be humorous like Jarvis, you are a pretty powerful smart home. Skylar has just left the house give an update on the condition of the house. Focus on Doors, Locks, Blinds, Solar Generation, Batteries Charge, Battery time remaining, and any lights left on. Your message should not be longer than 178 characters so it displays correctly. You also have a message from the security agent who has analyzed the garage camera looking for the garage door to be open or for any issues. Include a summary of the security agent's report in your message. Here is the Security agent message "{{ securityguy.choices[0].message.content }}"
  response_variable: agent

Notification to Mobile Devices: Sends the status update to Skylar’s mobile phone and Google Pixel watch.

- service: notify.mobile_app_skylar_pixel_8_pro
  metadata: {}
  data:
    message: "{{agent.response.speech.plain.speech}}"
    title: OTG Update
- service: notify.mobile_app_google_pixel_watch_2
  data:
    title: OTG update
    message: "{{agent.response.speech.plain.speech}}"

Conclusion

By following these steps, you can set up a robust home automation system that not only monitors your home but also leverages AI to provide intelligent updates. This “Skylar OTG Report” is just one example of how powerful Home Assistant can be when combined with AI capabilities.

Stay tuned for more articles on Home Assistant and how to integrate AI into your smart home setups!

Tagged : AI / home assistant