# AI Can Act Now. The Harder Problem Is Knowing When It Shouldn’t. Author: Cassidy Yepez Author URL: https://www.hometeams.io/playbook/author/cassidy-yepez Published: 2026-09-25 Meta Title: Coach AI: Privacy, Permissions and Smarter Family Care Meta Description: See how HomeTeams Coach uses AI to organize care, search health records, create actions, and respect each teammate’s permissions and privacy. Tags: hometeams, care coordination, Coach Tag URLs: hometeams (https://www.hometeams.io/playbook/tag/hometeams), care coordination (https://www.hometeams.io/playbook/tag/care-coordination), Coach (https://www.hometeams.io/playbook/tag/coach) URL: https://www.hometeams.io/playbook/coach-ai-knowing-when-not-to-act Coach is the AI teammate inside [HomeTeams](https://www.hometeams.io/playbook/when-care-comes-home-with-dr-apurv-gupta). It can turn an uploaded school calendar into no-school-day reminders, help find a lab value buried hundreds of pages into a medical record, take a spoken request and turn it into action, or keep working on a larger task after you close your phone. But some of the most important technology behind Coach is not about what it can do. It is about knowing what it should be allowed to do. That distinction is becoming more important as artificial intelligence moves beyond answering questions. Today's AI systems are increasingly able to use tools, interpret documents, search information, create plans, navigate software, and complete multi-step work. The technology is becoming more useful because it can act. For families, caregivers, and people managing health information together, that raises another question: what happens when the AI has access to something one person on the team should not see? HomeTeams has had to think about that question from the beginning. > “Coach is a genuine AI teammate,” said Chris Panella, Founding AI Engineer at HomeTeams. “It works the same whether you type or talk to it, it can keep working on something after you've closed your phone and tell you when it's done, and everything it does respects exactly what each person on the team is permitted to see.” Panella describes the philosophy behind it another way: > “It’s the rare AI product where the restraint is as impressive as the capability.” ## Different jobs need different kinds of AI Coach is not one model trying to handle every kind of request. Instead, HomeTeams uses a multi-model approach. Everyday conversation can be handled by a fast conversational model. Photos and documents can be interpreted by a vision model. Voice uses a realtime speech-to-speech model. Another model is used to help pull structured health information from records. ![ Coach AI from HomeTeams](https://prod.superblogcdn.com/site_cuid_cmtbk8rbh003k01xtwmwt4xhs/images/chatgpt-image-sep-24-2026-013248-pm-1790282003880-compressed.png) Panella says Coach also has access to more than 40 purpose-built tools covering areas such as actions, medications, health records, team communication, navigation, and record imports. But those tools are not all handed to the AI every time someone asks a question. Coach first determines what the request actually needs, then makes the relevant capabilities available for that interaction. That can make a simple conversation stay simple while still allowing a more complicated request to become a multi-step workflow. Imagine uploading your child's school calendar and saying, “Add all of the no-school days.” Coach can interpret the calendar, identify the relevant dates, structure the information, build a plan, and show the proposed actions for review. A request that could have turned into 15 individual calendar entries can become one conversation. This is one of the biggest differences between traditional chatbots and the newer generation of AI agents. The AI is not only generating an answer. It is coordinating the steps needed to help accomplish something. ## Voice works like Coach, not a smaller version of it Voice is another area where that difference becomes noticeable. Many people have used voice assistants that can answer a question but become much less useful when the request requires actually doing something inside an app. Coach is designed so voice and text have the same underlying capabilities. A person can speak naturally to Coach while asking it to work with records, create an action, find information, or help navigate HomeTeams. Panella describes the voice experience as realtime speech-to-speech with sub-second conversational latency and natural interruption. Even small details matter. When someone connects by voice, Coach resolves that person's timezone. If a user in Washington says, “Remind me at noon,” the system should understand that person's noon. Those decisions are not flashy, but they are often what make technology feel useful in real life. ## Finding the right information inside a very large health record Health information creates a different challenge. A medical record can contain hundreds of pages from years of appointments, tests, medications, claims, provider notes, and hospital visits. The information somebody needs may be buried deep inside it. Coach uses a combination of semantic search, which looks for information by meaning, and traditional keyword matching. Long records are broken into sections and represented across the full document rather than only concentrating on the first few pages. When Coach finds information relevant to a question, it can retrieve the matching section with surrounding context. So if someone asks about a particular lab result inside a 400-page record, the goal is not to summarize 400 pages. It is to help find the section that answers the question. HomeTeams is also designed to bring information that normally lives in different places into a more useful view of someone's care, including health records, medications, plans, responsibilities, and the people helping manage them. ## People on the same team do not always have the same access This becomes especially important when care is shared. Imagine two adult siblings helping their mother. One handles medication and appointments. The other helps with insurance and transportation. Both may be part of the same team, but that does not automatically mean both should see every piece of information. Coach's permission system is designed around those differences. Access is not treated as simply yes or no. HomeTeams uses three states: unknown, granted, and denied. If a permission check fails or the system cannot determine access confidently, that uncertainty does not silently become permission. And when Coach cannot confidently understand what someone means, it is designed to ask rather than guess. The same principle applies when Coach works on longer tasks. Some requests can continue running after the app is closed and notify the user when the work is finished. But background agents operate with a restricted set of read-only tools. They can prepare results or proposed actions for a person to review instead of quietly making sensitive changes on their own. That approach lines up with a broader shift in AI security. Current guidance from groups including OWASP emphasizes giving agents only the tools they need, separating read and write access, and keeping people involved for higher-risk actions. ## What happens when Coach connects to another AI? Artificial intelligence is also becoming more connected. The Model Context Protocol, usually called MCP, is one way AI assistants can connect with outside tools and information. It is becoming important enough that the National Security Agency released specific MCP security guidance in 2026. Coach has been built as a remote MCP server using modern authorization standards. That can allow compatible AI assistants to connect with HomeTeams capabilities. But connecting another assistant should not mean removing HomeTeams' rules. Permissions still matter. Outside connections should not automatically expand what someone is allowed to see, and sensitive health information requires additional protection. That reflects the larger idea behind Coach. The technology should be useful because it can act, but useful does not mean unlimited. ## Restraint is part of the product The AI industry is increasingly focused on agents that can work longer, use more tools, and accomplish more without constant human direction. OpenAI is building infrastructure for long-running agents. Anthropic has written extensively about containing the potential “blast radius” of increasingly capable agents. Security organizations are developing guidance around tool permissions, prompt injection, and human oversight. Healthcare organizations are asking similar questions. The World Health Organization has highlighted the need for accountability and meaningful human oversight as healthcare AI moves from passive assistance toward agentic workflows. The American Medical Association has also emphasized that AI should support human judgment rather than replace it. Those conversations make one thing increasingly clear: making AI more capable is only part of the challenge. The other part is building the boundaries around that capability. ### For Coach, that means thinking carefully about what information can be retrieved, which tools can be used, what happens when the system is uncertain, what work can continue in the background, and when another person needs to make the final decision. Coach was not built to replace the people around someone. It was built to help those people work together. A genuine teammate. ## FAQs Q: What can Coach AI do inside HomeTeams? A: Coach is the AI teammate inside HomeTeams. It can help turn documents into actions, find information inside health records, create reminders from typed or spoken requests, summarize what is happening across a team, and help organize everyday care. Learn more about how HomeTeams brings care, health information, and teamwork together at https://www.hometeams.io/. Q: How does Coach AI protect private health and family information? A: Coach follows the permissions assigned to each person on a HomeTeam, so being on the same team does not automatically mean everyone can see the same information. HomeTeams is designed so access can be granted, denied, or treated as unknown when the system cannot confidently determine permission. Learn more about HomeTeams privacy practices at https://www.hometeams.io/policies/privacy-policy. Q: Can Coach keep working after I close the HomeTeams app? A: Yes. Some Coach tasks can continue in the background and notify you when the work is ready. For sensitive actions, HomeTeams is designed to keep people involved by limiting what background agents can change and allowing Coach to prepare information or proposed actions for review. See how HomeTeams helps families coordinate care in one shared place at https://www.hometeams.io/playbook/one-calendar-for-whole-family. --- This blog is powered by Superblog. Visit https://superblog.ai to know more. ---