Writing a useful answer and completing a task are different accomplishments.
An AI system may produce a convincing troubleshooting guide without knowing which machine is in front of the technician. It may recommend a supplier without checking purchasing restrictions. It may draft a report that reads well but leaves out information the reviewer needs to approve it.
These gaps become more consequential when AI moves beyond generating content and begins taking action.
Agent AI: Surveying the Horizons of Multimodal Interaction, a research survey co-authored by Zane Durante, Qiuyuan Huang, Fei-Fei Li, Jianfeng Gao, and colleagues, offers a useful perspective on this shift. First submitted to arXiv in January 2024, the paper examines AI systems that perceive and act within physical and virtual environments. Its authors describe Agent AI as a promising avenue toward artificial general intelligence, rather than an established solution to it. Read the paper.
For professionals and business leaders, the immediate question is practical: what does it take to put systems that can act to work responsibly and effectively?
This article considers that question through the lens of AI-native professional and organizational capability. The business examples and recommendations below are our own analysis, informed by the paper.
Start with the task and its environment
The paper describes Agent AI as interactive systems that use visual, linguistic, and other information grounded in their environment to produce meaningful actions. It brings together research across areas including gaming, robotics, healthcare, and multimodal understanding. See the abstract and full text.
For workplace applications, this suggests a useful starting point: examine the conditions under which a task must be completed.
Consider equipment maintenance. Producing a generic checklist is relatively straightforward. Helping a technician investigate a particular fault requires more context: the equipment model, its current condition, previous troubleshooting attempts, and any procedures that require a qualified specialist.
A team designing an AI assistant for this work needs to specify the whole sequence:
Define the objective → establish the relevant context → choose an action → check the result → adjust or hand over.
Each stage raises a design question. Does the system have enough information? Can it access the right tool? How will it know whether an action worked? When should a person take over?
Answering these questions makes the proposed application easier to build, evaluate, and improve.
More inputs are useful when they provide better evidence
Workplace information rarely arrives in a single format. A customer complaint may include an email, a voice message, product photographs, and a service history. A training session may depend on spoken instructions, a physical demonstration, and readings from a machine.
Reducing everything to text can discard details that matter.
Imagine a customer reporting that a machine “keeps making a noise” and uploading a short video. The message tells us what concerns the customer. The audio may help identify the fault. The footage may show an indicator light or an operating error.
A useful system would assess these sources together and identify what remains uncertain.
For a product team, the first question should be: what evidence is necessary to make a sound decision in this task?
The ability to accept images, audio, and video does not guarantee that the evidence is sufficient. A photograph may be blurred. A maintenance record may be outdated. Two sources may contradict each other.
The appropriate next step could be to request a clearer image, verify a record, or refer the case to a specialist. Handling missing or conflicting information belongs in the product design from the outset.
Design the full workflow
A demonstration can follow a carefully chosen path. A working service has to cope with incomplete requests, conflicting requirements, and failed actions.
Take a hypothetical procurement assistant. It might consolidate requirements, compare quotations, and draft a purchase request. Before it can place an order, however, it must account for budgets, authorization, contract terms, and delivery requirements.
Those constraints need to become explicit operating rules.
The objective must be clear. Finding the lowest price is different from finding an offer that meets specifications, budget, and delivery deadlines.
The information available to the assistant must also be defined. Which supplier records can it access? How will it identify an expired quotation? What should it do when a requirement is ambiguous?
Its authority needs similar precision. Preparing a comparison table, submitting a request, and committing expenditure are separate actions. They may require different permissions and review points.
Finally, the team needs a completion standard. A generated purchase request is an intermediate output. The task may only be complete when the request contains all required information, reaches the correct approval process, and has a verifiable status.
Domain expertise is essential to this work. Experienced practitioners can identify which conditions are routine, which require interpretation, and which decisions carry responsibilities that must remain with an authorized person.
For AI product managers, this is a central part of the role: connecting user needs, system behavior, and operational accountability.
Measure completion, quality, and the effort left to people
Suppose an AI assistant drafts a report in two minutes, but a reviewer spends forty minutes correcting it. Has the workflow improved?
The answer depends on the previous process, the quality required, and the cost of errors. Generation speed alone cannot settle the question.
We recommend assessing workplace pilots across four dimensions:
- Task completion: Did the system meet the agreed acceptance criteria?
- Quality: Were the facts, required steps, and business rules handled correctly?
- Human effort: How much review, correction, intervention, and rework was needed?
- Operating cost: What resources were required for tool use, maintenance, and exception handling?
Keep the existing process as a baseline. Count the work needed to supervise and maintain the AI system alongside the work it performs.
Testing should cover both ordinary cases and consequential exceptions. Representative everyday cases help establish whether the application delivers value. Unusual cases help reveal its limits.
Before the pilot begins, the team should also agree on how it will use the findings. What would justify expanding the application? What would require changes? What would trigger a pause?
Evaluation becomes useful when its results guide those decisions.
Make human oversight workable
“Subject to human review” sounds reassuring. Its value depends on how the review actually works.
Someone must be responsible for it. They need enough time, relevant evidence, and the authority to stop or change an action.
A useful handover should explain what the system has done, what information it relied on, and what remains unresolved.
For example, when a customer service assistant escalates a complaint, it should provide the customer’s request, verified order details, steps already attempted, and the outstanding dispute. The person receiving the case can then continue the work without making the customer start again.
Teams also need to be precise about how they improve the system. Adding context to a current task, updating a knowledge base, changing workflow rules, and retraining a model are different interventions.
Recording an interaction does not, by itself, ensure that the system will avoid the same mistake next time. Improvement requires an owner, a proposed change, and a way to check whether that change helped.
The capabilities professionals and organizations need
These design choices have direct implications for professional development.
Professionals need to become better at defining work. Clear objectives, relevant inputs, constraints, and acceptance criteria make collaboration easier to assess. Professional judgment includes recognizing weak evidence, spotting exceptions, and deciding when to intervene.
Business knowledge remains valuable here. People who can explain why a procedure exists—and when it should change—provide something a vague task instruction cannot.
Product teams need to design execution that can be evaluated. Their responsibilities extend to tool access, task status, failure handling, and permissions. Each part should support a result that matters to the user and can be checked.
This requires collaboration among product, technical, operational, and subject-matter specialists. Together, they must decide what the system can reasonably undertake and what evidence would demonstrate that it works.
Organizations need ways to turn individual success into shared practice. A useful experiment becomes a dependable capability through common data rules, evaluation methods, clear responsibilities, and documented lessons.
If effective practices remain buried in individual chat histories, other teams cannot readily adopt them. If failures never reach the evaluation process, they are likely to recur.
Capability development therefore includes the organizational work of capturing experience and making it usable.
Begin with a task you can verify
The Agent AI paper provides a research perspective on systems that perceive, act, and respond to their environment. Applying that perspective at work requires careful choices about objectives, evidence, authority, and evaluation.
A practical first step is to choose a bounded task with an observable result. Establish how the current process performs. Define the AI system’s role and handover conditions. Test it against real cases, then use the findings to decide what to change or expand.
This connects with APD Institute’s focus on AI-native capability: helping professionals, teams, and organizations apply AI responsibly, improve workflows, and produce measurable value.
As AI takes on more of the execution, professional responsibility extends to defining worthwhile objectives, setting the conditions for reliable work, and judging whether the result meets the need.
Research reference
Durante, Z., Huang, Q., Wake, N., et al. (2024). Agent AI: Surveying the Horizons of Multimodal Interaction. arXiv:2401.03568. This article refers to version 2, revised January 25, 2024.
Abstract and version history · Full text · PDF
This article offers an independent interpretation of the paper. Business scenarios are illustrative; the implementation recommendations are our own analysis and should not be read as findings validated by the paper.