Today, AI can automate large parts of digital journeys - faster and more scalable than manual processes. At the same time, it is becoming clearer with every automation step that there are times when users need more than a "right answer" - namely guidance, categorization and the feeling that responsibility is clearly defined.
This is exactly where human-centered AI comes in. As a combination of AI and human-centered design, it is about systems that support people in a targeted manner: comprehensible in interaction, clear in guidance, convincing in experience.
Many AI initiatives follow a clear logic: reduce costs, increase speed, optimize conversion and resolution. This logic is necessary - but falls short as soon as interactions go beyond purely functional transactions. For example, when a delayed parcel is not just a logistics problem, but a disappointed customer experience.
Studies show a recurring pattern: AI is primarily accepted where tasks are simple, low-risk and can be solved reliably. In emotionally charged or complex situations, on the other hand, the expectation that human support will remain available increases. (YouGov/Pega, 2026)
This is not a fundamental debate about "man vs. machine", but an architectural decision: Where does AI lead - and where must a human be able to lead or take over seamlessly?
In practice, this architectural question is particularly evident in conversational interfaces: chatbots, assistants and generative UI formats. Conversational design is not "wording", but the design of decision-making moments along the journey - situations in which users experience uncertainty and need guidance, recommendations or a planned handover. This is precisely where context, recovery path and human-in-the-loop decide whether AI generates efficiency or damages trust.
The line can be drawn pragmatically: The greater the emotional impact - anger, uncertainty, money, risk - and the more complex the process, the more important human responsibility becomes. In simple, recurring cases, AI can provide very reliable support or lead completely. In high-stakes moments, human leadership must be available - including proper handover and context.
An order confirmation, for example, is usually standard. An invoice complaint or a delivery problem at a critical moment, on the other hand, requires guidance, scope for goodwill and clear accountability.
In May 2025, Klarna CEO Sebastian Siemiatkowski admitted that an excessive focus on cost reduction in customer service had led to lower quality. At the same time, he emphasized how important it is that customers always have the opportunity to speak to a human. The case shows: AI can make service more efficient - but where quality, trust and a sense of responsibility count, human accessibility remains a central part of the experience. (Bloomberg, 2025)
What is becoming apparent in service as a question of responsibility and handover also increasingly applies to commerce journeys: if AI is already shaping discovery and pre-selection, trust is not only decided at the support stage, but also at the purchase initiation stage.
Product discovery is increasingly shifting to AI interfaces in many categories. Buyers no longer necessarily start their orientation in the classic search or on the store homepage. Instead, needs are specified iteratively via prompts, alternatives are compared more quickly - and a pre-selection is often made before direct touchpoints with a brand even occur.
At the same time, AI increases comparability: price, availability and product fit become visible more quickly, which can increase interchangeability in parts of the product range. Differentiation thus shifts more strongly towards brand, trust and loyalty. Stores need a clearer reason for customers to not only buy once, but to come back. This is precisely why CRM and loyalty are becoming more important: every direct interaction - on the PDP, in service or in after sales - must contribute more to building relationships, fulfilling the brand promise and creating concrete reasons for repeat purchases.
AI agents also potentially shorten the path from need to decision, which may shift access to brands further towards platforms and AI ecosystems.
Gen Z in particular is sensitive to the use of AI in brand communication: in recent studies, such brands are perceived as inauthentic or unethical significantly more often than by millennials. (IAB/Sonata Insights, 2026)
Many organizations primarily optimize what is easily measurable: speed, costs, completion rates. Relationships, trust and decision-making reliability are often not taken into account.
This is precisely where modern AI systems could provide support - through early signals, risk forecasts and prepared handovers. However, these dimensions must be consciously incorporated into model, metrics and architecture decisions.
The measurement logic is also important: a high agent rate sounds good - but if the quality of problem solving suffers, this can damage trust and loyalty in the long term.
Human-centered AI is not an add-on, but a structural issue: journey design, service setup and measurement logic must work together. The following principles bundle the most important experience decisions and the necessary system guard rails to make human-centered AI scalable and reliable.
1. evaluate touchpoints according to emotional weight, not just conversion
Touchpoints are prioritized according to emotional impact, not just drop-off. In addition, there needs to be a key figure that shows whether interactions "felt resolved" - for example via follow-ups, sentiment analysis or qualitative feedback. CSAT alone says what happened - not how customers felt about it.
2. secure the four most critical moments, don't treat everything the same
Particularly critical moments determine memory, trust and loyalty:
In high-stakes cases, "human availability" is not enough: clear responsibilities and decision-making powers are needed for exceptional cases.
3. keep human help available, with a clean handover context
The hurdle to the real person remains as low as possible (ideally: ≤2 steps) when frustration or uncertainty increases. Handovers provide the full context - what happened, what was tried, why it was handed over - so that nobody starts from scratch. Conversational recovery avoids loops through short clarification questions, sensible alternatives and clean escalation.
4. research & testing with people, not just assumptions and metrics
User needs and context are systematically collected and prototypes are tested with real users at an early stage. The results flow iteratively into dialog logic, handoffs and agent mandates - instead of learning about KPIs only after go-live.
5. set up governance & monitoring, do not decide ad hoc
A governance setup defines roles, criteria and decision paths for risk assessment, approvals and monitoring over the entire life cycle. The aim is a procedure that is auditable and makes decisions consistent.
6. enable control, not blind trust
The system enables people to understand and override output and limit risks - especially in high-stakes cases. In agent-based setups, mandates are explicit: what the agent is allowed to do, who is responsible and which actions are reversible.
7. prove & test answers, not just prompt them
Answers are linked to reliable sources of knowledge, rules and current data and are systematically checked: Test cases, quality criteria, monitoring. In this way, edge cases, quality drops and rule violations become visible at an early stage - and can be improved in a targeted manner.
Human-centered AI is not a counter-model to technology. It is its consistent further development. At Diconium, we do not see human-centered AI as a limitation of technological possibilities, but as their consistent further development. Our experience from commerce, CX and AI projects shows that successful organizations are not those with the highest automation rate, but those that decide precisely how and when automation transfers responsibility to people - and which information must be retained in the process.
This means
The real question for the future is therefore not how much AI is possible. It's about how clearly companies decide where AI can lead - and where humans must lead. Getting this right not only scales efficiency, but also trust. And this is precisely what will make the decisive difference in agent-based commerce.