AI ethics is typically framed as a philosophy problem. It is not. For engineering teams building AI features, ethics is a series of concrete product and architecture decisions that happen every week: whether to log conversations, what to tell users when the AI is wrong, whether user consent is required to use their inputs for model improvement, and when to stop an automated AI decision and involve a human.
This guide covers the decisions, not the theory.
The Questions Engineering Teams Actually Face
Should we log user conversations?
Almost always yes, with specific requirements. Logs are essential for debugging AI behavior, investigating user complaints, and improving quality. The ethical requirements: inform users in your privacy policy that conversations may be logged, define retention limits (90 days is common; longer requires stronger justification), restrict access to logs to people who need them, and give users a way to delete their conversation history.
What is not acceptable: logging conversations without disclosure, using conversation logs to retrain your model without separate consent, providing logs to third parties without a data processing agreement.
Do we need user consent to use their inputs to improve the model?
Yes, in most jurisdictions. The legal basis for processing personal data to improve your own AI model is typically consent, not legitimate interest -- because the benefit to the user of the model improving is indirect and hypothetical, while the cost to their privacy is immediate and real. GDPR Article 6 requires a clear legal basis for processing; using customer conversations for model training typically requires explicit opt-in.
In practice: add a checkbox in your settings (opt-in, not opt-out) that allows users to contribute their usage to model improvement. Make the benefit clear ("Your feedback helps improve responses for everyone"). Do not make this checkbox part of the initial onboarding flow where users click through without reading.
What do we tell users when the AI makes a mistake?
Tell them immediately and directly. AI mistakes are not software bugs -- they are inherent to how the technology works, and users know it. A straightforward "Our AI got this wrong -- here's the correct information" is better received than a defensive explanation.
For consequential mistakes (an AI-generated legal summary that was wrong, a medical information error, a financial calculation that was incorrect), the response must be more active: proactively reaching out to users who received the wrong information, correcting the record, and providing access to the right resource.
Do not try to explain away mistakes by saying the model "interpreted the request differently." Own the error. Fix it. Tell the user.
Should we disclose when content is AI-generated to users who did not generate it?
Yes. If user A uses your tool to generate content that user B will read or act on, user B has a legitimate interest in knowing the content was AI-generated. This applies to: AI-generated emails sent from a customer to a business, AI-generated meeting summaries distributed to participants, AI-generated proposals or documents sent to clients, AI-generated social media content.
Disclosure does not need to be prominent or disruptive. A small "Generated with AI" label is sufficient. The absence of disclosure is deceptive.
The Framework for Ethical AI Decisions
When facing an ethical question about an AI feature, work through these five questions:
Who could be harmed? Identify the people who might be negatively affected by this AI feature's output or behavior. Include users, third parties, and marginalized groups who may not be obviously visible in the use case.
How could they be harmed? Be specific. Not "data privacy could be affected" but "a user's employment status could be revealed to their employer through a feature that analyzes calendar patterns."
How likely is the harm? Is this a theoretical edge case or a predictable outcome for a significant percentage of users? A 0.01% risk of a minor harm is different from a 10% risk of a significant harm.
How severe is the harm? Can the affected person recover easily? Is the harm reversible? Does it affect just one person or many?
What mitigation is reasonable? Given the likelihood and severity of harm, what steps are worth taking? At low likelihood and low severity, acknowledgment and a feedback mechanism may be sufficient. At high likelihood or high severity, more significant intervention is required.
Concrete Decisions: The Lines That Should Not Move
Always disclose when content is AI-generated to people who did not generate it. No exceptions. Users who generated the content know it is AI-generated. Everyone else who sees it deserves to know too.
Always give users a way to correct AI mistakes. No AI feature should be a dead end. There must be a path: flag the error, request human review, or provide an alternative. Users who cannot correct AI errors become users who cannot use your product.
Never use biometric data without explicit consent. Biometric data (face recognition, voice print, gait analysis) is uniquely sensitive because it is permanent and intrinsic. Using it without consent is a severe privacy violation and is now illegal in many jurisdictions.
Never make consequential decisions solely with AI. For decisions that have significant impact on a person's life -- credit approval, employment decisions, medical recommendations, child protective services actions -- AI can inform the decision but should not make it without human review. The consequences of error are too high and the accountability too thin.
Never hide AI from users in a way designed to manipulate them. AI companions, customer service bots, and recruiting chatbots that pretend to be human are deceptive. Users have a right to know they are talking to an AI. In the EU, this is now a legal requirement under the AI Act.
The Practical Checklist for New AI Features
Before shipping any AI feature, the engineering team should be able to answer yes to each of these:
- [ ] Users are informed that AI is involved in this feature
- [ ] Privacy policy discloses data logging and retention practices
- [ ] Users have a mechanism to correct or dispute AI output
- [ ] Model training on user data requires explicit opt-in consent
- [ ] The feature has been tested for performance across relevant demographic groups
- [ ] There is a fallback path for when the AI fails or produces low-confidence output
- [ ] Consequential decisions have human review in the workflow
- [ ] The team has documented what constitutes a mistake and what the response process is
- [ ] Monitoring exists to detect quality degradation
If any item is no or unknown, that item is a risk that needs to be addressed before launch.
When AI Mistakes Have Already Happened
Despite best efforts, AI features will produce mistakes that reach users. The response process:
- Identify the scope: how many users were affected, what was the incorrect output, what actions might they have taken based on it.
- Notify affected users if the mistake was consequential. Do not wait for them to notice.
- Correct the AI output if possible (update summaries, retract incorrect information, flag affected documents).
- Investigate the root cause: was this a one-time model error or a systematic problem?
- Add the failure case to your evaluation set so you can test for it going forward.
- Decide whether the feature needs temporary suspension while the issue is fixed.
The worst response to an AI mistake is silence. Users notice, and the discovery that a company knew about an AI error and did not tell them is significantly more damaging than the error itself.
Keep Reading
- Responsible AI for Product Teams -- the risk classification framework and GDPR requirements
- AI Product Management Guide -- building evaluation and monitoring systems that catch problems early
- AI in Customer Support Guide -- applying the ethics framework to support automation
Pristren builds AI-powered software for teams. Zlyqor is our all-in-one workspace -- chat, projects, time tracking, AI meeting summaries, and invoicing -- in one tool. Try it free.
