AI is playing a valuable and growing role in care by offering opportunities to enhance outcomes and improve efficiency. But in order to fully realise the benefits that an AI solution can provide, establishing a strong ethical framework is essential.

In care settings, decisions can have immediate and profound impacts on patients. That is why trust, safety and transparency must form the foundation of any successful strategy. AI language models operate by predicting rather than thinking and may occasionally generate inaccurate information.

This is understandably a concern and barrier to overcome for practitioners across health and social care. However, this challenge is driving innovation and collaboration across the sector to make AI models safer. By combining AI’s capabilities with human judgement, we can create and provide ethical AI solutions that are both reliable and transformative for health and social care.

As one social care professional emphatically stated: “The practitioner’s professional judgement must always be the final arbiter. AI is a tool for support, not a replacement for critical thinking.”

AI Report Findings

Recently, we conducted an AI survey that highlighted the priorities and concerns of the healthcare, social care and education sectors around the use of AI. Drawing on feedback from nearly 300 professionals, these important findings focus on reducing administrative burden, improving care quality, and giving time back to practitioners to focus on what matters most: the people they support.

Within the report, we identified that some practitioners are becoming concerned that AI designed without compassion and ethical principles risks undermining human judgement. This sentiment is echoed in healthcare, where a flawed AI recommendation could have consequences. AI-generated outputs must therefore be verifiable, and practitioners must be trained to critically appraise them.

“I’m an analyst, and when I build a report and share it with someone, there are very strict rules on where and how to share. We have a policy from the trust, and we follow the legal rules and everything when we submit data nationally. However, if AI is doing a similar kind of role, whose responsibility is it to check all that? There shouldn’t be any compromise on patient privacy or consent. These things are very important, not just for legal purposes, but for ethical purposes.” Archana Lingampalli, Pharmacy Business Intelligence Analyst, University Hospitals Bristol and Weston NHS Foundation Trust.

“My main concern currently is around the education of how to use these tools and making sure that every clinician understands where the responsibility lies and their input alongside it.” Helena Langford, Hospital Pharmacist, NHS.

This leads directly to the issue of safety and regulation. The NHS, in particular, is developing a sophisticated and multi-layered approach to AI governance.

As Dr Jon Shaw, System C’s AI Director for Healthcare, stated: “AI tools that have a direct impact on clinical decision-making are increasingly being classified as medical devices, subjecting them to rigorous standards of clinical validation and safety certification. This process is not merely a bureaucratic hurdle; it is a fundamental mechanism for building trust.”

“It involves demonstrating not only the technical efficacy of an AI model but also its safety and reliability in a real-world clinical setting.” Dr Jon Shaw, AI Director for Healthcare, System C “The use of synthetic data to test and validate AI models before they are deployed in live environments is key, allowing for rigorous testing without compromising patient safety.”

Register now for the AI Thought Leadership Webinar – 21st January 2026

To continue the conversation, we are hosting a live AI thought-leadership webinar – AI with Purpose: Discussion and Insights from 300 Voices – on 21st January 2026 (11:00 AM – 12:00 PM), bringing together experts, council leaders, and practitioners to discuss:

  • Key findings from the report
  • Real-world AI applications in healthcare
  • Ethical considerations and governance
  • Q&A with AI specialists and healthcare representatives

Next Steps

Explore our extensive collection of blogs to stay informed, inspired and ahead of the curve. Whether you’re seeking expert advice, industry trends or thought-provoking perspectives, we have something for everyone.

Please get in touch with our friendly team of experts today. We’re here to assist you with any questions or concerns you may have.