BACKGROUND

Questions related to artificial intelligence (AI) in healthcare literature often focus on how AI will obviate the need for functions performed by health professionals potentially leading to their replacement or, preferentially, how healthcare professionals best utilize AI to optimize care. In other words, will healthcare professionals be subservient to AI or vice versa? Fundamental questions such as the latter are not amenable to answers from the usual scientific approach of evaluating the best available clinical research through the lens of evidence-based medicine. Rather, these are questions that require philosophical and ethical considerations recognizing that knowledge associated with these considerations has at its basis a shared set of facts that must compete in the marketplace of ideas.1 Moral leadership is then necessary for disseminating these facts. As stated by Robert Coles, “To be a moral leader is to reason, directly or indirectly, with others, to expand their sense of the possible, the desirable, the undesirable, and so at times to restrain others, warn them of dangers, even as one is alerting them to possible gains, achievements; to uplift, to try to help enable ideals, give them the life of a personal and social reality.”2

Ethical questions related to advances in AI abound as exhibited by the following headline, “New IVF Startup Claims It Can Predict an Embryo’s IQ.”3 This statement presumes that intelligence is a single measurable entity, despite the sordid history of attempts to use intelligence testing to solve many of society’s ills.4 As noted by Buerki and Vottero in their overview of pharmacy ethics, ethical problems are only likely to increase with advancements in both pharmacy practice and new technologies such as AI.5 Therefore, the purpose of this paper is to discuss ethical issues raised by new technologies such as AI tools that could occur now or in the future with ongoing innovations.

TERMINOLOGY

There is no standardized terminology related to ethics but the following definitions will provide guidance for the use of ethics-related terminology in this paper. Ethics prescribe what humans ought to do based on questions of what is right or wrong or good or bad and is different from scientific, legal, or religious/cultural considerations.6 Ethics can provide us with moral principles or universal rules but it also details the character of the individual (i.e., virtue ethics). Values are more personal and subjective beliefs that humans have about issues and encompass more than character. The terms values or preferences are often employed in comparative effectiveness research without a clear distinction between the two.7

Concern 1: Overreliance on Large Language Models

A concern with one form of new technologies is overreliance by healthcare professionals on the responses generated by large language models (LLMs) such as ChatGPT. Currently, LLMs are known to be prone to hallucinations (i.e., incorrect, nonsensical information) often making the user suspicious of responses and therefore not assuming them to be factual without another form of verification. Healthcare professionals need to be skeptical of any source of medical information, particularly when that information is derived from secondary or tertiary sources; however, LLMs present unique challenges as programming continues to improve leading to more correct responses to questions. As the responses by LLMs improve through new knowledge, reinforced learning, and human feedback, there will be a natural tendency for the user to be less suspicious of the generated answers. This over-trust of LLM responses is termed automation bias. This is a concern when there is an expectation that the LLMs will provide the most relevant and current responses to clinical care-related questions in areas of healthcare that are rapidly evolving. Ethical violations by the pharmacist associated with this over-trust include non-maleficence (do no harm) and beneficence (acting in the best interests of the patient) for not verifying the calculation performed by the AI program. It also raises questions of patient autonomy by assuming an informed patient would make this same decision.

Concern 2: Asking the Right Questions of Large Language Models

The scenario of a clinical pharmacist query of an LLM for ways to improve efficiency and maximize profits without requesting inclusion of ethical considerations is an example of the ethical approach known as consequentialism, which leads to ends justifying means. In this case, humans choose the goals but the means with possible ethical implications receive delegation to technology assuming there is no human review of the LLM responses. This not only disrespects human dignity but also has the potential to foster and twist desires in untenable way making humans more dependent on LLM technology. By constantly letting the LLM determine the means, humans are likely to become less skilled at deciding what are appropriate means, a process that is referred to as deskilling.8 Ultimately, the LLM is deciding the means and the ends with a concomitant loss of human dignity. Human dignity as defined by respect for humanity is an historical foundation for several important ethical concepts with some ethicists considering it to be a universal moral value.9 Additionally, UNESCO’s recommendation on ethics in AI refers to human rights and dignity as the cornerstone of their resource.10

Concern 3: Making Accountability Subservient to Adaptability with New Technologies

Administrators and health professionals must be cautious about ceding decision making to new technologies with no accountability by any specific clinician. In this circumstance, accountability has become subservient to adaptability. This can happen when clinicians are not directly involved in the development, testing, and implementation of new technologies related to patient care. For example, assume machine learning allows for continuing updating of a testing procedure to increase its accuracy. It is not surprising that a hospital might want to implement this updating but human involvement is important from development to implementation to avoid uncritical application of the predictive clinical algorithm.11 In such scenarios, it is unlikely clinicians would fully understand the limitations of the technologies or be able to explain errors that might occur with implementation if they were not involved in the development process. Yet, clinicians are responsible for questions about such technologies raised by patients and family members. Such scenarios raise several concerns of biomedical ethics beyond the obligations of the clinicians. One example relates to possible equity and accompanying effectiveness issues since there is no appreciation of the populations involved in the development of technologies or their ongoing adaptations.

The National Academy of Medicine addresses some of these issues in a guide concerning AI code of conduct for health and medicine released in 2025.12 This guide raises important ethical issues related to AI and provides recommendations that include appropriate participation by all key stakeholders from development to implementation of new technologies. While a key step in patient care decision making focuses on disease, other important steps include empathetic patient communication, situational awareness of the healthcare team, and discussions between healthcare team members including the clinical pharmacist and patient to seek shared goals of care with ethical principles guiding each step.13

Concern 4: Consideration of Perspective When Using New Technologies

Technologies such as LLMs by default respond to queries based on the perspective and input of the clinician seeking information. This is an important consideration since perspectives between clinicians, patients, and other healthcare stakeholders may differ. An example of where the personal perspectives of clinicians may differ from that of patients include conscience clauses (clinicians’ right to refuse to participate for moral or religious reasons) and questions related to unconsciousness in end-of-life care. Few would doubt that the induction of unconsciousness by clinicians is appropriate in situations such as rapid sequence intubation and major surgical procedures. However, some clinicians might argue that purposefully inducing unconsciousness for end-of-life care is always untenable from an ethical perspective. This view might conflict with the autonomous decision of an informed patient who chooses what they consider as a reasonable and ethical alternative in end-of-life decision making. Additionally, only accounting for the clinician’s perspective would preclude input of patient surrogates in patients who already have diminished consciousness from necessary medications (e.g., for pain control) or disease states. Regardless, the perspective of all stakeholders is key to the decision-making process.

This issue of perspective and conscience clauses from a pharmacist standpoint may come into play when considering medications for end-of-life care. For example, titrating a sedative to achieve unconsciousness for end-of-life care has its basis in articles debating the use of the sedative propofol for this purpose.14,15 However, neither the paper arguing for the use of propofol or the one arguing against its use questioned the moral permissibility of aiming for unconsciousness in end-of-life care but rather the debate focused on the use of propofol specifically. Regardless of the details of the propofol debate, there are ethical issues in this case with some clinicians holding the view that purposefully inducing unconsciousness for end-of-life care is always untenable.

Again, the perspective of the clinician, while important, is not the only stakeholder involved in such a scenario. An informed patient may consider unconsciousness as a reasonable and ethical alternative particularly when the patient has or is expected to have diminished consciousness from medications or disease states. It should be recognized that while family members are sometimes necessarily patient surrogates for this decision-making process, there is data suggesting the surrogates do not accurately or reliably assess patient distress particularly during critical illness.16 Clinicians such as the clinical pharmacist in this scenario who consider actions on the part of the patient or surrogate to be inconsistent with their moral or religious beliefs can refuse to participate based on so-called conscience clauses; however statutes associated with these clauses usually require the provider to ensure a patient has ongoing access to care.17,18 The latter involves ethical principles such as justice, non-maleficence, and patient autonomy. One of the most controversial aspects of end-of-life care relates to medical aid in dying or clinician-assisted dying.19

Concern 5: Diminution of Moral Skill Development with New Technologies

It is not unlikely that in the future AI technologies will continue to advance with studies suggesting that AI programs perform at least as well and often better than health professionals for ethical decision making associated with patient care. This could lead to some questioning of the need for these professionals to develop and practice their own virtue ethics or learn and adhere to professional responsibilities associated with core biomedical ethical principles (autonomy, beneficence, non-maleficence, and justice).

Moral skill development is necessary for enhancement of a virtual character and has intrinsic value recognizing inherent human flaws associated with ethical decision making. Human care is inherently a human responsibility that has at its basis moral skill development. We should not seek to yield this responsibility to technologies because it would diminish us as human beings and make us unnecessary in a machine controlling world. Technological advancements should focus on advancing and not diminishing humanity.

There is a possible future in which new technologies begin to replace humans rather than assist them. Moral skill development requires ongoing introspection and practices to ensure appropriate ethical decision making so this is not appropriate for delegation to AI technologies. Additionally, human care is inherently a human responsibility. These are arguments against deskilling moral skill development and allowing technologies to provide human care.20 Both evidence and ethics are necessary for care provision by healthcare providers with virtues providing the basis for good moral action.21,22 The latter is reflected in the original code of ethics for pharmacists that referred to "moral obligations and virtues.23 Aristotle considered virtue to be intellectual and moral with the intellectual aspect from learning by experience and time and the moral aspect from good habits.24 A recent study adds confirmation to the conclusion that our character is malleable as demonstrated by a study evaluating changes in character strengths before and after the September 11 terrorist attacks in the US.25 While there are also country specific virtues, some virtues (e.g., honesty, respect, kindness) seem universal.26

DISCUSSION AND NEXT STEPS

There is a quote attributed to Rabbi Hillel that demonstrates the importance of both personal care and care for others that is at the heart of being an ethical healthcare professional, “If I am not for myself, who will be for me? But if I am only for myself, what am I?” Pharmacists should develop and practice their own virtue ethics and abide by their covenantal relationship with patients while caring for themselves to mitigate burnout.27 Avoiding compassion fatigue and especially burnout can be challenging when important aspects of the decision-making process are in the hands of administrators associated with health plans and policymakers.28 This is illustrated by the ethical issues related to glucagon-like peptide-1 receptor agonists (GLP-1Ras) for weight management given their demonstrated efficacy but high cost leading to issues of equity in their availability.29

The ethical issues related to AI and research endeavors are complex leading to calls for health AI ethics centers involving personnel with expertise in ethics beyond the usual IRB concerns.30 Ideally, adoption of such technologies would result from a priori multidisciplinary collaboration of some type of AI governance committee including healthcare providers, technologists, policymakers, ethics/legal experts, and public engagement with specific focus on those individuals most likely affected depending on the specific innovation.31

For broader-based ethical decision making (e.g., governments, inter-governmental organizations), the World Health Organization (WHO) details six principles of AI for health: (1) protecting human autonomy, (2) promoting human well-being, safety and public interest, (3) ensuring transparency, explainability and intelligibility, (4) fostering responsibility and accountability, (5) ensuring inclusiveness and equity, and (6) promoting responsive and sustainable AI.32 However, pharmacists should be cautious about generalizing to groups of patients (e.g., smokers, obese) but instead recognize the context-specific nature of patient care decision making.33

There has been some interesting discussion of the ethics of specific AI technologies such as LLMs. For example, one study assessing the moral profiles of eight LLMs found that there were differences with the majority using a utilitarian approach (i.e., greatest good for greatest number of people). With additional instruction, the investigators were able to steer the models to different ethical paradigms.34 This is an area in need of ongoing research.

There are several steps clinical pharmacists can take to avoid the types of ethical concerns raised in the scenarios in this paper. The most basic step is to be skeptical of the responses to queries by AI technologies. No matter how often responses are accurate, the user should not become complacent but rather seek other means of verification. Pharmacists should not cede agency to technology, which is a disrespect of their own human dignity. Another key step is to not fall into the trap of letting the desired end of clinical decision making justify any means to achieve that end. Pharmacists should continue to develop their own moral character with ethics that are consistent with broader profession-wide ethical obligations that dictate that the means to achieve ends is a necessary and important consideration. Finally, pharmacists need to thoughtfully consider their responsibilities and obligations to patients beyond legal requirements when taking an action is at odds with their personal moral beliefs.