The moral authority of ChatGPT. (arXiv:2301.07098v1 [cs.CY])

ChatGPT is not only fun to chat with, but it also searches information,
answers questions, and gives advice. With consistent moral advice, it might
improve the moral judgment and decisions of users, who often hold contradictory
moral beliefs. Unfortunately, ChatGPT turns out highly inconsistent as a moral
advisor. Nonetheless, it influences users’ moral judgment, we find in an
experiment, even if they know they are advised by a chatting bot, and they
underestimate how much they are influenced. Thus, ChatGPT threatens to corrupt
rather than improves users’ judgment. These findings raise the question of how
to ensure the responsible use of ChatGPT and similar AI. Transparency is often
touted but seems ineffective. We propose training to improve digital literacy.