Blog 7: Right to Fair Representation
Published on:
Misrepresentation of culture in generative AI
Case Study:
AI’s Regimes of Representation
Case Study Summary
This case study discusses how generative AI is able to further accentuate bias in society, specifically in South Asia. It discusses how it depicts stereotypes that are external to South Asia, as well as stereotypes that come from external social classes within.
Discussion Questions
1. What does cultural representation mean to you, and how might this definition and your experience with past representation impact how you would evaluate representation of your identity in generative AI models? What aspects of your identity do you think you would center when evaluating representations in AI model output?
Cultural representation to me is how the ways of life are depicted in media, from clothing to architecture. It is the way that we see a culture through the media, often through a narrow lens that does not explain all of the nuances culture can have. For my identity, I would see how closely I feel an AI model output is able to get to what my life feels like. I would input certain characteristics of myself and see if it finds very common stereotypes and portrays those, and certain characteristics that go along with the portayal. Parts of my identity that I would be interested in seeing AI mimic are my ethnic and cultural background, as well as gender/sexuality.
2. What do you think is the role of small-scale qualitative evaluations for building more ethical generative AI models? How do they compare to larger, quantitative, benchmark-style evaluations?
Smaller scale evaluations are important because they are authentic one-on-one human interaction, which is how AI is being used by the public. These can show biases and misrepresentations better because they do not count on an overall average or popularity. Larger quantitative evaluations will take what media often represents, which is often a stereotyped and/or misconstrued characteristic. These can often be from external groups as well, which furthers it’s inability to accurately depict a group.
3. Participants in this study shared “aspirations” for generative AI to be more inclusive, but also noted the tensions around making it more inclusive. Do you think AI can be made more globally inclusive?
AI could be made more globally inclusive, but it would be hard and might have cons. Inclusivity is important because it reaffirms people’s cultural and personal identities, but making sure every person is accounted for is impossible. To an extent I think AI can be improved to better show differences between cultures that are often combined, and show more nuance to cultures that are often stereotyped and given an overly homogenous depiction.
4. What mitigations do you think developers could explore to respond to some of the concerns raised by participants in this study?
Developers could try to do more training on authentic data that comes from within a culture, and have more diverse input. This would cause the training to be more varied on not just one stereotype that is most often shown, such as western views on the east, and how the western media is portraying the east.
5. As mentioned in this case study, representation is not static; it changes over time, is culturally situated, and varies greatly from person to person. How can we “encode” this contextual and changing nature of representation into models, datasets, and algorithms? Is the idea of encoding at odds with the dynamism and fluidity of representation?
I think a way to encode the true nature of cultural is by forcing a model to account for different types of sources, using data from diverse people and groups. This would mean that the AI model is taking a culmination of many different perspectives, which allows for more fluidity in its output. This would hopefully prevent the perpetual change that cultural has from causing inaccurate portrayals, and lessen the societal damage that it produces.
6. How can we learn from the history of technology and media to build more responsible and representative AI models?
By seeing how current AI is inaccurate to cultures around the world, we can make sure that we are fostering ethical developers who seek an educated model that is structurally different and will create less bias. This means that we are not looking to completely eliminate bias in one fell swoop, but rather over time change the design of models in order to steer it in the direction of accurate and positive representation.
My Discussion Question
Should there be legal requirements of how companies are data training in order to ensure that the data they are using is not just from one perspective?
I think it’s important to question whether there should be legal conditions a company must meet because not all companies will necessarily care about the societal impacts that misrepresentation has on groups of people they don’t belong to. It would be interesting to see how companies would change their training algorithms if there was government intervention in the process.
Reflection
This assignment made me reflect on how my own identity can be stereotyped and misrepresented through AI. The generated material can easily affect how people see certain groups, and perpetuate stereotypes that are not representative of the whole group. It’s interesting to think about how AI models can be trained differently in order to account for the fluctuation that can be seen in many cultures. I think that this issue will be an ongoing issue for a while, as there is no on/off switch that will simply stop all future bias and misrepresentation.