Undercover Ethnographic Journey Inside Scale AI

As an anthropologist deeply fascinated by the intersection of technology and culture, I created an opportunity to conduct a covert ethnographic study at Scale AI, one of the leading companies in AI training and data annotation. My goal was to peel back the curtain on the process of Reinforcement Learning from Human Feedback (RLHF) and understand the human dynamics behind this crucial component of AI development.

What I discovered was both illuminating and concerning. While Scale AI ostensibly aims for diverse perspectives in their feedback process, and rigorous training and standards of excellence, the reality on the ground tells a different story. We RLHF workers found ourselves constrained by a singular set of metrics, heavily skewed towards American values and standards. This became particularly problematic as we were tasked with responding to queries from across the globe, inevitably filtering global diversity through an American lens.

The process itself was far from the precise, data-driven operation one might expect. When faced with unfamiliar topics, we often resorted to quick Google searches or impromptu group solutioning among workers. While this collaborative approach had its merits, it also risked erasing nuance and reinforcing dominant perspectives.

One aspect that particularly struck me was the option to opt out of tasks requiring specialized knowledge, especially in STEM fields. However, this was counterbalanced by a pervasive incentive to provide the "best answer" whenever possible. This push towards authoritative responses, even in areas outside our expertise, raises questions about the accuracy and reliability of the feedback being provided.

The peer review system, while well-intentioned, did little to mitigate these issues. More experienced workers would review our responses, but this process was rife with subjectivity and often reinforced existing cultural biases rather than challenging them.

What's truly mind-boggling is the scale and impact of this work. The feedback generated by thousands of workers like us directly shapes the responses of the world's largest AI chatbots, including those developed by OpenAI, Meta, Microsoft and more. The implications of this are staggering - our biases, guesswork, and cultural perspectives are being baked into the very foundations of AI systems that millions will interact with daily.

This experience has fundamentally altered my perception of AI. Far from the infallible, objective systems they're often portrayed as, these AI brains are profoundly human constructs, reflecting our biases, limitations, and cultural blind spots. As we continue to integrate AI into our lives and societies, it's crucial that we reckon with this reality and work towards more truly diverse and globally representative training processes.

As an anthropologist, I'm left with more questions than answers. How can we ensure that AI systems reflect the true diversity of human experience and knowledge? What are the long-term implications of embedding predominantly American values into global AI systems? And perhaps most importantly, how can we make the human labor and decision-making behind AI more visible and accountable?

These are questions we must grapple with as AI continues to shape our world. My experience at Scale.AI has made one thing clear - the future of AI is not just a technological challenge, but a deeply human one.

EPIC 2016- Inclusiveness in Ethnography (Published in Anthropology News)

http://www.anthropology-news.org/index.php/2016/10/12/epic-2016-inclusiveness-in-ethnography/

National Association for the Practice of Anthropology

Vyjayanthi Vadrevu

As a first time attendee at the 2016 EPIC conference, I was thrilled to find a community I could call home. Conference chair, William O. Beeman kicked off the conference with a musical introduction and Karen Ho concluded with a keynote address on her Wall Street ethnography. EPIC conference attendees included academics from the social sciences (predominantly anthropology) and humanities, as well as professionals versed in design thinking, agile business processes and more. These attendees practice in industries from architecture to tech. While the EPIC community agreed on the conference theme—Pathmaking: emphasizing the power of ethnography and business in combination to create transformative innovation and growth for companies and communities— I was left with the question: What does inclusion in ethnographic practice mean and what are the limitations to including new frameworks within its parameters?

Each day of the conference featured Pecha Kucha sessions (a presentation format of 20 images, shown for 20 seconds each, combined with a narrative). One such session was themed “Frames of Thinking,” and I was particularly intrigued by Deepa Butoliya’s presentation, where she introduced the Indian practice of Jugaad (Hindi): the re-engineering of a device or organization which bypasses traditional systems in order to achieve a particular end goal, given the constraints of existing means. The closest term we have to it is hacking. An example given of Jugaad was a low-income father catering to his children’s request for a flat screen TV by carving a hole in the wall and inserting their unwieldy TV into this hole, securing it with cement. This created the appearance of a flat screen, within the financial means of the family, and allowed a similar TV experience as their wealthier counterparts.

As I wondered why this talk resonated with me so deeply, I realized it was because Jugaad can be used as a framework to look at the practice of ethnography. The deconstruction of Jugaad into its three elements—frugal, flexible and inclusive—works mostly in the context of ethnographic practice. As ethnographers, we are frugal (in that we require little to no instruments to collect and analyze our data), flexible (to be reflexive, holistic, and context-based requires utmost flexibility), but are we truly inclusive?

When it comes to the ethos of Jugaad in ethnography—as a work around or hack for conventional methods of understanding consumers, products, sustainable innovation—I feel it works well. The spirit of Jugaad and hacking involves bypassing existing systems, using knowledge acquired from the masses and weaving together eclectic resources, all in order to create more accessible solutions. In the same way, ethnography weaves together a wide-range of viewpoints from a multitude of voices from the field, in order to provide solutions that are more “user-friendly” or “sustainable” for the consumers and beneficiaries of a company or organization. In the presentation “Applying Theory to Applied Ethnography”, Red Associates provided a hack for using complex academic theory. They call this approach “Low Fidelity” and define it as selectively drawing upon aspects of a theory as modes of engaging with the world. This approach cultivates the understanding that theories are not all-encompassing nor fully developed, and allows the researcher to blend their interpretation of data with an existing theory. This approach also highlights the economical nature of ethnography, and flexibility to dip in and out of theory.

This same example illustrates how we can see inclusiveness in ethnographic practice as well. With the “Low Fidelity” theory-blending approach, the ethnographic practice becomes inclusive of novel interpretation and synthesis techniques developed by the researcher, and allows for these techniques to co-exist and mingle with established theoretical frameworks. Ethnography practice is also inclusive of standard business practices; an adaptation which is required for ethnography to be trusted by non-ethnographers in the non-academic world. While some corporations have embraced ethnography, there are still myths of ethnography being overly time-consuming and theoretical. Ethnography’s adaptability allows for incorporation of business norms, such as translation of field notes into event marketing strategies and feature lists for software devices.

However, ethnographic practice should not always be “all”-inclusive. Shaheen Amirebrahimi expressed such perspective in his presentation, arguing for ethnography to “divorce UX”. UX is a research technique used in many technology companies as a part of the product design process, consisting in examining the consumer’s interactions with existing and new technologies. Amirebrahimi claimed that UX “sterilizes the world to an oversimplified binary reduced to how people use products and how often they buy them,” and that in response to the “ux-ificition” of ethnography (a term used by Thomas Cook), we should “re-inject politics and emotion and messiness back into design.”

There is no doubt that ethnography contains the first two elements of Jugaad; frugality in terms of instruments required for practice (the ethnographer is the collector and the synthesizer), and flexibility, in its ability to be used across industries (due the ethnographer being reflexive and holistic). There is also no doubt that ethnography embodies the soul of Jugaad—the spirit of “hacking” or re-engineering the method of discovery in business in order to get truer, more holistic information that ultimately shapes a more utilizable solution. In this sense, we can conclude that ethnographic practice is also inclusive. The only methods that ethnography excludes are those that excessively simplify social reality, creating “ux-ified” versions of ethnographic practice. If these too were included, ethnography would cease to exist in its holistic form, and thus, lose the ability to extract profound meaning and provide perspicacious strategy.

Vyjayanthi Vadrevu is an independent ethnographer and consultant (www.rasa.nyc).

To submit contributions to NAPA Section News, please contact contributing editor Alice Larotonda (Alice_Larotonda@brown.edu).

Anything You Can Do I Can Do Better: Tribal Warfare Within the Company Walls

Engineering vs. marketing. Legal vs. Finance.  Haven’t we all seen these long-standing rivalries? Yet they continue to exist. In efforts to dissolve these rivalries, companies often treat the individual (usually the leader), and expect positive change to stem solely from the leader to the team, and eventually, between teams or departments.  What this method fails to do is to see the big, complex, nuanced picture; the problem lies in the system of relationships between these groups, and thus it is the entire system that needs to be examined and treated, not just the individual and the team.

This is not to say the team and the individual should be ignored; they are vital to the pieces that make up the system. Quite often, companies start splitting into factions, often along formal departments, and these departments become their own tribes, with unique rituals, values, and artifacts which are embodied to some extent by each individual and leader of these tribes.  The familial grouping and the camaraderie that comes with the tribal association is excellent for small group dynamics and employee retention within startups, for example (if values and communication styles are in synch or at least complementary).  But when companies grow, and roles are no longer served by individuals but by entire teams, formal and informal tribes start to emerge. When these individuals are placed in roles within a team doing similar work with a similar goal, rituals start to develop, unified values are created, and a subcultural pattern emerges. Once this happens, a team may start viewing their team in relation to other teams and start to compare their subcultural traits; their competing priorities.

To understand what these traits are, we need ethnographic research to identify behaviors, values, etc. and an anthropological lens to view each of the teams in relation to one another, and furthermore, how together they link together create one complex cultural system. We can then use these identified differences to start building systems of inter-departmental communication that aligns and honors both or all cultural patterns to an equal degree.  For example, if company leadership is its own chiefdom and begins to a resemble, let’s say, an ineffective monarchy, it is time for the “monarchy” to begin to practice an ethnographic research method called participant observation.  A great example of this brand of participant observation is the show “UnderCover Boss”, where the leaders employ themselves in the various activities of their subordinates to the best of their ability, and simultaneously take note of the group dynamics, values, and context of the individuals. While the show does not train their leadership to look at their company through an ethnographer’s lens and the big picture analysis of the company cultural system as a whole, at its heart, it is still the method that would most likely re-connect the leadership to their tribes. Bringing in a trained ethnographer to do this “undercover” work will have the added advantages of having a social scientist identify hidden cultural patterns that inform a lot of employee behaviors and sentiments, a participant observer who can move nimbly within the company to get a synopsis on each team/tribal subculture up close, and a company outsider with fresh eyes and a bird’s eye view of the cultural and social ecosystem of the entire organization.

Cultural Anthropology: defined

Culture is an elusive concept, perhaps due to the very nature of the term. "Culture" is constantly changing, due to organic and external factors, due to time and travel, immigration and emigration. Even within our own lifetimes, our own cultural context can shift due a myriad of factors. Culture can loosely be defined as the behaviors that define the human condition, and the values we assign as reasons for these behaviors.

Cultural anthropology is the study of these behavioral norms and values. Here lies the essence, the rasa, of anthropology. It is the study that, as anthropologist Karl Heider puts it, "seeks to understand what is means to be human." Our values, art, politics, religions, etc. are all ways in which we simultaneously create and react to our human existence. Whether we are aware of these systems or not, they play a role in how we respond to situations and individuals involved in these situations. The notion of us vs. them is all relative, depending on the players in each situation, and their cultural contexts. For example, two tribal groups from a particular country could see each other as having very different value systems within their country, but if they were removed from that context and both placed in a completely different country AND continent, they may identify more strongly with each other since their "us vs. the other" dichotomy completely changed.

As anthropologists, we have to be hyper aware of our own cultural norms, biases, and the lens with which we view our own human existence.  The more reflect on our own cultural contexts, the more we can detect our cultural conditioning and how it is influencing our research.  Gaining this awareness is a tremendous skill when working with different cultural groups, so that we inject as little bias as possible in our research, or at the very least, are able to step outside of our own context to view various cultural and subcultural groups with various lenses.