Sustainability risks and allocative harms

Artificial intelligence (AI), autonomous systems, and associated technologies are increasingly being presented as disruptive technologies with the ability to fundamentally change research, society, and industry. How to develop such technologies responsibly, and how to best monitor and govern possible risks, has led to a growing body of scientific literature. This includes research on AI ethics,¹ law,² sociology,³ allocative harms that affect vulnerable social groups,^4-6 and the political economy of an increasingly digitized and data-driven world^7,8—just to mention a few.

A number of sustainability risks and ethical challenges emerge from increased uses of AI.⁹ We discuss some of these below but would like to emphasize that the list is not exhaustive. Sustainability scientists, large research programs, and research centers planning on using AI in their work need to consider each of these risks, while also looking into other potential ones, and mitigate harmful effects.

Material, carbon, and water footprint

As the use of AI as a consumer product increases—for example in digital services like chatbots, and the generation of synthetic content like videos and text—so does the ecological and climate footprint of its underpinning infrastructure. This includes the amount of e-waste, such as obsolete specialized processors (graphics processing units [GPUs], tensor processing units [TPUs]), data center infrastructure (servers, hard drives, networking equipment), and end-of-life AI devices (robots, autonomous vehicles, smart sensors).

E-waste represents a growing risk as it is known to release toxic substances like lead, mercury, and cadmium into the environment. These pollutants infiltrate soil and water sources, and can lead to significant ecological degradation and health risks for communities near e-waste disposal sites.¹⁰ Communities affected by e-waste pollution often lack adequate legal protection or avenues for recourse against corporations responsible for harm to the environment and, indeed, living beings.

The impacts of AI training and use are also associated with notable increases in energy use and resulting CO₂ emissions.^11,12 Calculating the resulting emissions has proved challenging,¹³ but a number of attempts have been made in recent years to assess the carbon footprint of individual AI models, company level emissions, and future estimates at the country level.¹¹ Whether the longer-term emissions of AI development

and use will be net positive remains a contested issue. Some analyses point to potential climate net benefits, provided that AI development and use is matched by targeted investments and supportive climate policies.^14,15 Others instead point at secondary and rebound effects, which could lead to growing climate harms despite increased efficiency over time.^16,17

There is also a growing understanding and concern regarding the freshwater use associated with AI development and use. Such uses emerge as the result of onsite server cooling (to avoid server overheating) and offsite electricity generation (i.e., through cooling at thermal power and nuclear plants, and expedited water evaporation caused by hydropower plants).¹⁸ Recent estimates show that the growing use of AI is likely to lead to increased water use in many parts of the world, at times even in areas that already face water scarcity.¹⁹

The expansion of infrastructure needed for AI compute could thus increase the risk of tensions over water rights, land use, and access to rare earth minerals.20 This is an issue that requires further attention, especially considering marginalized and underserved populations that are already disproportionately impacted by climate change-related events such as extreme weather and natural hazards, and resource scarcity resulting from increased energy and water use.

Algorithmic bias and hallucinations

The risks and impacts of possible algorithmic biases and their allocative harm^21,22—that is, when opportunities or resources are withheld from certain people or groups—have gained considerable attention in the last few years. Algorithmic biases of this sort can have a number of sources,²³ and could emerge in the sustainability domain in the following ways:²⁴

Training data bias could emerge if AI systems are designed with poor, limited, or biased datasets. For example, the use of AI based on deep learning (DL) for precision and decision support can help smallholder agrarian communities adapt to a changing climate. However, algorithmic biases created by data gaps could also put the same communities at serious risk if the system’s recommendations are flawed or are not validated properly with local knowledge and expert opinion.

Transfer context bias could emerge when AI systems designed and trained on one ecological, climate, or social-ecological context are applied to a different, incorrect, or unintended context. While the training data and the resulting model may be developed and suitable for the initial social-ecological situation (say, a large industrial farm in a data-rich context), using it in a different setting (e.g., a small farm) could lead to flawed, biased, and inaccurate results. AI systems built on historical ecological conditions may also fail as ecosystems across land and seascapes shift in unexpected and sometimes irreversible ways.

Even if both the training data and the context in which the algorithm is used are appropriate, their application can still lead to interpretation bias. In this type of bias, an AI system might be working as intended by its designer. The user, however, might not fully understand its utility or might try to infer a different meaning that the system might not support.

Bias and the risks of harm can also be found in new forms of AI. Large language models (LLMs) have demonstrated a well-documented ability to rapidly summarize, tailor, and communicate climate and environmental information (see later chapters). However, their potential to create hallucinations—that is, inaccurate or misleading information, for instance false facts or citations—remains a concern.^25,26 LLMs can generate malfunctioning code; refer to or cite non-existent material like legal briefs and articles; produce factually incorrect information; introduce subtle inaccuracies, oversimplifications or biased responses; and make common-sense reasoning failures.

Hallucinations are not only caused by the technical features of LLMs, but also evolve as the result of how humans interact with such AI models through prompting and feedback.²⁷ Several private and public–private initiatives have evolved in recent years to address these issues through technical research and development, and safety protocols (examples in Appendix 1).^28–30 The current lack of universal industry standards, as well as agreed-upon and regulated best practices, continues to pose challenges to mitigating harmful uses of LLMs, however.³¹ In some extreme cases, LLMs have been used to plagiarize or produce low-quality scientific papers and academic books, risking the erosion of integrity and public trust in science.^32,33 Such risks should not be taken lightly as sustainability scientists strive to experiment and use AI for research.

Fig. 2. The number of occurrences of the ten most frequently mentioned countries, along with a selection of the least frequently mentioned ones (chosen to illustrate overall geographical distribution), including affiliations of first and second authors, across the >8,500 studies on AI and the sustainability sciences included in our literature review. Large world economies dominate, such as the USA, China, India, and countries in Western Europe. Map produced by Maria Schewenius.

Data Gaps and Bias

Growing volumes of environmental, social, and ecological data are a fundamental prerequisite for the application of AI in, for example, forest biodiversity monitoring³⁴ and climate modeling.³⁵ Environmental and ecological data have well-known limitations, however, both in their temporal coverage, geographical spread, and information about biological and ecosystem diversity.^36–39 The rapid growth of data from cell phones and satellites offers vast opportunities to map and respond to social vulnerabilities, such as poverty and malnutrition. Meanwhile, it has become increasingly evident that solutions driven by so-called “big data” and AI analysis can be skewed, since the most disadvantaged populations tend to be the least represented in emerging digital data sources.⁴⁰ Indeed, as indicated by the results of the literature analysis conducted in preparation for this report (see AI for the Sustainability Sciences—A Literature Review), most research sites and researchers in the field of AI for the sustainability sciences are associated with high- to higher-medium-income countries (e.g., China, the USA, and countries in Europe) (Fig. 2).

Several of the countries that are the least represented in AI for the sustainability sciences research are also projected to experience some of the most profound changes related to the global issues discussed in this report. For example, Niger, which was not observed in the dataset, has one of the most rapidly increasing populations in the world. It’s projected to double from 26 million people in 2024 to 56 million by 2054.⁴¹ Tuvalu and the Maldives are examples of low-lying islands that are particularly vulnerable to extreme sea-level rise, but had zero and two occurrences, respectively.⁴²