Architectural design choices made by the programmer for example, the value of a learning rate parameter, the acquisition of the representation of knowledge and state, or a particular wiring of a convolutional neural network determine or influence the kinds of behaviours that the algorithm exhibits. In a more complex AI system, such as a driverless car, the behaviour of the car develops over time, from software development and changing hardware components that engineers incorporate into its overall architecture.
Behaviours can also change as a result of algorithmic upgrades pushed to the machine by its designers after deployment. A human engineer may also shape the behaviour of the machine by exposing it to particular training stimuli. For instance, many image and text classification algorithms are trained to optimize accuracy on a specific set of datasets that were manually labelled by humans. The choice of dataset—and those features it represents 60 , 61 —can substantially influence the behaviour exhibited by the algorithm.
Finally, a machine may acquire behaviours through its own experience. For instance, a reinforcement learning agent trained to maximize long-term profit can learn peculiar short-term trading strategies based on its own past actions and concomitant feedback from the market Similarly, product recommendation algorithms make recommendations based on an endless stream of choices made by customers and update their recommendations accordingly.
In the study of animal behaviour, adaptive value describes how a behaviour contributes to the lifetime reproductive fitness of an animal. The focus on function helps us to understand why some behavioural mechanisms spread and persist while others decline and vanish. Function depends critically on the fit of the behaviour to environment.
In the case of machines, we may talk of how the behaviour fulfils a contemporaneous function for particular human stakeholders. The human environment creates selective forces that may make some machines more common. These dynamics are ultimately driven by the success of institutions—such as corporations, hospitals, municipal governments and universities—that build or use AI.
The most obvious example is provided by algorithmic trading, in which successful automated trading strategies could be copied as their developers move from company to company, or are simply observed and reverse-engineered by rivals. These forces can produce unanticipated effects. For example, objectives such as maximizing engagement on a social media site may lead to so-called filter bubbles 91 , which may increase political polarization or, without careful moderation, could facilitate the spread of fake news.
However, websites that do not optimize for user engagement may not be as successful in comparison with ones that do, or may go out of business altogether. Similarly, in the absence of external regulation, autonomous cars that do not prioritize the safety of their own passengers may be less attractive to consumers, leading to fewer sales Sometimes the function of machine behaviour is to cope with the behaviour of other machines.
Adversarial attacks—synthetic inputs that fool a system into producing an undesired output 44 , 92 , 93 , 94 —on AI systems and the subsequent responses of those who develop AI to these attacks 95 may produce complex predator—prey dynamics that are not easily understood by studying each machine in isolation. These examples highlight how incentives created by external institutions and economic forces can have indirect but substantial effects on the behaviours exhibited by machines Understanding the interaction between these incentives and AI is relevant to the study of machine behaviour.
These market dynamics would, in turn, interact with other processes to produce evolution among machines and algorithms. In the study of animal behaviour, phylogeny describes how a behaviour evolved. In addition to its current function, behaviour is influenced by past selective pressures and previously evolved mechanisms. For example, the human hand evolved from the fin of a bony fish. Its current function is no longer for swimming, but its internal structure is explained by its evolutionary history.
Non-selective forces, such as migration and drift, also have strong roles in explaining relationships among different forms of behaviour. In the case of machines, evolutionary history can also generate path dependence, explaining otherwise puzzling behaviour.
At each step, aspects of the algorithms are reused in new contexts, both constraining future behaviour and making possible additional innovations. For example, early choices about microprocessor design continue to influence modern computing, and traditions in algorithm design—such as neural networks and Bayesian state—space models—build in many assumptions and guide future innovations by making some new algorithms easier to access than others.
As a result, some algorithms may attend to certain features and ignore others because those features were important in early successful applications. Machine behaviour evolves differently from animal behaviour. Most animal inheritance is simple—two parents, one transmission event. Algorithms are much more flexible and they have a designer with an objective in the background.
The human environment strongly influences how algorithms evolve by changing their inheritance system. AI replication behaviour may be facilitated through a culture of open source sharing of software, the details of network architecture or underlying training datasets.
For instance, companies that develop software for driverless cars may share enhanced open source libraries for object detection or path planning as well as the training data that underlie these algorithms to enable safety-enhancing software to spread throughout the industry.
However, other institutions apply limits as well. For example, software patents may impose constraints on the copying of particular behavioural traits. And regulatory constraints—such as privacy protection laws—can prevent machines from accessing, retaining or otherwise using particular information in their decision-making. These peculiarities highlight the fact that machines may exhibit very different evolutionary trajectories, as they are not bound by the mechanisms of organic evolution.
With the framework outlined above and in Fig. Individual machine behaviour emphasizes the study of the algorithm itself, collective machine behaviour emphasizes the study of interactions between machines and hybrid human—machine behaviour emphasizes the study of interactions between machines and humans.
Here we can draw an analogy to the study of a particular species, the study of interactions among members of a species and the interactions of the species with their broader environment. Analyses at any of these scales may address any or all of the questions described in Fig.
AI systems represent the amalgamation of humans, data and algorithms. Each of these domains influences the other in both well-understood and unknown ways. Data—filtered through algorithms created by humans—influences individual and collective machine behaviour. AI systems are trained on the data, in turn influencing how humans generate data.
AI systems collectively interact with and influence one another. Human interactions can be altered by the introduction of these AI systems. Studies of machine behaviour tend to occur at the individual, the collective or the hybrid human—machine scale of inquiry. The study of the behaviour of individual machines focuses on specific intelligent machines by themselves.
Often these studies focus on properties that are intrinsic to the individual machines and that are driven by their source code or design. The fields of machine learning and software engineering currently conduct the majority of these studies. There are two general approaches to the study of individual machine behaviour. The first focuses on profiling the set of behaviours of any specific machine agent using a within-machine approach, comparing the behaviour of a particular machine across different conditions.
The second, a between-machine approach, examines how a variety of individual machine agents behave in the same condition. A within-machine approach to the study of individual machine behaviours investigates questions such as whether there are constants that characterize the within-machine behaviour of any particular AI across a variety of contexts, how the behaviour of a particular AI progresses over time in the same, or different, environments and which environmental factors lead to the expression of particular behaviours by machines.
For instance, an algorithm may only exhibit certain behaviours if trained on particular underlying data 98 , 99 , Fig. Then, the question becomes whether or not an algorithm that scores probability of recidivism in parole decisions 7 would behave in unexpected ways when presented with evaluation data that diverge substantially from its training data. The second approach to the study of individual machine behaviour examines the same behaviours as they vary between machines.
For example, those interested in examining advertising behaviours of intelligent agents 63 , , may investigate a variety of advertising platforms and their underlying algorithms and examine the between-machine effect of performing experiments with the same set of advertising inputs across platforms. The same approach could be used for investigations of dynamic pricing algorithms 23 , 24 , 32 across platforms.
Other between-machine studies might look at the different behaviours used by autonomous vehicles in their overtaking patterns or at the varied foraging behaviours exhibited by search and rescue drones In contrast the study of the behaviour of individual machines, the study of collective machine behaviour focuses on the interactive and system-wide behaviours of collections of machine agents. In some cases, the implications of individual machine behaviour may make little sense until the collective level is considered.
Some investigations of these systems have been inspired by natural collectives, such as swarms of insects, or mobile groups, such as flocking birds or schooling fish. For example, animal groups are known to exhibit both emergent sensing of complex environmental features and effective consensus decision-making In both scenarios, groups exhibit an awareness of the environment that does not exist at the individual level.
Fields such as multi-agent systems and computational game theory provide useful examples of the study of this area of machine behaviour. Robots that use simple algorithms for local interactions between bots can nevertheless produce interesting behaviour once aggregated into large collectives. For example, scholars have examined the swarm-like properties of microrobots that combine into aggregations that resemble swarms found in systems of biological agents , Additional examples include the collective behaviours of algorithms both in the laboratory in the Game of Life as well as in the wild as seen in Wikipedia-editing bots Other examples include the emergence of novel algorithmic languages between communicating intelligent machines as well as the dynamic properties of fully autonomous transportation systems.
Ultimately, many interesting questions in this domain remain to be examined. The vast majority of work on collective animal behaviour and collective robotics has focused on how interactions among simple agents can create higher-order structures and properties. Although important, this neglects that fact that many organisms, and increasingly also AI agents 75 , are sophisticated entities with behaviours and interactions that may not be well-characterized by simplistic representations. Revealing what extra properties emerge when interacting entities are capable of sophisticated cognition remains a key challenge in the biological sciences and may have direct parallels in the study of machine behaviour.
Such social learning does not need be limited to machines learning from machines, but we may expect machines to learn from humans, and vice versa for humans to learn from the behaviour of machines.
In addition, human-made AI systems do not necessarily face the same constraints as do organisms, and collective assemblages of machines provide new capabilities, such as instant global communication, that can lead to entirely new collective behavioural patterns.
Studies in collective machine behaviour examine the properties of assemblages of machines as well as the unexpected properties that can emerge from these complex systems of interactions. For example, some of the most interesting collective behaviour of algorithms has been observed in financial trading environments. These environments operate on tiny time scales, such that algorithmic traders can respond to events and each other ahead of any human trader Under certain conditions, high-frequency capabilities can produce inefficiencies in financial markets 26 , In addition to the unprecedented response speed, the extensive use of machine learning, autonomous operation and ability to deploy at scale are all reasons to believe that the collective behaviour of machine trading may be qualitatively different than that of human traders.
Furthermore, these financial algorithms and trading systems are necessarily trained on certain historic datasets and react to a limited variety of foreseen scenarios, leading to the question of how they will react to situations that are new and unforeseen in their design. Flash crashes are examples of clearly unintended consequences of interacting algorithms , ; leading to the question of whether algorithms could interact to create a larger market crisis.
Humans increasingly interact with machines They mediate our social interactions 39 , shape the news 14 , 17 , 55 , 56 and online information 15 , that we see, and form relationships with us that can alter our social systems. Because of their complexity, these hybrid human—machine systems pose one of the most technically difficult yet simultaneously most important areas of study for machine behaviour. One of the most obvious—but nonetheless vital—domains of the study of machine behaviour concerns the ways in which the introduction of intelligent machines into social systems can alter human beliefs and behaviours.
As in the introduction of automation to industrial processes , intelligent machines can create social problems in the process of improving existing problems. Numerous problems and questions arise during this process, such as whether the matching algorithms that are used for online dating alter the distributional outcomes of the dating process or whether news-filtering algorithms alter the distribution of public opinion.
It is important to investigate whether small errors in algorithms or the data that they use could compound to produce society-wide effects and how intelligent robots in our schools, hospitals and care centres might alter human development and quality of life 54 and potentially affect outcomes for people with disabilities Other questions in this domain relate to the potential for machines to alter the social fabric in more fundamental ways.
For example, questions include to what extent and what ways are governments using machine intelligence to alter the nature of democracy, political accountability and transparency, or civic participation. Other questions include to what degree intelligent machines influence policing, surveillance and warfare, as well as how large of an effect bots have had on the outcomes of elections 56 and whether AI systems that aid in the formation of human social relationships can enable collective action.
Notably, studies in this area also examine how humans perceive the use of machines as decision aids 8 , , human preferences for and against making use of algorithms , and the degree to which human-like machines produce or reduce discomfort in humans 39 , An important question in this area includes how humans respond to the increasing coproduction of economic goods and services in tandem with intelligent machines Ultimately, understanding how human systems can be altered by the introduction of intelligent machines into our lives is a vital component of the study of machine behaviour.
Intelligent machines can alter human behaviour, and humans also create, inform and mould the behaviours of intelligent machines. We shape machine behaviours through the direct engineering of AI systems and through the training of these systems on both active human input and passive observations of human behaviours through the data that we create daily.
The choice of which algorithms to use, what feedback to provide to those algorithms 3 , and on which data to train them are also, at present, human decisions and can directly alter machine behaviours.
An important component in the study of machine behaviour is to understand how these engineering processes alter the resulting behaviours of AI, whether the training data are responsible for a particular behaviour of the machine, whether it is the algorithm itself or whether it is a combination of both algorithm and data. The framework outlined in Fig. Examining how altering the parameters of the engineering process can alter the subsequent behaviours of intelligent machines as they interact with other machines and with humans in natural settings is central to a holistic understanding of machine behaviour.
Although it can be methodologically convenient to separate studies into the ways that humans shape machines and vice versa, most AI systems function in domains where they co-exist with humans in complex hybrid systems 39 , 67 , , Questions of importance to the study of these systems include those that examine the behaviours that characterize human—machine interactions including cooperation, competition and coordination—for example, how human biases combine with AI to alter human emotions or beliefs 14 , 55 , 56 , , , how human tendencies couple with algorithms to facilitate the spread of information 55 , how traffic patterns can be altered in streets populated by large numbers of both driverless and human-driven cars and how trading patterns can be altered by interactions between humans and algorithmic trading agents 29 as well as which factors can facilitate trust and cooperation between humans and machines 88 , Another topic in this area relates to robotic and software-driven automation of human labour Here we see two different types of machine—human interactions.
Another is that machines can replace humans, such as in driverless transportation and package delivery. This leads to questions about whether machines end up doing more of the replacing or the enhancing in the longer run and what human—machine co-behaviours will evolve as a result.
The above examples highlight that many of the questions that relate to hybrid human—machine behaviours must necessarily examine the feedback loops between human influence on machine behaviour and machine influence on human behaviour simultaneously. Scholars have begun to examine human—machine interactions in formal laboratory environments, observing that interactions with simple bots can increase human coordination 39 and that bots can cooperate directly with humans at levels that rival human—human cooperation However, there remains an urgent need to further understand feedback loops in natural settings, in which humans are increasingly using algorithms to make decisions and subsequently informing the training of the same algorithms through those decisions.
Furthermore, across all types of questions in the domain of machine behavioural ecology, there is a need for studies that examine longer-run dynamics of these hybrid systems 53 with particular emphasis on the ways that human social interactions , may be modified by the introduction of intelligent machines Furthering the study of machine behaviour is critical to maximizing the potential benefits of AI for society.
The consequential choices that we make regarding the integration of AI agents into human lives must be made with some understanding of the eventual societal implications of these choices.
To provide this understanding and anticipation, we need a new interdisciplinary field of scientific study: machine behaviour.
For this field to succeed, there are a number of relevant considerations. First, studying machine behaviour does not imply that AI algorithms necessarily have independent agency nor does it imply algorithms should bear moral responsibility for their actions. Nonetheless, it is useful to study the behavioural patterns of animals to predict such aberrant behaviour. Machines operate within a larger socio-technical fabric, and their human stakeholders are ultimately responsible for any harm their deployment might cause.
Second, some commentators might suggest that treating AI systems as agents occludes the focus on the underlying data that such AI systems are trained on.
Indeed, no behaviour is ever fully separable from the environmental data on which that agent is trained or developed; machine behaviour is no exception. As such, scholars of machine behaviour should focus on characterizing agent behaviour across diverse environments, much as behavioural scientists desire to characterize political behaviours across differing demographic and institutional contexts. Third, machines exhibit behaviours that are fundamentally different from animals and humans, so we must avoid excessive anthropomorphism and zoomorphism.
Even if borrowing existing behavioural scientific methods can prove useful for the study of machines, machines may exhibit forms of intelligence and behaviour that are qualitatively different—even alien—from those seen in biological agents. Furthermore, AI scientists can dissect and modify AI systems more easily and more thoroughly than is the case for many living systems. Although parallels exist, the study of AI systems will necessarily differ from the study of living systems.
Fourth, the study of machine behaviour will require cross-disciplinary efforts 82 , and will entail all of the challenges associated with such research , Addressing these challenges is vital Universities and governmental funding agencies can play an important part in the design of large-scale, neutral and trusted cross-disciplinary studies Fifth, the study of machine behaviour will often require experimental intervention to study human—machine interactions in real-world settings , These interventions could alter the overall behaviour of the system, possibly having adverse effects on normal users Ethical considerations such as these need careful oversight and standardized frameworks.
Finally, studying intelligent algorithmic or robotic systems can result in legal and ethical problems for researchers studying machine behaviour. Reverse-engineering algorithms may require violating the terms of service of some platforms; for example, in setting up fake personas or masking true identities.
The creators or maintainers of the systems of interest could embroil researchers in legal challenges if the research damages the reputation of their platforms. Moreover, it remains unclear whether violating terms of service may expose researchers to civil or criminal penalties for example, through the Computer Fraud and Abuse Act in the United States , which may further discourage this type of research Understanding the behaviours and properties of AI agents—and the effects they might have on human systems—is critical.
Society can benefit tremendously from the efficiencies and improved decision-making that can come from these agents. At the same time, these benefits may falter without minimizing the potential pitfalls of the incorporation of AI agents into everyday human life. Simon, H. Milner, R. A modal characterisation of observable machine-behaviour.
In this invited lecture, Robin Milner outlines the idea of studying machine behaviour using formal logic. Thomaz, A. Teachable robots: understanding human teaching behavior to build more effective robot learners. Google Scholar. Stone, P. Artificial Intelligence and Life in This book articulates some of the risks posed by the uncritical use of algorithms in society and provides motivation for the study of machine behaviour.
Future of Life Institute. Dressel, J. The accuracy, fairness, and limits of predicting recidivism. Binns, R. In Proc. Hudson, L. Drone warfare: blowback from the new American way of war. Middle East Policy 18 , — Kahneman, D. Noise: how to overcome the high, hidden cost of inconsistent decision making. Kleinberg, J. Human decisions and machine predictions. Crawford, K. Amodei, D. Concrete problems in AI safety. Bakshy, E. Exposure to ideologically diverse news and opinion on Facebook.
Science , — Bessi, A. Social bots distort the U. Presidential election online discussion. First Monday 21 , 11 Ferrara, E. The rise of social bots. ACM 59 , 96— Lazer, D. The rise of the social algorithm. Tufekci, Z. Engineering the public: big data, surveillance and computational politics. First Monday 19 , 7 Lee, T. A two-stage hybrid credit scoring model using artificial neural networks and multivariate adaptive regression splines. Expert Syst. ADS Google Scholar. Roszbach, K. Bank lending policy, credit scoring, and the survival of loans.
Huang, C. Credit scoring with a data mining approach based on support vector machines. Tsai, C. Using neural network ensembles for bankruptcy prediction and credit scoring.
Chen, L. Observing algorithmic marketplaces in-the-wild. SIGecom Exch. An empirical analysis of algorithmic pricing on Amazon marketplace. Bias in Online freelance marketplaces: evidence from TaskRabbit and Fiverr. Cartlidge, J. Too fast too furious—faster financial-market trading agents can give less efficient markets.
Kearns, M. Empirical limitations on high-frequency trading profitability. Trading 5 , 50—62 Wellman, M. Ethical issues for autonomous trading agents. Minds Mach. Farmer, J. An ecological perspective on the future of computer trading. Finance 13 , — MathSciNet Google Scholar. Perry, W. Bonnefon, J. The social dilemma of autonomous vehicles. Kooti, F. Zeng, X. Design and performance evaluation of voice activated wireless home devices.
IEEE Trans. Hendriks, B. Robot vacuum cleaner personality and behavior. Hitsch, G. Matching and sorting in online dating. Finkel, E. Online dating: a critical analysis from the perspective of psychological science. Public Interest 13 , 3—66 PubMed Google Scholar. Park, H. Growing growth mindset with a social robot peer. Bemelmans, R. Socially assistive robots in elderly care: a systematic review into effects and effectiveness.
Shirado, H. Locally noisy autonomous agents improve global human coordination in network experiments. Nature , — In this human—machine hybrid study, the authors show that simple algorithms injected into human gameplay can improve coordination outcomes among humans. Pichai, S. AI at Google: Our Principles. Roff, H. The strategic robot problem: lethal autonomous weapons in war. Ethics 13 , — Krishnan, A. Voosen, P. The AI detectives.
Science , 22—27 Szegedy, C. Intriguing properties of neural networks. Zhang, Q. Visual interpretability for deep learning: a survey. Electronic Eng. Doshi-Velez, F. Towards a rigorous science of interpretable machine learning. Gebru, T.
Datasheets for datasets. Mitchell, M. Model cards for model reporting. Lakkaraju, H. Identifying unknown unknowns in the open world: representations and policies for guided exploration. Johnson, N. Abrupt rise of new machine ecology beyond human response time. Appel, K. Every planar map is four colorable. Part II: reducibility. Illinois J.
Part I: discharging. Westlund, J. Lorenz, T. Synchrony and reciprocity: key mechanisms for social companion robots in therapy and care. Vosoughi, S. The spread of true and false news online.
This study examines the complex hybrid ecology of bots and humans on Twitter and finds that humans spread false information at higher rates than bots. The science of fake news. Roberts, M. Press, Corbett-Davies, S. Algorithmic decision making and the cost of fairness. Inherent trade-offs in the fair determination of risk scores. Buolamwini, J. Gender shades: intersectional accuracy disparities in commercial gender classification.
Bolukbasi, T. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Advances in Neural Information Processing Systems — Caliskan, A. Semantics derived automatically from language corpora contain human-like biases. Sweeney, L. Discrimination in online ad delivery.
Queueing Syst. Ensign, D. Runaway feedback loops in predictive policing. Angwin, J. Machine bias. Chouldechova, A. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions.
Jennings, N. Human—agent collectives. ACM 57 , 80—88 Campbell, M. Deep blue. Schaeffer, J. Checkers is solved. Silver, D. Mastering the game of Go with deep neural networks and tree search. Mastering the game of Go without human knowledge. Bowling, M. Bellemare, M. The arcade learning environment: an evaluation platform for general agents. Designing the market game for a trading agent competition.
Kitano, H. RoboCup: the robot world cup initiative. Russakovsky, O. Published by Research in Outdoor Education is a peer-reviewed, scholarly journal seeking to support and further outdoor education and its goals, including personal growth and moral development, team building and cooperation, outdoor knowledge New and recent books published in the field of medieval studies by Cornell University Press and its imprints.
View the PDF or the Issuu version. New and recent books published in the field of military history by Cornell University Press and its imprints. New and recent books published in the field of urban studies by Cornell University Press and its imprints. New and recent books published in the field of international studies by Cornell University Press and its imprints.
New and recent books published in the field of Asian studies by Cornell University Press and its imprints. New and recent books published in the field of classics by Cornell University Press and its imprints. New and recent books published in the field of literary studies by Cornell University Press and its imprints. New and recent books published in the field of history by Cornell University Press and its imprints.
Books Authors Posts Showing Results for:. Filter Your Results. Books By subjects History. Nature and Science. Asian Studies. Political Science. Medieval and Renaissance Studies. Russian and Eurasian Studies. US History. Legal Studies. Labor Studies. Cultural Studies. Health and Medicine.
Literature, Literary Criticism, and Poetry. New York State. Urban Studies. Middle East Studies. Cornell Studies in Security Affairs. The United States in the World. Battlegrounds: Cornell Studies in Military History. Medieval Societies, Religions, and Cultures. Expertise: Cultures and Technologies of Knowledge. Cornell Studies in Political Economy. Cornell Series in Environmental Education.
Agora Editions. Cornell Modern Indonesia Collection. The Environments of East Asia.
0コメント