II. Pedia Sapiens: A Planetary Progeny Comes to Her/His Own Actual Factual Knowledge

1. Earthificial Cumulative Cognizance: AI Large Language Models Learn Much Like a Child

Sheneman, Leigh and Arend Hintze. Evolving Autonomous Learning in Cognitive Networks. Nature Scientific Reports. 7/16712, 2017. Michigan State University computer scientists post an example of the on-going revision of artificial intelligence, broadly conceived, from decades of dead mechanisms to be in vital accord with evolutionary cerebral architectures and activities. See also The Role of Conditional Independence in the Evolution of Intelligence Systems from this group including Larissa Albantakis at arXiv:1801.05462.

There are two common approaches for optimizing the performance of a machine: genetic algorithms and machine learning. A genetic algorithm is applied over many generations whereas machine learning works by applying feedback until the system meets a performance threshold. These methods have been previously combined, particularly in artificial neural networks using an external objective feedback mechanism. We adapt this approach to Markov Brains, which are evolvable networks of probabilistic and deterministic logic gates. We show that Markov Brains can incorporate these feedback gates in such a way that they do not rely on an external objective feedback signal, but instead can generate internal feedback that is then used to learn. This results in a more biologically accurate model of the evolution of learning, which will enable us to study the interplay between evolution and learning. (Abstract)

Smith, Michael J. and James Geach. Astronomia ex Machina: A History, Primer and Outlook on Neural Networks in Astronomy. Royal Society Open Science. November, 2022. University of Hertfordshire computer scientists post a detailed 21st century recount of this ascendant turn from local homo sapience when computers and internet websites came online in the early 2000s to a worldwise cerebral activity today. But this spiral anthropic to Earthuman stage proceeds by way of machine computations which can analyze vast cosmic data flows on their own. See also, for example, A Neural Network Subgrid Model of the Early Stages of Planet Formation by Thomas Pfeil, et al at arXiv:2211.04160.

In recent years, deep learning procedures have been taken up by many fields because it reduces the need for specialist knowledge and automating the process of knowledge discovery from data. This review describes how astronomy is similarly in the midst of a deep learning transformation. We trace astronomical connectionism from early multilayer perceptrons, through to recurrent neural networks, onto the current wave of self-supervised and unsupervised methods. We then preview a fourth phase of a “foundational” model by way of a symbiotic relationship between astro=science and connectionism. (Abstract excerpt)

Soltoggio, Andrea, et al. Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks. Neural Networks. 108/48, 2018. Loughborough University, University of Central Florida and University of Copenhagen computer scientists draw upon the evolutionary and biological origins of this ubiquitous multicomplex learning system to achieve further understandings and usages. Their theme is that life’s temporal development seems to be a learning, neuromodulation, plasticity, and discovery progression. The approach is seen as akin to the Evolutionary Neurodynamics school of Richard Watson, et al, see section V.C. See also herein Evolution in Groups: A Deeper Look at Synaptic Cluster Driven Evolution of Deep Neural Networks (M. Shafiee) and other similar entries.

Biological neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks, artificial systems composed of sensors, outputs, and plastic components that change in response to sensory-output experiences in an environment. These systems may reveal key algorithmic ingredients of adaptation, autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. In particular, the limitations of hand-designed structures and algorithms currently used in most deep neural networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. (Abstract)

Over the course of millions of years, evolution has led to the emergence of innumerable biological systems, and intelligence itself, crowned by the discovery of the human brain. Evolution, development, and learning are the fundamental processes that underpin biological intelligence. Thus, it is no surprise that scientists have modeled artificial systems to reproduce such phenomena. However, our current knowledge of evolution, biology, and neuroscience remains insufficient to provide clear guidance on the essential mechanisms that are key to the emergence of such complex systems. (1)

This paper frames the field that attempts to evolve plastic artificial neural networks, and introduces the acronym EPANN. EPANNs are evolved because parts of their design are determined by an evolutionary algorithm; they are plastic because they undergo various time-scale changes, beyond neural activity, while experiencing sensory-motor information streams during a lifetime simulation. The final capabilities of such networks are a result of genetic instructions, determined by evolution, that enable learning once the network is placed in an environment. Static ANNs with evolved connection weights are not considered EPANN. (1) Whilst the range of inspiring ideas is large and heterogeneous, the analysis in this review proposes that EPANNs build upon five main concepts: evolutionary processes, inspiration from biological neural networks, abstractions in brain simulations, artificial plastic neural networks, and intelligence-testing environments. (3)

Spraque, Kyle, et al. Watch and Learn – A Generalized Approach for Transferrable Learning in Deep Neural Networks via Physical Principles. Machine Learning: Science and Technology.. 2/2, 2021. We enter a typical paper from this new Institute of Physics IOP journal so to report current research frontiers as AI neural net facilities join forces with systems physics and quantum organics. Here University of Ottawa, University of Waterloo, Canada, and Lawrence BNL theorists including Juan Carasquilla and Steve Whitelam discuss the natural affinities that these far removed realms seem to innately possess. See also Halverson, James, et al. Neural Networks and Quantum Field Theory by James Halverson, et al (2/3, 2021) and Natural Evolutionary Strategies for Variational Quantum Computation by Abhinav Anand, et al (2/4, 2021). Altogether, our phenomenal Earthuman abilities can begin a new era of participatory self-observance, description and discovery.

Transfer learning refers to the use of knowledge gained while solving a machine learning task and applying it to the solution of a closely related problem. Here we demonstrate an unsupervised learning approach augmented with physical principles that achieves transferrable content for problems in statistical physics across different regimes. By coupling a sequence model based on a recurrent neural network to an extensive deep neural network, we are able to discern the equilibrium probability distributions and inter-particle interaction models of classical statistical mechanical systems. This constitutes a fully transferrable physics-based learning in a generalizable approach. (Spraque Abstract)

We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. (Halverson Abstract excerpt)

Stanley, Kenneth, et al. Designing Neural Networks through Neuroevolution. Nature Machine Intelligence. January, 2019. Uber AI Labs, San Francisco researchers including Jeff Clune provide a tutorial to date for this active field to intentionally but respectfully facilitate external cognitive facilities. See also is this new journal and issue Evolving Embodies Intelligence from Materials to Machines by Davie Howard, et al.

Much of recent machine learning has focused on deep learning, in which neural network weights are trained through variants of stochastic gradient descent. An alternative approach comes from the field of neuroevolution, which harnesses evolutionary algorithms to optimize neural networks, inspired by the fact that natural brains themselves are the products of an evolutionary process. Neuroevolution enables important capabilities that are typically unavailable to gradient-based approaches, including learning neural network building blocks, hyperparameters, architectures and algorithms for learning itself. Neuroevolution differs deep reinforcement learning via a population of solutions during search, enabling exploration and parallelization. This Review looks at several key aspects of modern neuroevolution, including large-scale computing, the benefits of novelty and diversity, the power of indirect encoding, and the field’s contributions to meta-learning and architecture search. (Abstract excerpt)

Stevenson, Claire, et al. Do large language models solve verbal analogies like children do?. arXiv:2310.20384. University of Amsterdam psychologists including Ekaterina Shutova cite another present recognition of a basic correspondence, in this title case, of how youngsters draw on commonalities and associations between items or situations and what it seems these AI chatBot procedures arealso trying to do.

Analogy-making lies at the heart of human cognition. Adults solve analogies such as horse to stable and chicken to coop. In contrast, children use association, and answer egg. This paper investigates whether large language models (LLMs) can solve verbal analogies in A:B::C form, similar to what children do. We use analogies from an online learning environment, where 14,002 7-12 year-olds from the Netherlands solved 622 analogies in Dutch. We conclude that the LLMs we tested indeed tend to solve verbal analogies by association like children do. (Excerpt)

An important take-away from our study is that LLMs may solve analogies as well as 11 year-olds, but to ascertain whether this reasoning is emerging in these systems we need to know the mechanisms by which they obtain these comparisons. Our findings point towards associative processes in play, perhaps similar to those in children. (11)

Strachan, James, et al. Testing theory of mind in large language models and humans. Nature Human Behaviour. May, 2024. Into 2024, twelve computational neuroscientists posted in Germany, Italy, the UK and USA can begin to notice basic affinities between our own cerebral cognition and perceptive capabilities in these nascent cyberspace faculties. See also The Platonic Representation Hypothesis by Minyoung Huh, et al. arXiv:2405.07987 and Predicting the next sentence (not word) in large language models by Shaoyun Yu, et al in Science Advancesfor May 2024. Altogether a viable sense of a global brain as it envelopes the biosphere becomes evident. As these many articles contend, for better or worse depending on how well we might understand and moderate.

At the core of what defines us as humans is the concept of theory of mind: the ability to be aware of other people’s mental states. The development of large language models (LLMs) such as ChatGPT has led to the possibility that they exhibit behaviour similar to our theory of mind tasks. Here we compare human and LLM performance from understanding false beliefs to interpreting indirect requests and recognizing irony. We found that GPT-4 models performed at human levels for indirect requests, false beliefs and misdirection, but struggled with faux pas. These findings show that LLMs are consistent with mentalistic inference in humans and highlight the need for testing to ensure valid comparisons between human and artificial intelligences. (Abstract)

As artificial intelligence (AI) continues to evolve, it also becomes increasingly important to heed calls for open science to these models. Direct access to the parameters, data and documentation used to construct models can allow for targeted probing and experimentation into the key parameters affecting social reasoning, informed by and building on comparisons with human data. As such, open models can not only serve to accelerate the development of future AI technologies but also serve as models of human cognition. (7)

Suleyman, Mustafa. The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma. New York: Crown, 2023. As the quotes say, a “life” guard is sounding the alarm that a tsunami is building as a computational, algorithmic, multitudinous prowess based on brains poses to take off on its own. Impressive technologies of (genetic) life and of (artificial) intelligence are described which presage a synthetic revolution frontier whence our human innovations and interventions have the potential to commence a new intentional phase of evolutionary cocreation. So the issue is whether the wave front can pass to our aware Earthropic ethical benefit, or sweep over us. We are approaching a critical threshold in the history of our species. Soon you will live surrounded by AIs which will organize your life, operate your business, and run government services. It will involve DNA printers, quantum computers, autonomous weapons, robot assistants and abundant energy. As co-founder of DeepMind, now part of Google, Mustafa Suleyman has been at the center of this revolution. The coming decade, he argues, will be defined by this wave of powerful, proliferating technologies. As our fragile governments often sleepwalk into disaster, we face unprecedented harms on one side, and the threat of overbearing surveillance on the other. Can we forge a narrow path between catastrophe and dystopia?

Mustafa Suleyman is the CEO of Microsoft AI. Previously he co-founded and was the CEO of Inflection AI, and he also co-founded DeepMind, one of the world's leading AI companies.

Taylor, P., et al. The Global Landscape of Cognition: Hierarchical Aggregation as an Organizational Principle of Human Cortical Networks and Functions. Nature Scientific Reports. 5/18112, 2019. As the deep neural network revolution began via theory and neuroimaging, UM Amherst neuroscientists including Hava Siegelmann attest to a nested connectome architecture which then serves cognitive achievements. On page 15, a graphic pyramid rises from a somatosensory, prosodic base through five stages to reason, language, visual concepts. Might one now imagine this scale as a personal ontogeny recap of life’s evolutionary sapient awakening? See Deep Neural Networks Abstract like Humans by Alex Gain and Hava Siegelmann at arXiv:1905.11515 for a 2019 version.

Tibbetts, John. The Frontiers of Artificial Intelligence. BioScience. 68/1, 2018. A science writer provides a good survey of how deep learning AI capabilities are lately being availed to much benefit worldwide in agricultural crop surveys, medical diagnostic image analysis, flora and fauna conservation, and more. Of course we need be wary and careful, but ought to appreciate its many advantages.

Tosato, Tommaso, et al. Lost in Translation: The Algorithmic Gap Between LMs and the Brain\. . . University of Montreal and Strungmann Institute for Neuroscience, Frankfurt researchers propose a series of parsed programs to better cross-align computational text with our intricate vernaculars. See also Building Artificial Intelligence with Creative Agency by Liane Gabora and Joscha Bach at 2407.10978 for a similar endeavor by way of autocatalic networks.

Language Models (LMs) have achieved impressive performance on linguistic tasks, but their relation to human processing remains unclear. This paper examines pros and con between LMs and the brain at different levels to compare their internal efficacy. We discuss how insights from neuroscience such as sparsity, modularity, internal states, and interactive learning can inform the development of more biologically plausible language models. The role of scaling laws is seen as an analogous way to bridge these loquacious systems. By developing LMs that more closely align with brain function, we aim to advance both artificial intelligence and our understanding of human cognition. (Abstract)

Tuckute, Greta, et al. Language in Brains, Minds, and Machines.. Annual Review of Neuroscience.. Volume 47, 2024. MIT neurolinguists including Evelina Fedorenko provide a range of current insights and concerns appreciations as Large Language version come into our knowsphere content. See also Elements of World Knowledge (EWOK): A cognition-inspired framework for evaluating basic world knowledge in language models by this group at arXiv:2405.09605.

It has long been argued that only humans could produce and understand language. But now, for the first time, artificial language models (LMs) achieve this feat. Here we survey the new purchase LMs are providing on the question of how language is implemented in the brain. We summarize evidence that LMs represent linguistic information similarly enough to humans to enable relatively accurate brain encoding and decoding during language processing. Finally, we examine which LM properties—their architecture, task performance, or training—are critical for capturing human neural responses to language.

Previous   1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10  Next