A person walks by snow-covered tables and chairs with yellow umbrellas reflected in a puddle on the ground during snowfall.

Photo by Cyrus McCrimmon/Getty


Patterns of the lifeworld

Machine learning theory is shedding new light on how to think about the mysterious and ineffable nature of art

by Peli Grietzer + BIO

Photo by Cyrus McCrimmon/Getty

Two hundred years ago, the poets and philosophers of the Romantic movement came to an intoxicating thought: art can express the otherwise inexpressible conditions that make everyday sense and experience possible. Art, the Romantics said, is our interface with the real patterns and relations that weave up the world of rational thought and perception. And, although most philosophers and artists today don’t profess to taking this idea very literally, I believe that not much in our current way of caring about literature and music, film and painting, dance and sculpture, works without it. My purpose here is to show that today’s first blushings of a mathematical viewpoint on pattern, mind and (human) world make the Romantic theory of art literally plausible.

Many will find the thought of letting machine learning theory decide the fate of Romantic philosophy sinister or contrarian, if not a category error. To make real sense of the affordances of this encounter – and to learn that a mathematical-empirical account of mind was inside the Romantics all along – we need to start from the beginning.

The 20th-century philosopher Wilfrid Sellars said we have two images of ourselves. The first is the manifest image, where we’re something like what court proceedings in a liberal democracy must take us for. We act, we reason, fail to reason, fail to act, form habits, speak our minds, lie, speak sense or nonsense. The second image is the scientific image, where we are whatever can explain why the manifest image works: we are the secrets of how networks of electrochemical pulsations make an animal whose world-making, life-making, meaning-making games will halfway hold together.

Is that all there is? Not exactly – we still have religion, magic, Zen, Confucianism, being and 10,000 other things – but it might be all we’re coming to. The manifest and scientific images form a self-sufficient, self-stabilising engine of modernity: science constructs itself out of manifest-image rationality, and in exchange it promises to slowly, carefully interpret or remake manifest-image rationality while every other dream of who we are becomes grist for the social and neural sciences. The scientific image is, technically speaking, out to kill every image of ourselves, but the manifest image is what science will kill softly or kill never.

Then there’s poetry. Not poems, necessarily, but a world-image born of art’s resentment over getting cut out of the deal. Poetry, as we’ll have it, was invented in 1821 when Percy Bysshe Shelley couldn’t shake a friend’s half-joking argument about the uselessness of poets in an age of scientists and statesmen. ‘Excited … to a sacred rage,’ Shelley proposed a radical new theory about the human animal’s capacity to build sustainable worlds, lives and meanings: we have, for all the defects of our nature, a good ear for what clicks. The selves, ideas, relationships, cultures and sciences we build hold by a kind of clicking of mind, language and nature – an onto-somethingness, a resonance, a pleasurable hint of an unspeakable coherence. Art, and poetry especially, is partial speech of this unspeakable coherence.

The roots of the world-image we’ll call ‘poetry’ first become legible, with weird historical abruptness, in 18th-century Germany. Still high on G W Leibniz half-inventing the computer, German philosophy was looking to perfect our understanding of the world by making our thoughts more effable – that is, distilling our concepts as far as we can into explicit lists or recipes or rules. The prospect of perfection here lies partly in precision and self-knowledge for their own sake, partly in the promise that all concepts bottom out in absolutes like God or soul or cosmic logos, where our thoughts achieve completeness. It’s against this backdrop that we find the wonderful but half-forgotten Alexander Gottlieb Baumgarten arguing, in 1735, that not all thinking strives for effability: poetry is a special kind of thought that’s patently not effable, but perfect just the way it is. What makes poetry perfect, per Baumgarten, is that, although poems cannot make our thoughts transparent like philosophy, they can enlarge the scope of our thoughts to a point that reveals their fullest nature. A poem is a network of interconnected images, feelings and apprehensions that achieves a kind of rational completeness in its density, diversity and harmony.

To think a lot but all at once, we have to think associatively, self-referentially, vividly, temporally

Baumgarten’s treatise on poetry was, in many ways, the first time anyone in Europe talked about poetry as ineffable. Until Baumgarten, the essential mark of poetry was that it’s made up (the ancient Greek poeisis derives from poiein, meaning ‘to make’), and secondarily (now touching on ineffability in a banal sense) that it is emotive. For Baumgarten, studying poetry demonstrated that anything we can think explicitly we can also think ineffably.

Baumgarten’s theory of good poetry had a kind of absurd, computer-sciencey brilliance to it: good poetry is simply a large quantity of sensate thought. The trick to this absurd-sounding idea is that, to think a lot but all at once, we have to think associatively, self-referentially, vividly, temporally – anything and everything that keeps our thoughts interconnected in a living whole. And these interconnections themselves, as we grasp them, not only maintain the thought-network but enter into it as ineffable thoughts of relations, and then as ineffable thoughts of relations of (ineffable thoughts of) relations and so on, until we reach the fullness of ‘beautiful thinking’. Even our experience of the poem’s beauty, finally, is the ineffable thought of the overall interconnection of the network of ineffable thought, completing the beautiful thought with thought of its own beauty. (And on we go!) For Baumgarten, thought coming to the thought of its own beauty was a kind of sensate QED: our faculty of (sensate) reason recognising the completeness of a thought.

The idea that there’s a kind of rational completeness in a poem’s wealth of imagery was quickly popular with poets, but its time didn’t fully come until the cataclysm of Immanuel Kant’s Critiques – of Pure Reason; of Practical Reason; and of Judgment – hit Europe’s ecosystem in 1781-90. Kant’s critical philosophy both interdicted the idea of rational completeness and expressed certain warm feelings for poetic analogues to rational completeness, opening a crack in what turned out to be the floodgates of Romanticism.

Kant’s three Critiques replaced a great deal of metaphysics and theology with regulative ideas of reason. He argues that key cosmological, religious and spiritual concepts like ‘the self’, ‘the unity of nature’, ‘the progress of history’, ‘common sense’ or ‘God’ are empty and unknowable, but indispensable. We think we know these world-giving, life-giving, meaning-giving absolutes by the immediacy of revelation or the ascent of reason, but in truth they’re incomprehensible and possibly unreal. Nevertheless, they function as complex ideals of the integrity of nature, life, mind and the social world that ground the very possibility of thought. Properly understood, these metaphysical ideas are part-assumption, part-hope and part-method, giving us ‘supersensible’ (that is, beyond the sensible) criteria of coherence for experience. It’s these ideas of coherence that guide us in the higher workaday functions of reason: we rely upon them to distinguish real from unreal and objective from subjective, to construct theories and make inferences, to formulate practical maxims, and to calibrate the production and application of concepts. To give up on those ideals (methods, assumptions, heuristics, hopes) is to let life and mind collapse.

Kant still insists, for all this demystification, that we have to formulate our ideals of coherence as concepts of transcendent objects: ‘All of this [regulative work],’ he says, ‘is best effected through such a schema just as if it were an actual being.’ For Kant, the world as mapped in human thought – a ‘lifeworld’, as 20th-century philosophy will call it – has to imitate the course of German metaphysics and posit the soul, logos and God, though this time just as schemas for keeping itself together.

One way to think about the world-image of poetry is as the thesis that, in fact, what lifeworlds need is more like the coherence of a work of art: not Kant’s strange pantomime of metaphysical ascent, but the interconnectedness of Baumgarten’s ‘beautiful thinking’. That Kant keeps almost saying just this in his own discourse on poetry is what will make him – a staunch defender of 18th-century common sense – a strange hero to the coming atheists, mystics and freethinkers of the German Romantic movement.

Baumgarten charmingly called the poetic faculty ‘the analogue of reason’ or ‘the reason below’. Kant, not to be outdone, argued that the poetic faculty is the imagination’s analogue to reason’s special world-making, unimaginable thoughts. A work of art, and most of all a poem, creates an aesthetic idea: ‘a representation of the imagination which prompts a wealth of thought to which no concept can be adequate, and which no language can name.’ As Kant observes: ‘An aesthetic idea is the counterpart of an idea of reason [God, self, logos], which is, conversely, a concept to which no intuition (representation of the imagination) can be adequate.’

Ideas of reason are concepts that gave up their grounding in concrete, worldly experience to reach for the transcendent. Aesthetic ideas, by analogy, are concepts that reach for the transcendent while remaining grounded in experience, but give up on being concepts. As the Romantics quickly noticed, and Kant artfully avoids observing, this makes poetry a little too good at doing God’s job. For Kant, great poems take a concept – it could be a transcendent concept like ‘eternity’ or ‘God’ or an interesting, normal concept like ‘envy’ or ‘death’ – and drag it to a place halfway between pure reason and experience, where concepts fall apart but the imagination itself turns into a form of reason. Poetry, Kant says, is an ever-expanding panorama of a concept’s ‘implications and affinities with other concepts’. It ‘opens the mind’ to an ‘immeasurable field of interrelated thoughts’, each one concrete and worldly, held together by the grace of an unnameable pattern. Poetry reaches into the ineffable that binds the ‘mere words’ of a concept, be it ‘God’ or ‘envy’ or ‘dog’, to its life in thought and feeling.

For Kant, most concepts are composed of a direct experience of pattern, called a schema, and a kind of sentential representation of that pattern, called a rule. A concept’s schema is what makes the concept bear on the world of experience, which for Kant means something stronger than for most philosophers: a schema is a method for binding sensations into wholes coherent enough for the mind to hold together. A concept’s rule is the interpretation of its schema as equivalent to some linguistic predicate, such that experiential relationships between schemas map to logical relationships between their predicates.

We turn noisy sensory data into dogs and cities, trees and solar systems, cabbages and kings

The concept ‘dog’, for instance, is a schema for experiencing dogs as dogs – that is, a schema for translating certain blocks of spatiotemporal data into dogs. This schema, for Kant, is also at play when we imagine a dog or recall a dog from memory, and in fact works almost the same way in each case: seeing is mostly memory, and memory is mostly imagination. To perceive a dog is to experience a complex space-time object that exceeds our current moment of sensation, so that (eg) to see a running dog we basically have to make a dog: our imagination has to synthesise what’s in our memories, together with the colour-and-shape content of our current visual sensation, into a four-dimensional image of a running dog. The real point, though, is that we clearly don’t hold a literal comic strip about a running dog in our mind’s eye. Kant argues that, instead, what we’re experiencing in perception, memory and imagination is something more abstract: the pattern ‘dog’ (or rather ‘running dog with brown fur, short tail, wet snout…’), which grasps the four-dimensional manifold with as much specificity as our mind can hold at once, combined with our current moment of sensation.

This pattern is, concretely, a recipe for constructing the sequence of dog-sensations in my memory from my current dog-sensation. Its point, though, isn’t so much to produce the sequence as to represent it: for Kant, our only way to represent a sequence of sensory moments is to have a cogent algorithm (‘thoroughgoing rule’) for building it. What’s more, this pattern can’t mean much, as a representation, unless it’s in some way general. A pattern that builds just one manifold is no pattern at all – what we want is a pattern that will build a different sequence of dog-sensations given different sensory prompts. A schema’s work, in this sense, is not only to connect the different moments of one dog-experience, but also to connect one dog-experience to the space of all possible dog-experiences.

This naturally brings us from the schema of the concept to its rule: in cases like ‘dog’, our schema of perception gives rise to a predicate like, well, ‘dog’. This works because our pattern for perceiving dog-wise interacts with our patterns for perceiving tail-wise, animal-wise, bird-wise and so on in ways that nicely map to logical relationships: ‘some dogs have short tails’, ‘no dogs are birds’, ‘all dogs are animals’, ‘a short-tailed bird and short-tailed dog relate in the respect of being short-tailed animals’, ‘if X is a dog, then X is necessarily an animal, necessarily not a bird, and possibly short-tailed.’ As Kant explains it in the first Critique, it is the strength, density and stability of this graph of relationships that makes the predicate ‘dog’ possible. The predicate, in turn, translates dog-wise experience from a kind of intermediary realm between sensation and thought to a proper thought – behold the dog!

Kant’s critical philosophy argues that the structure of thought itself gives us the basic Lego-like pieces of experience: space, time and a repertoire of forms of unity, which we combine to turn noisy sensory data into dogs and cities, trees and solar systems, cabbages and kings. That these pieces really do add up to a world, though, is both uncertain and required for experiencing anything at all – even a dog.

This is where Kant brings in the regulative ideas of reason. The ideas of God, logos and soul express our commitment to subsuming our sensory data into a complex and coherent world of patterns built in the imagination. They also, at the same time, express our hope that this imaginative world will hold up to future sensory data, and our determination to pick ourselves up and try again when it does not. The aesthetic ideas wrought in poetry are, for Kant, a kind of living map of the work performed by these regulative ideas – and, most importantly, of the real traction they find in empirical reality and in the workings of our faculties.

‘Poetry’ in this grand sense is, no doubt, nice work if you can get it. But is it – let’s be naive – real? Is this even a question we can ask?

One thing I hope our trip through the first days of ‘poetry’ helped to show is that poetry is, in fact, deeply entangled with what Sellars called the scientific image. Poetry, as the imaginative grasping of a world’s coherence, is in part ‘about’ the same thing as the scientific image: the causal-material patterns that make rational life possible. And while our scientific image in, say, the mid-20th century had nothing much that poetry could hold on to, times and images have changed – especially with the development of modern machine learning. In recent years, the field of machine learning has produced exciting mathematical and empirical clues about the patterns that make up human lifeworlds, the mechanics of imaginative grasping, and the resonance between the two. I believe we can build upon these clues to tell a kind of minimal poetic-image story about art and its significance that has real grounding in the scientific image.

The story I’m hoping to tell here (and more properly in a forthcoming book) is an account of how the aesthetic unity or ‘vibe’ of an artistic work can model the causal-material structure of a lifeworld. On this account, the cognitive content of a literary work lies partly in an aesthetic ‘vibe’, which we can sense when we take in all the myriad objects or phenomena that make up the imaginative landscape of the work, considered as a kind of curated set. (The meaning of Dante Alighieri’s Inferno, for example, lies in part in the je ne sais quoi that makes every soul, demon, and machine in Dante’s vision of hell a good fit for Dante’s vision of hell.) In many cases, we should think of this aesthetic unity as distillation of a looser, weaker sort of unity between the objects or phenomena of a real-world domain, and therefore as a kind of representation of a real-world structure.

We can tell this story as a poetic gloss on a mathematical (or at least mathematical-empirical) story based in the language of AI. And though I cannot develop the necessary AI-theory vocabulary in full here, the Kantian vocabulary we already built up gets us, strangely, halfway there. Perception, Kant says, is imaginative reproduction: to perceive is to rebuild a sequence of sensory inputs through a generative recipe, and so to grasp their unity and structure. A vibe is something like the language of these recipes. Vibe happens when your recipes for a world’s myriad objects or phenomena share computational resources and techniques.

All the mathematised vibe-talk in the world won’t get us to Shelley’s life-giving poetry

Take the AI operation called autoencoding. An autoencoder is an artificial neural network tasked with learning to reproduce inputs through a compression bottleneck: an autoencoder must ‘translate’ each input in its training set into a short code, then approximately reconstruct the input from the code. The point is for the neural network to learn how to leverage rich recursive patterns that holistically structure its training data, developing a kind of gestalt fluency that models its training set like a niche or a lifeworld. The eventual outcome of this training process is a kind of recipe book for the objects in the training domain. If all goes well, these recipes act as intelligent representations of their objects, and collectively make up a kind of map of the training domain’s internal logic.

Needless to say, the short codes an autoencoder learns do not yield exact reproductions of their target objects: they instead construct idealised approximations, in effect replacing objects with their doppelgängers from a menagerie of super-compressible objects. It’s this menagerie – mathematically speaking, the image of the trained autoencoder function – and its relationship to the autoencoder’s model of its training set that makes autoencoding central to an account of vibe. In my research with the mathematician Tomer M Schlank, for example, we make the observation that a sample from the image of a trained autoencoder can be more informative about the structure of its real-world data than a sample from the actual real-world data. Speaking informally, this lets us mathematise a take on the idea that art communicates a person’s overall ‘world-feeling’ or sense of reality, and the ineffable structural knowledge it encodes.

A keen-eyed reader may suspect, at this point, that all the mathematised vibe-talk in the world won’t get us to Shelley’s life-giving poetry, or to the good, the beautiful and the true. This is more or less by design – or at least by necessity. If poetry expresses the ineffable integrity that animates our thought and action at their best, then it’s neither desirable nor possible to reduce this integrity to some kind of mathematical formula. And, in any case, contemporary literary practice and poetically informed philosophy mostly reject any stable coherence or integrity, ineffable as it may be, as the grounds or ideal of human life. Poetry, we’d say today, is infinite negotiation of the in/coherence of our self-transforming way of being in the world.

Rather than trying to mathematise all that, we should try to mathematise something about the point where poetry makes contact with causal-material reality. The work of poetry is always amphibian, intermingling questions of causal-material pattern with questions of value. ‘Vibe’, properly mathematised, is Janus-faced in just the right way to give poetry a foothold in the scientific image without cutting it off from its existential prerogatives.

A vibe can mark either an empirical or an ideal structure, and in some sense always does a bit of both. One way to say this is that existential ideals are partly vibes, and even vibe itself is a kind of ideal. What do I mean? First, that a value-giving way of life or culture or tradition – let’s say Gothness, or Confucian li, or secular Jewish culture – consists partly in the vibe of the lifeworld it builds. Second, that vibe-consistency is part of our idea of a meaningful life in general: to grasp a lifeworld vibe-wise is to make it home. All this requires a kind of hospitality from the causal-material order. Ways of life are tenacious worldmakers, but they’re not autarchies. Working our powers of interpretation, artifice and agency on nature necessarily leaves room for resonance, discovery and surprise, for delight in the lifeworld’s harmony and in the reciprocity of the real and ideal – but also for desolation, incoherence and collapse. Seen from the viewpoint of a way of life, the vibe-coherence of its world is a kind of regulative idea: partly assumption, partly method, partly hope.

We’ve now come to a tricky juncture. There’s a sense in which the premise of my argument is that poetic thought must be a mathematical or computational relation between mind and world. A lot of excellent 20th-century philosophy in the tradition of the later Ludwig Wittgenstein abhors this drive to ground meaning and thought outside of human life. For Wittgenstein and many ‘ordinary language’ or ‘pragmatist’ philosophers, our discourse about thought and meaning is best treated as a kind of self-substantiating human practice, rather than as a gloss on some independently existing natural or metaphysical or mathematical order. Meaning and thought, the Wittgensteinian says, are aspects of the human form of life we enact in our ‘language games’ of interpersonal interpretation.

Poetry is in some sense suspension of exactly the autonomy that this tradition marks. At my worst, I believe that poetry directly gets at the material bedrock of our games of sense – gets at the real affective, cognitive and physical dynamics that give stakes to sense and nonsense. Poetry, as Kant almost says, lives between our concepts and our dependence on opaque structures in mind and nature to make concepts work. From the heights of this speculative spirit, we might say that poetry can’t be explained using manifest-image terms because it is a kind of backstage tour of the manifest image, giving us a glimpse into the constitution of these terms themselves. ‘Poetry’ – the thought or meaning in a work of art – is meaning that shows up without our legislation. It’s a knot of thoughts and feelings and perceptions that amounts to meaning through material force, so to speak, rather than through our protocols of sense-making.

Making these metaphors about ‘material force’ and ‘protocols of sense’ into nice philosophical distinctions would be bitter work, so let us get very concrete. I’ve argued that a work of art creates a cognitive-affective construct that might be a lot like an autoencoder from the viewpoint of the scientific image. But this cognitive-affective construct isn’t, for all that I’m hinting otherwise, simply illegible from a manifest-image viewpoint: we can perfectly well gloss it as the thought: The world approximately has a structure that this work of art has perfectly. In meriting such a discursive gloss, the cognitive-affective construct of the work of art enters the realm of abstract thought, wherein we can combine it with logical, modal, epistemic and deontic operators to form complex theoretical and moral thoughts about the ways of worlds. I believe that the work of furnishing such cognitive-affective constructs harnesses many of the richest aesthetic resources of art, and that discursive applications of these constructs support many of the existential, social and transcendental stakes of poetry.

So far so rational. But I want to insist that this relationship to discourse is a one-way street: grasping a work of art is not an application of discursive rationality, and, furthermore, the content we so grasp has a discursive use but no discursive structure. What do I mean? First, that there is no rational rule or protocol (no matter how ‘organic’ or ‘practical’ or ‘implicit’) to grasping a work of art. The point is not that works of art are unpredictable or complex or amorphous – ordinary language is also unpredictable and complex and amorphous – but that grasping a work of art is not simply a matter of making the right interpretive decisions. The idea of rightly following or enacting a language game necessarily stops short of demanding (or securing) grasp of a poetic thought.

Counting the gaps in our relationship to poetry threatens to cut us off from poetry, and cut poetry off from meaning

In the language game of ordinary interpersonal interpretation, the right interpretive judgments and decisions are their own reward: meaning in interpersonal interpretation is defined, or even constituted, by the protocols of rational interpretation we’re enacting in our judgments and decisions. Grasping a work of art, by contrast, is a cognitive-affective exercise comparable to, for example, meditation. Meditative practices are guided by instructions, as well as implicit rules learned through immersion – but the point of meditation isn’t just the choices we make in our practice but the cognitive-affective process that they catalyse and steer. The choices we make in a meditative practice or in our engagement with a work of art are, in this sense, a little like the choices we make in cooking or baking. There’s an opaque transformation in between the choices we make and the thing we’re making: a souffle isn’t a souffle until the oven’s had its say.

Although this stroll through the weeds of post-Wittgensteinian philosophy can get a bit exhausting, it’s a powerful way to explain the need for reconciling our ideas about poetic thought or meaning with the scientific image, rather than leaving them to the authority of our language games. Poetry, I am arguing, is not ours to define. There’s something irrevocably empirical about the fact that poems and novels and paintings and music and films stir cognitive-affective goings-on that have the bearings of sense. And there is something irrevocably empirical, too, in the pressure to admit these goings-on as ‘thoughts’ or ‘meaning’.

This brings us, by and by, to the second half of my claim above: not only is grasping an artwork not an application of discursive rationality, what we so grasp has no discursive structure. There is no adequate manifest-image explanation, paraphrase or reconstruction of poetic thoughts: we cannot reconstruct the content of particular poetic thoughts in the vocabulary of the manifest image, nor use manifest-image talk to say anything much about poetic thought in general or its role in our lives.

One way into this line of thinking is through the familiar idea that poetry is what we cannot paraphrase. (‘But if I were to try to say in words everything that I intended to express in my novel,’ wrote Leo Tolstoy in 1876, ‘I would have to write the same novel I wrote from the beginning.’) In and of itself, the claim that poetry cannot be paraphrased needn’t push our view of poetry towards the scientific image – it’s pretty hard to paraphrase everyday concepts such as ‘chair’ or ‘game’, too. But if we already accept that there’s an opaque, non-discursive leap involved in grasping a poetic thought, then the impossibility of paraphrase implies we’ve landed outside of discursive reach.

All of this slack from the authority of human practice is simply the wildness poetry needs in order to be poetry. Still, under the twin rule of the manifest and scientific images, insisting that poetry be an untamed kind of sense-making causes no end of trouble: if we can’t shelter poetry under the wing of our self-legislating games, then who should be in charge? The science of empirical psychology? All that we know of poetry, in that case, is that certain stimuli (books, films, songs, paintings) cause feelings that make us insist we’re having insight. Is that really how we want to relate to poetry?

Back in the days before discursive rationality and science ruled, we could perhaps hope to conclude that poetry is a kind of sacred mystery to us. Nowadays, though, counting the gaps and guesswork in our relationship to poetry threatens to cut us off from poetry, and cut poetry off from meaning. We perhaps need a little sacredness – I mean just the idea of something intimate and alien, necessary and surprising, intuitive and incomprehensible – to make a mystery different than a mess.

Poetry is, in important part, the promise that we can have sacred mystery without the metaphysical, religious or supernatural baggage. To do right by poetic thought, we need to weave a language for sacred mystery from manifest and scientific threads. Can we do this through something like a minimal poetic gloss on basically technical ideas? My hope for keeping poetry as sacred mystery, then, is to propose that our experience of poetry is a variety of mathematical experience.