Hermann Hesse published his novel Das Glasperlenspiel (1943) in Zurich1 and went on to receive the Nobel Prize in Literature in 1946. Hesse commented on the novel’s substance, “It (the glass bead game) represented a symbolic form of seeking for perfection, an approach to that Mind which beyond all images and multiplicities is one within itself—in other words, to God.”2 There is no precise description of this glass bead game, but one passage describes to some degree what the game is about:
Under the shifting hegemony of now this, now that science or art, the game of games had developed into a kind of universal language through which the players could express values and set these in relation to one another. Throughout its history the game was closely allied with music, and usually proceeded according to musical or mathematical rules. One theme, two themes, or three themes were stated, elaborated, varied, and underwent a development quite similar to that of the theme in a Bach fugue or a concerto movement. A game, for example, might start from a given astronomical configuration, or from the actual theme of a Bach fugue, or from a sentence out of Leibniz or the Upanishads, and from this theme, depending on the intentions and talents of the player, it could either further explore and elaborate the initial motif or else enrich its expressiveness by allusions to kindred concepts. Beginners learned how to establish parallels, by means of the game’s symbols, between a piece of classical music and the formula for some law of nature. Experts and masters of the game freely move the initial theme into unlimited combinations.3
The combination of mathematics and music was inspired by the work of music theorist Hans Kayser (1891-1964), who also lived in Switzerland when Hesse was creating his novel and whose work is a Neo-pythagorean mathematical theory of music.4
Frozen Glass Beads of Facticity
Hesse’s approach is clearly interdisciplinary. The technical character of the game, although not explicated in detail, is a combinatorial one. This follows from Hesse’s enthusiasm for the famous Chinese I Ching, that yarrow stalk oracle and cosmology which results from the 64 = 26 combinations of six broken or unbroken lines (represented by yarrow stalks). This combinatorial cosmology gives the user a corresponding number of cosmological treads. It is not creative, but strictly selective. Hesse’s combinatorial perspective is also evident from his enthusiasm for Gottfried Wilhelm Leibniz’s calculus of logic, that idea of a perfect language where its conversations would be reduced to an exchange of formal logical formulas.5
This approach to reality as a combinatorial setup that can be controlled by formal logical calculus is the ontology we know from Ludwig Wittgenstein’s tractatus logico-philosophicus,6 the first sentence of which reads, “Die Welt is alles, was der Fall ist” (“The world is everything that is the case”). This point of view is utterly reductionist. “What is the case” means what has been made and is there now. Once, forever? And who made it, and how? This does not matter. It is a perspective on existence as a collection of frozen objects. Take it or leave it, but you are only the user, much as in the medieval cosmology where humans could just observe God’s world without any option of active intervention. A recent book7 confirms this view, describing a computer program that implements those combinatorial options in the spirit of I Ching and Leibniz. Hesse also stresses that his glass bead game is a search for God, that hidden universal spirit which connects all those diversities at the phenomenal surface.
In this style, Hesse’s Glass Bead Game idea was realized through my own work in mathematical music theory in 1985.8 However, it is not a neo-Pythagorean theory, but relies on modern mathematics (such as module theory, category theory, and algebraic topology). I wish to give a short overview of this work in order to demonstrate both its power and the fundamental limitations of the combinatorial method.
My approach started with Beethoven’s famous Hammerklavier sonata, op. 106, and applied mathematical models of tonal modulation and paradigmatic classification of melodies (the latter in the spirit of Jean-Jacques Nattiez)9 to a thorough analysis of Beethoven’s harmonic and motivic construction. In the vein of Hesse’s sketch of the game, I derived a mathematical formula that would describe his musical operations. This formula is the symmetry group Sym(C#-7 ) of the diminished seventh chord C#-7 = {c#, e, g, a#}. For the composition of a new sonata, we then exchanged this chord with an equally famous combination of pitches, the augmented triad C#+ = {c#, f, a}, and then used its symmetry group Sym(C#+) to construe all modulations and the motivic germs of the new sonata. In other words, I threw the mathematical formula back into the musical realm. The resulting sonata L’essence du bleu has been published and released on CD.10 The result could appear to be a real product of creativity. But it is simply a restatement of Beethoven’s thoughts (viewed through the microscope of our mathematical analysis) with changes of corresponding structures, mutatis mutandis. I don’t qualify this type of game as a truly creative one. There is no box whose walls are being opened into an unknown space outside; rather we are opening the door leading from one box (music) to the neighboring box (mathematics).
In what follows I shall explain how and in which domains of creative knowledge production Hesse’s Glass Bead Game can be morphed into a less passive, factual, and rigid activity.
The Revolution of Functors
A first conceptual revolution occurred in the new mathematics introduced by Samuel Eilenberg and Sounders Mac Lane with their theory of categories in 1945.11 They analyzed the basic objects of mathematics, namely points—those objects that have no parts according to the classical approach by Euclid: punctus est cuius pars nulla est. For categories, points were replaced by new elementary objects: arrows. Category theorists considered points as a result of a pointer action; “punctus” is the result or head of an arrow that pricks its target. The new concept of an arrow has three parts: its head and its tail, which are connected by a shaft symbolizing a pricking movement from tail to head. This seemingly harmless change introduced not just its pricked result, but the point of departure for a conceptual movement.
With this dramatic change, the subject (an arrow’s point of departure) was introduced and thereby the inclusion of the point of view or “address” from which the target point was addressed. In Alexander Grothendieck’s (1928-2014) algebraic geometry,12 this paradigmatic revolution introduced a completely new conceptualization, a kind of relativity theory in geometry. This theory eventually led to the solution of hard problems, such as the Weil and Fermat conjectures, by Pierre Deligne (1974) and Andrew Wyles (1995), respectively.
Yoneda’s Lemma replaces “whatness” by “suchness (Guerino Mazzola)
The big program was therefore to understand algebraic geometry as a science of relative points. The strongest general argument for this revolution was a simple, but powerful lemma, a result introduced by the Japanese computer scientist Nobuo Yoneda in 1954. He could prove that a space X can be completely described by its functor, i.e., by the system of all arrows f: A —> X where the address A runs over all possible choices. This functor, which we denote as @X replaces the space X as such by its system of variable arrows. More precisely, the lemma states that two spaces X, Y are isomorphic if and only if their functors @X and @Y are isomorphic as functors.
This result allows mathematicians to replace spaces (and in fact any objects of general categories) by their functors. In other words, the spaces qua objects with their intrinsic identity are replaced by their functors, which are systems of relative behavioral perspectives from variable addresses. In philosophical terms: the “whatness” of X is replaced by the “suchness” of the functorial entities @X. Mathematics is no longer interested in the abstract identities X but only in their behavioral systems, the functors @X.
This revolution generated a new paradigm of conceptual manipulation: Abstract spaces became only relevant via their behavior under specific but arbitrarily variable addresses. This means that a mathematical object could now be conceived as a distributed behavior, its identity reified as a collaborative system of addressed perspectives. With respect to Hesse’s original God-oriented approach, we now see that divine whatness being replaced by a “God-functor” that is realized in its (not necessarily representable) howness, and which our collaboration in a distributed identity experiences.
In mathematical music theory, I have composed a piece which is actually a recomposition of Pierre Boulez’s structures pour deux pianos I, following the functorial methodology13 and applying Boulez’s idea of a creative analysis. My analysis of Boulez’s original composition followed György Ligeti’s analysis14 which we reinterpreted as being a functorial procedure. Of course, Boulez did not know category theory when he composed that piece, but in the light of Yoneda’s lemma, his compositional strategy was in perfect congruence with modern functorial methods. I then implemented the functorial analysis in the rubato composer software and was able to recompose new variants of that piece by canonical address changes.15 It is not an exaggeration to understand Boulez’s structures as the historically first genuinely functorial composition.
Gestures in Philosophy and Science
This revolution generated a new paradigm of conceptual manipulation: Abstract spaces became only relevant via their behavior under specific but arbitrarily variable addresses. This means that a mathematical object could now be conceived as a distributed behavior, its identity reified as a collaborative system of addressed perspectives. With respect to Hesse’s original God-oriented approach, we now see that divine whatness being replaced by a “God-functor” that is realized in its (not necessarily representable) howness, and which our collaboration in a distributed identity experiences.16 A function is similar to an industrial plant: Input – Black Box – Output. As the great mathematician Henri Poincaré stated,
To localize an object simply means to represent to oneself the movements that would be necessary to reach it.17
He was aware that the gestural origin of geometry is beyond the formal mathematical mechanisms known at his time.
Let us clarify the radical abstraction from movements in mathematical formulas. Take a rotation in real three-space. This movement of the space’s point around an axis is represented by a 3 × 3 matrix M in linear algebra. But the matrix M does not show any information about the axis or the rotation’s angle. You have to work quite hard to exhibit such an axis from M. The catchword is eigenvalues, and there are corresponding eigenvectors (that could play the role of rotational axes); the rotation’s angle—if an eigenvector of a rotation can be found at all—is yet another problem to be solved by linear algebra. In other words: The mathematical representation of movements by matrices is a radical abstraction, a compactification of a movement in a formula, a kind of Fregean prison, where the original movement has been encapsulated. If we were to relate music to mathematics we could state here that they relate formulas to gestures, but in opposite directions: While mathematics compactifies gestures into formulas, music unfolds formulas into gestures.
Let us now focus on the gestural ontology which has thus far been hidden in contemporary functorial mathematics. For an excellent treatise on the history of gestures in European philosophy, we refer to Jean-Claude Schmitt’s, La raison des gestes dans l’Occident médiéval.18 He has given the most complete and important contribution to a history of the concept, philosophy, social and religious role of gestures during the early centuries of our modern Western culture. He exhibits the first (and still one of the best) definitions of a gesture, set forth by Paris-based theologist Hugues de Saint-Victor (1096-1141):19 “Gestus est motus et figuratio membrorum corporis, ad omnem agendi et habendi modum” (“Gesture is the movement and figuration of the body’s limbs with an aim, but also according to the measure and modality proper to the achievement of all action and attitude.”) Observe that in Saint-Victor’s definition, the specification ad omnem agendi et habendi modum is not semiotic, but merely describes the generic modality of action and being.
A gesture is a pre-semiotic concept; it does not automatically mean anything. The pointer gesture is also pre-semiotic since Saussurean structuralist semiotics is built upon the pointer from signifier to signified. A pointer is not a sign, but a “prequisite” to any semiotic concept architecture.
In the twentieth century, gesture philosophy was above all developed by French philosophers, linguists, and mathematicians. Their works also differ from the Anglo-Saxon linguistic philosophy of gestures that was developed by Adam Kendon and David McNeil.21 22 They focus on gestures that are co-present in linguistic utterances, and in this perspective, their concept of a gesture is strictly semiotic: Gestures are special signs that support the building of linguistic syntagms and contents. And they are always related to the body’s actions; no more abstract concept of a gesture, such as a gesture in a musical melody or a thought gesture, is addressed. We shall not go into this rather restrictive conceptual line in more detail, and refer the reader to a forthcoming book23 for further information.
The French tradition of gesture philosophy is characterized by the thesis that gestures constitute a proper ontology that is independent of, and typically precedes, semiotic systems; gesture is pre-semiotic.
Ahead of his time, French mathematician and philosopher Jean Cavaillès in 1938 stated a core property of gestures that bypasses any semiotic basis: “Understanding is catching the gesture and being able to continue.”24
In a remarkable series of interviews,25 the painter Francis Bacon made clear that his diagrammatic gestures were more important to his creative work than mentally planned strategies: “The marks are made, and you survey the thing like you would a sort of graph. And you see within this graph the possibilities of all types of fact being planted.” These insights inspired the French school of diagrammatic philosophers, led by Gilles Deleuze’s comments on Bacon.26 His ideas were then developed and deepened by gesture theorists and philosophers, such as Gilles Châtelet27 and Charles Alunni.28
Cavaillès’ dancing thought (also shaped in Pierre Boulez’s reflection on gesture in music)29 was in fact stated with respect to mathematical theories, and as such it was one of the very first principles of gestural embodiment in mathematics, an idea now quite fashionable through the work of George Lakoff and Rafael Núñez,30 but also anticipated in Châtelet’s observation that the Fregean concept of a function in mathematics is a dramatic (and questionable) abstraction.31
Gestures—except when “tamed” by social codes—are not signs in a semiotic environment. They are not a realization of Ferdinand de Saussure’s classical signification process. Châtelet is very clear on this point: “The concept of a gesture seems crucial to approach the amplifying movement of abstraction in mathematics. (…) A diagram can immobilize a gesture, put it to rest long before it is hidden in a sign, this is why geometers and contemporary cosmologists love diagrams and their power of preemptive evocation.”32 A gesture can be immobilized by a diagram (which in this French theory is a kind of disembodied gesture) before it becomes a sign. And Alunni confirms this creative pre-semiotic role of gestures: “It is not the rule which governs diagrammatic action but it is action which causes the rule to emerge.”33
The Ontology of Music (Guerino Mazzola).
Musical ontology requires a fourth dimension of embodiment comprising facts, processes, and gestures. This one is added to the three classical dimensions of signs, realities, and communication.
In short, we learn that gestures are generally understood as pertaining to a proper ontology that is not subordinate to semiotic lines of thought. In particular, the dominant French diagrammatic philosophy exhibits a sharp dichotomy between “wild” and “tamed” gestures, the former being independent or antecedent of semiotic realms, while the latter serve semiotic purposes as special types of signs. Conceptual creativity is exhibited in the layer of wild gestures. The communicative characteristic of (wild) gestures stresses their “howness” as opposed to their substantial “whatness.” Gestures are understood in their behavior, not in their absolute being (such as Kant’s thing-in-itself [Ding an sich]).
It is astonishing that despite the sensational success of Grothendieck’s “mathematical relativity theory,” there has been some work in the direction of replacing abstract Fregean functions and their formalism in category theory by gestural concepts. In Categorical Gestures,34 I have started an investigation of the possibility of enriching Yoneda’s lemma with gestures, and the ultimate goal would be to replace the present abstract foundational entities of mathematics, such as sets or arrows, with gestures.
In an extraordinary interview with the Notices of the American Mathematical Society, the prominent mathematician Yuri Manin states his vision of future foundations of mathematics:
Instead of sets, clouds or discrete elements, we envisage some sort of vague spaces, which can be very severely deformed, mapped one to another, and all that while the specific space is not important, but only the space up to deformation. If we really want to return to discrete objects, we see continuous components, the pieces whose form or even dimension does not matter.…I am pretty strongly convinced that there is an ongoing reversal in the collective consciousness of mathematicians: the right hemispherical and homotopical picture of the world becomes the basic intuition, and if you want to get the discrete set, then you pass to the set of connected components of a space defined only up to homotopy.…That is, the Cantor points become continuous components, or attractors, and so on—almost from the start. Cantor’s problems of the infinite recede to the background: from the very start, our images are so infinite that if you want to make something finite out of them, you must divide them by another infinity….I see in this an analogy with the rebuilding of pragmatic foundations in terms of a category theory and homotopic topology.35
Gesture Theory in Music
In music philosophy, music theory, and performance research, gestures have played a role of conceptual enrichment for a long time, but a full-fledged theory of musical gestures has still not been forthcoming, probably also because of the difficulty of an epistemologically valid conceptualization of gestures. Let us give a summary of some important gestural perspectives in the science and art of music.
Already in Eduard Hanslick’s determination of musical content as “tönend bewegte Formen,”36 (not just forms, but forms that are moved in a sounding manner, or moved in sound for short) the formal aspect, the formula, of a cadence, for example, is not sufficient to generate content. The form(ula) needs to be moved; it is deployed in a gestural dynamics. Hanslick illustrates his idea with the kaleidoscope, a dynamical arrangement of forms that receive their aesthetical value in a self-referential internal relationship.
In the twentieth century, German music theorist Wolfgang Graeser applied the mathematical theory of symmetry groups to restructure Bach’s The Art of Fugue, but while observing dancers who interpreted Bach’s Goldberg Variations, switched from abstract symmetries to what he called “Körpersinn,” the sense of embodiment in music.37 His book project on this topic could not be brought to fruition as Graeser felt completely alienated by such advanced ideas and committed suicide in 1928 at the age of 22.
In the theory of musical performance, Theodor Wiesengrund Adorno in 1946 wrote an essay on performance which set forth strong arguments for the gestural essence in performance. He followed Paul Valéry’s famous dictum: “C’est l’exécution du poème qui est le poème.” In this essay, Adorno asserts that “the idea of performance pertains to music as such and isn’t an accidental attribute.” His analysis of the gestural basis of performance is as follows: “Correspondingly the task of the interpreter would be to consider the notes until they are transformed into original manuscripts under the insistent eye of the observer; however not as images of the author’s emotion—they are also such, but only accidentally—but as the seismographic curves, which the body has left to the music in its gestural vibrations.”38 Adorno argues for what I have called “the score as a repertory of frozen gestures.” He does not suggest that gestures have an emotional message; rather, he defends their nature as “vibrating” bodily utterances. At first sight, this may look overly materialistic and far from the symbolic meaning of musical creation, but Adorno insinuates a spiritual component in the gestural dynamics. This perspective is in fact supported by the very history of score notation. Originally, scores encoded the gestural hints in the graphemes of Medieval neumes. These graphemes then successively morphed to the present notation, which has abstracted neumatic threads to discrete point symbols.
Adorno’s student Renate Wieland and her fellow scholar Jürgen Uhde clarify their teacher’s approach and apply it to their system of piano performance.39 Wieland says: “Originally affects were actions, related to an exterior object, along the process of internalization they were detached from their object, but they are still determined by the coordinates of space….There is therefore something like gestural (space) coordinates.” She makes it clear that gestures are abstractions from concrete actions which, however, remain geometric entities in some more generic space. Wieland also argues that the emotional connotation in music originally is e-motion, out-movement, and so the gestural transmutation is not an artificial construct, but the restatement of the original phenomenon.
American composer and music critic Roger Sessions has offered this beautifully clear description of the crucial but still underestimated role of gestures in performance:
What I am saying is that we experience music as a pattern of movement, as a gesture; and that a gesture gradually loses its meaning for us insofar as we become aware of having witnessed it, in its total identity, before. If it is to retain this meaning in its full force, it must be on each occasion reinvested with fresh energy. Otherwise we experience it, to an increasing degree, as static; its impact, as movement, diminishes, and in the end we cease to experience it as movement at all. Its essentially static nature has imposed itself on our awareness….Each performance is a new one, and the work is always studied and approached anew, even by the composer. The same, it should be obvious, is true of professional performers. I would go even much further and point out that there is no such thing as a “definitive” performance of any work whatever.40
On the level of music performance in technology, Manfred Clynes with his sentograph, and Johan Sundberg and McAgnus Todd with their performance software and gesture-driven concepts of the final retard in music, state in particular that “the performance and perception of tempo/musical dynamics is based on an internal sense of motion.” Similar approaches to cognitive models of gestures in music are shared by Marcelo Wanderley, Claude Cadoz, and Marc Leman.41
Coming from a different angle, namely music theory, the great American music theorist David Lewin introduced the gestural perspective in his seminal 1987 book,42 or almost, as the theory and the textual representation are in fact more complex. Lewin’s book describes what is now called “transformational theory,” later adapted by his student Henry Klumpenhouwer to become K-nets. Such networks replace an “amorphous” set of tone objects by a diagram, where the tone objects are placed in diagram’s vertices, while the diagram’s arrows designate (affine) transformations mapping tone objects into each other.
Lewin argues against what he calls “Cartesian thinking,” which observes musical objects as res extensae. Opposed to this passive attitude, Lewin suggests that transformations between musical points (such as pitch classes, for example) are the new path to pursue. In Generalized Musical Intervals and Transformations, we read: “If I am at s and wish to get to t, what characteristic gesture should I perform in order to arrive there?” Now, this language sounds very gestural, but is based on different mathematical principles. Let us clarify this subtle mathematical point, which may escape the nonprofessional. Lewin’s theory uses classical transformations and then, in Klumpenhouwer’s networks, diagrams of transformations. Diagrams are systems of transformations between a set of spaces, and they relate points in those spaces by determined transformations. But they are not identical with the point systems generated by the so-called mathematical limit construction. An intuitive, and incidentally mathematically correct, way of characterizing diagrams is as generalized equations, whereas the objects from the limits are solutions of such equations. So the diagrams play the role of industrial plants, producing facts (“factum,” what is made), namely Klumpenhouwer’s K-nets. So the Lewinian digression from Cartesian facticity (or extensionality) is the step to mathematical processes, but not to gestures.
Lewin’s formalism and his wording are different, but they show what his objective is: a gestural theory of music. It would be very interesting to investigate Lewin’s text with the subtext of gestural thinking in mind, since he repeatedly uses this metaphor in a speaking way. Relating to his question about the movement of s to t, he adds: “This attitude is by and large the attitude of someone inside the music, as idealized dancer and/or singer. No external observer (analyst, listener) is needed.”
This is a remarkable statement, which leads to the question set forth by music theorist Robert S. Hatten in his book with the now explicitly gesture-related title Interpreting Musical Gestures, Topics, and Tropes:43 “Given the importance of gesture to interpretation, why do we not have a comprehensive theory of gesture in music?” For Hatten, gesturality became a core topic when he learned that performance of classical piano music—Mozart, Beethoven, Schubert—is strongly determined by gestural attitudes. This is best exemplified when comparing Glenn Gould’s interpretation of Beethoven’s op. 57 Appassionata with Vladimir Horowitz’s version. Gould’s performance completely lacks gesturality. His “analytical” reading is the opposite of what Adorno recommended, and amounts to Beethoven minus gestures, a substantial negation given the strongly gestural nature of Beethoven’s music. Hatten confirms this in theory, as does Gould by his counterfactual experiment. We may refer to Categorical Gestures,44 in which gesturality in Beethoven’s Hammerklavier sonata, op. 106, has been analyzed using a mathematical theory of gestures.
Hatten’s definition of a gesture reads as follows: “Gesture is most generally defined as communicative (whether intended or not), expressive, energetic shaping through time (including characteristic features of musicality such as beat, rhythm, timing of exchanges, contour, intensity), regardless of medium (channel) or sensory-motor source (intermodal or cross-modal).” He distinguishes his understanding of gestures from the school of Adam Kendon and David McNeill in that (i) semantic aspects are not characteristic and (ii) he stresses “energetic shaping through time,” an interesting wording, since the main subject is “shaping” an action—not the shaping of something, but pure action. The making in itself becomes a central feature, not the resulting facts generated by the making! In remarkable congruence with Wieland’s abstract geometry of gestures, he adds that: “At a higher, more symbolic cognitive level, the representation of gesture may be considered amodal, in that it is not restricted to any particular modality.”
We should also recall that Cecil Taylor, the monstre sacré of free jazz piano, describes his approach to creative improvisation with these words: “I try to imitate on the piano the leaps in space a dancer makes.” This is a completely gestural concept, and in a documentary entitled All the Notes by Christopher Felver, Taylor says about score reading that deciphering those symbols takes away most of the energy for creativity.45
In short, we recognize that there are numerous approaches to music and performance theory that stress the primordial role of gestures in their conception as pre-semiotic components of a deeper understanding of music.
Let us now provide a short overview of the mathematical theory of gestures that we have developed since 2002. But it is a theory of expressions that point in a complex way to gestures, those exotic “animals of human communication” that we try to understand in their dynamical behavior. It is in no way an attempt to turn gestures as such into signs. In this section, mathematical prerequisites are required, and reference to introductory texts46 or the forthcoming book47 is recommended. This approach is inspired by Saint Victor’s definition, but it gives it a formal shape that enables the development of an interesting concept architecture and the proof of powerful theorems. This theory was developed as a theory of musical gestures to capture many of the problems that have been mentioned above. However, this mathematical formalism is also applicable to more general contexts of gestural utterance, such as dance and painting.
To begin with, a semiotic setup that attempts to generate a precise formalism of gestural structures is proposed. Notice that this is not an attempt to recast gestures as signs, but as entities that may be expressed as mathematical structures. In terms of Hjelmslev semiotics, this amounts to establishing a gesture semiotic GestSem that has an expressive level Ex(GestSem), which realizes a mathematical theory of gestures, i.e., a classical system Ex(GestSem) = MathGest within the semiotic of mathematics that offers a set of “expressions” or “forms” that point to gestures, the content level Ct(GestSem) = Gestures of gestures proper. The signification level Sg(GestSem) is not restricted to any particular gesture, meaning that we may set Sg(GestSem) = GestSem. This means that the semiotics of gestures is its own signification level, that its expressive level is a mathematical theory of gestural forms, and that gestures remain closed off from metatheoretical specifications: gestures are not signs. A definition of the pointer gesture will be given below, when the formal theory is explained.
In what follows, “gestures” or “mathematical gestures” will be discussed, although the focus is on expressions that signify gestures. To define a mathematical gesture, we need a directed graph Σ (short form: digraph) and a topological category X. The latter is essentially a category where the category-theoretical operations are all continuous. A topological space is a special case of such an X. We turn X into a digraph C(X) by considering the space C(X) of all continuous curves, in fact continuous functors, c: I —> X on the unit interval I = [0,1]. The domain I is interpreted as a topological category, whose objects are the points x in I, and whose morphisms are the pairs (x, y) for x ≦ y. The digraph’s arrows are these curves, its vertices are the objects of X, and the head and tail maps are h(c) = c(1) and t(c) = c(0). A gesture is a digraph morphism g: Σ —> C(X), where Σ is called g’s skeleton, while X is called the gesture’s body. Intuitively, and realizing much of Saint Victor’s definition, a gesture is a representation of the skeleton as a configuration of curves in the gesture’s body.
A simple sign in GestSem would be the pointer P. We could set Ex(P): ↑ —> C(I), the map which sends the digraph ↑ (with one arrow that connects two different points) to the identity curve Id: I —> I, we would define Ct(P) = the gesture of pointing, and finally Sg(P) = P, the self-reference. One could then also declare the middle signification level in Hjelmslev’s model to be identified with GestSem, thereby replacing the simple pointer by any gestural structure.
A first noteworthy fact is that the set Σ@X of gestures with skeleton Σ and body X is canonically endowed with the structure of a topological category. This implies that we may now consider gestures in ∆@Σ@X, i.e., gestures with another skeleton ∆ and their body being the topological category Σ@X. These are gestures of gestures, or (mathematical) “hypergestures.” The iteration of the gestural construction is a powerful technique for constructing most complex gestural anatomies. The main result with regard to hypergestures is the Escher Theorem.48
Escher Theorem: If Σ1, Σ2, … Σn are digraphs, X a topological category, and if p is a permutation of the set {1,2,…n}, then the two topological categories of hypergestures Σ1@Σ2@ … Σn @X and Σp(1)@Σp(2)@ … Σp(n) @X are canonically isomorphic.
This theorem looks abstract, but it has important consequences for musical creativity. Let us look at two applications:
(i) The interaction with a fellow musician in improvisation often boils down to interpreting his/her musical utterances in a different way to create new perspectives. And in such an endeavor, the Escher Theorem helps reconsider a given hypergestural configuration from a new angle. A loop of rhythmical curves might be reinterpreted as a curve of rhythmical loops, for example. This is exactly what the Escher Theorem sets forth: it provides you with a new, albeit equivalent, perspective on what’s going on.
(ii) The second example relates to the first species: voice against voice, of classical Fuxian counterpoint. There are two interpretations of the conjunction of cantus firmus with discantus. One reads the discantus voice as being the punctus contra punctum, the cantus firmus being the first voice against which the discantus is set. Reading this configuration as a hypergesture, the gesture of cantus firmus is deformed in a hypergesture to the gesture of discantus. This interpretation reads contra (counter) as being an opposition of two voices. This interpretation is often given in counterpoint courses, but it is erroneous for two reasons. To begin with, the intervals between cantus and discantus are all consonant, so there is no rationale for calling this relation a contrapunctus. Second, it is historically inaccurate.49 The historical contra idea was understood as being a counterposition to successive intervals; the tension was to be built when stepping from one (consonant) interval to its (consonant) successor. Punctus meant interval, and it is in fact well known that contrarious movement of voices is used and preferred when building first species configurations. This second interpretation does not view the discantus as a deformation of the cantus firmus, but it considers the interval gestures as being deformed into each other along the time axis in a hypergesture of temporal deployment. The Escher Theorem however reconciles these two interpretations as being related to each other by the isomorphism that it provides.
Gestural Creativity
Musical creativity has always been a mysterious business since it is not only creativity as such, but moreover remains largely closed off from verbalization. Music is created in a nonverbal process. But if we look at creativity in general, it is largely pre-semiotic since it deals with opening boxes, transcending given languages and formal environments. Creativity is a semiotic generator.50 Therefore, the gestural layer is quite canonical as a candidate for understanding and performing musical creativity.
In my book Flow, Gesture, and Spaces in Free Jazz,51 I have described musical creativity in jazz, especially free jazz, using these ideas. It is about making the rules (in the sense of Charles Alunni), negotiating them in a gestural interplay, and establishing a distributed identity as a rotational movement around the axis of a distributed identity. This is what Ornette Coleman meant when he said he wanted to “play the music, not the background” in the liner notes of his famous album Free Jazz. He wanted to step away from the reproduction of others’ ideas, and to make music without copying a pre-established template. This is why Cecil Taylor imitates a dancer’s leaps in his creative piano universe. He does not want to imitate given forms. His aim is to express his dancing thoughts.
Gestures and Strings
The softening of creative knowledge production in a gestural approach not only softens the conceptual framework, it also softens the barriers between disciplines. Soft knowledge cannot be limited by traditional disciplinary walls. This type of transgression of disciplinary limits is in fact observed in our theory of hypergestures, especially for the example of counterpoint cited above. The Escher Theorem enables an exchange of roles in hypergestural combinations. External gestural skeleta can be exchanged with internal skeleta. A loop of lines can be transformed into a line of loops. This gestural insight is in fact also observed in physical string theory, where in S-duality, strings can be reinterpreted as branes, and gauge particles as bosons.52
Gesture Theory – String Theory? (Guerino Mazzola).
This situation is parallel to musical counterpoint, which can be interpreted as a hypergesture between cantus firmus and discantus or as a hypergesture of unfolding intervals.
This string-theory connection has been realized in a third application of mathematical gesture theory in music that relates to the investigation of the complex transitional process between reading a score in its symbolic realm and performing it in physical reality.53 The model here represents the score data as symbolic gestures and the performed events as physical gestures, and then connects these two components by a hypergesture that is constructed following the Lagrangian formalism from physical string theory. This model enables a detailed analysis of the artistic potential that shapes the connection between symbolic and physical gestural utterance.
Playing the Multiversed Game in a Pre-Semiotic Ontology
If we collect and summarize the dramatic changes in the basic conditions of Hesse’s glass bead game, we can state that it has the following new features:
(a) It is played by a distributed identity of collaborators;
(b) It is not following given rules but creates them on the spot and according to an interplay of equivalent partners, in a pre-semiotic layer;
(c) The collaborative interplay is made by gestural exchanges, by a comprehension that does not follow templates but is built upon the repercussion of gestural dynamic;
(d) The conceptual framework is successful in its creation of a rotational energy around the axis of a distributed identity. Success is possible and addressed, but it does not result from given criteria; it is established in a distributed harmony without the time-space invariance of traditional laws.
These characteristics redefine the glass bead game of “melting glass beads” with the following consequences:
(a) The gestural ontology is not auxiliary or preliminary to a factual layer of reality. It has its own persistent reality that does not serve what might become the case later on;
(b) The incessant gestural remaking of rules and concepts eliminates the world’s unicity, completing the historical suspensions of geocentricity (Copernicus), anthropocentricity (Darwin), chronocentricity (Einstein), and ratiocentricity (Turing), and adding the suspension of factocentricity;
(c) Creativity is no longer delegated to arcane divinities; it has also become a radically human endeavor. Creation is no longer limited to God’s initial Big Bang;
(d) The transdisciplinary parallel between music and physics shows these parallelisms: hypergestures—strings, Escher Theorem—duality, works (typically 1037 72-element motives)—universes (10450 string theory landscapes), and communication via gesture interaction—Interaction via exchange of bosonic strings.