Max Jensen
In this short essay I will briefly discuss a semio-linguistic model of sentence representation. The goal, as it pertains to the class, to give a basic sketch of how I see language interacting with the world – the medium is stemmatic syntax. For those not familiar with stemmatics, I will attempt to give a brief explanation, although there is a more detailed explanation on the wiki for anyone who takes special interest.
The following is a basic sketch of Stemmatics. Stemmatic syntax represents the relationship of the categories of words to each other by integrating them into a hierarchical, canonical structure: see appendix (Fig. 1). The nodes that are the simplest to understand and have the most overt relation to robotics are nodes 1, 3, 4, 5, and six. Hence, I have constructed a sentence with just these nodes, plus the verb (Fig 2). The verb heads the sentence, which is simply because sentences are either about actions or relations, and verbs, by nature, either describe action or a relation. The first node is the subject (she), the doer of the action. The third node is the object. This is easy to understand in the case of a transitive sentence, as in the example I have given. The ball receives the action of the verb. The fourth node is the recipient of the object, hence Jim. The fifth node is a little more abstract, but typically contains adverbs and constructions using with, of, or from. A way to think of this is the following diagram:
The idea behind the co-agent node is expressed semiotically by this diagram. The stereotpycical words for co-agent are of, from, and with. Think of the sentences “I am from Europe” and “I am of European descent.” In both cases they show a split (you may not be in Europe now) but also a parallel cotinuance (you are still European). The “with” diagram shows that there is a parallel meaning, for example “with the bat” shows a force being exerted in parallel by the subject and the co-agentive bat.
While this example is simplistic, I feel it may be very useful for robotics. With some major advancements in object recognition and gestalts, a robot may be able to understand objects interaction with each other. If an object is being acted upon by another object, it could label it as the direct object. If an object is doing the action, it could be called the subject. The end place of the object that was enacted upon could be marked as the indirect object (the fourth node). The idea of movement from one location to another could be seen as an interaction between the fourth and fifth node. Because stemmatic syntax is rooted (very abstractly, perhaps) in the physical relation of objects, a robot may be able to create a stemma tree. Many cognitive grammars discuss the idea of frames and image schemata, but do not relate them directly to how they function in language. I hope in a later paper, perhaps the final, to discuss this more specifically.
The major issue at hand, however, is the feasability of all this. In order for a robot to “stemmatize” it will have to go through many complex operations such as recognizing that an object is a unified whole. The major point of this paper was to introduce the idea of Stemmatic Syntax to the class and to show some of our goals for the (distant) future. We feel that it is a powerful tool as it relates linguistic relationships to those of the real world, something that would be necessary for a cognitive robot.
Figure 1
Figure 2
Comments (5)
Nicholas Davis said
at 12:54 pm on Oct 6, 2008
I like the idea of trying to label certain actions and actants within the robot's surroundings as having certain stemmatic properties. I was wondering if it was a possibility to stemmatic iconically. For example, in the subject stem, the robot could have a picture of who is doing the action, or maybe just the sensory data pertaining to viewing this agent. The dative could be the temporal clip of going 'to' someplace. If we did this, we would avoid integrating language in the beginning, thereby circumnavigating the grounding problem. So, what I am wondering is how this should happen from a developmental point of view, as mentioned in Emily's paper. Should we gradualy try to integrate this system so that it is more naturalistic, and what kind of relationship would a stemmatic organization of the environment have to do with the 1st and 3rd person perspectives? I have been thinking about a gradual increase of third person representational utility, as mentioned in the comments of Emily's paper, how could we use this stemmatic business with that learning curve?
Matthew McCroskey said
at 2:22 pm on Oct 10, 2008
Fascinating material, Max. You've definitely piqued my interest in stemmatics; I plan to read up on it elsewhere in the wiki. It seems theoretically feasible to incorporate stemmatics into our robot at least roughly in the way that you mention. I think that Nick has offered some excellent modifications to your basic idea, modifcations that should make it even more feasible. He made another excellent point about using a gradual increase in the number of nodes the robot can use as a means to emulate human language learning. Do you know of any papers (or how to find any papers) that might deal with the means and timetables for human learning of how to manipulate the different nodes? I assume this has been discussed extensively in developmental psychology, but wonder if a paper specifically in the context of stemmatics has been done. If so, it seems like it would be worth reading in the context of our goals for this course.
Nicholas Davis said
at 2:42 pm on Oct 10, 2008
Hmm...developmental stemmatics. I don't think a paper exists on this topic specifically, but there is some other stemmatic material on: stemma.pbwiki.com (stemmatic syntax link in the sidebar), however this is a good research topic to pursue I think. Maybe we could look at the list that leland provided and try to theorize which nodes correlate with the different steps in development.
smj14 said
at 5:40 pm on Oct 10, 2008
Max, I *think* I understand your approach to using stemmatics. So, by using categorizations of words, a robot can ultimately learn the meaning behind such words. Many words can go into one category and shape the category it is in (give it meaning). Or, the words can be put into multiple categories and the kind of meaning achieved depends on the combination of categories it is placed into. hmmmm interesting...
maxjensen said
at 12:04 pm on Oct 17, 2008
Stemmatics is pretty Case based, namely through Per Aage. I'd be more than happy to discuss stemmatics whenever any one would like. It's something we would like to develop, so the more people that here about it and understand it, the better we will understand what needs to happen with it.
Sarah, you've basically got the idea. Like i said, stemmatics is still in its infant stages, so the extensions of it aren't perfectly clear even to me still. But yes, what I'm trying to get at is that there are some basic image schemas that pertain semantic types (i.e. the nodes). So yes, I'm proposing that this may help solve the symbol grounding problem, because there is (in fact has to be) a relation between the content of the word and what's going on out the in the world.
Nick, your question is much more difficult. I was hesitant to write this paper, because it makes a pretty big leap. In order for this to happen, we have to have the robot making gestalts of objects, and for the objects to keep identity as they move through time and space. THIS IS VERY HARD! Let's see where it takes us though
You don't have permission to comment on this page.