1)      Have the robot’s perception created by a dual route model. In this dual route, the robot will have two spaces of information that interact in order to create a sense of agency in the robot. The first space is the raw sensory data coming from environmental stimuli. The second space is an internal representation of what it is seeing; a space that will provide meaningful information to the robot.

a.     The environment needs to be a dynamic place of interaction in order for the robot to be able to manipulate it in certain ways. If it is a genetic algorithm approach, then we need to consider rewards and punishments for the propagation of certain genes. If it is not a genetic algorithm approach (we can have both approaches simultaneously to help inform each other), then we still need to consider aspects of the environment that can be manipulated and defined by the robot.

                                                               i.      The conditions of environment

1.      Food

a.     Need to have a reason for some to succeed over others.

2.      Friends, Enemies

a.     Depending on the aid or non-aid of satisfying certain objectives such as happiness which could depend on a number of factors relating to the environment we design.

b.     Holding grudges towards those who didn’t aid in food before/ Helping out those who have aided

3.      Breeding

a.     There has to be a reason to mix the different genes of the robots. This mixing may bring about emergent aspects of consciousness that we are trying to solicit.

2)      Achieve a level of natural semantics in the internal representation by having the robot store knowledge about the environment based upon how its body can manipulate the object. Any semantic information about the environment will deal directly with how the robot can manipulate and interact with the object. In this way, the robot will be able to understand assigned meaning, and “not all meaning will be our meaning.”

a.     This requires the use of Image Schema’s[1]. Creating schematic representations of how the body can interact with the environment.

b.     An important goal is trying to figure out how we can program these image schemas into a robot.  Paul Cohen was trying to achieve this very thing. He has made some discoveries in the domain in recent work. See: http://www.isi.edu/~cohen/publications.html for a list of publications and information about this researcher. His work is a crucial component of this project. If you are interested in theoretical work, please look upon his research.

c.     For more information on Image Schema’s see Appendix A.

3)      Implement a theory of syntax created by Professor Brandt called Stemmatic Syntax and have this as a basis of communication with the robot. This system breaks down the structure of language into 8 nodes that reflect how humans perceive a situation. We plan to relate the semantic information the robot learns from its environment to these nodes; associate certain action schemas the robot does to certain nodes. This will allow the robot to have a schematic understanding of what the stemma input means based upon the semantics it has associated with each node. Then, it may be possible to tell the robot to link the words we use in the stemma to the internal representation the robot has created, eventually teaching the robot language.

a.     I am thinking something along the lines of having the robot employ a perceptual symbol system[2] approach using the stemmatic structure to organize and categorize the different actions schemas. What I mean by this is: part of an action schema will be recorded and associated with a certain node in stemma. Such as pushing a cup TO somewhere will be a four. A schematic representation of the sensorial data will be coded to the fourth node. Then, when we type something to it in a stemmatic format, when it sees a four node, it will activate that sensory information and know what a four means. In this way, it can look at the nodes and have an embodied understanding of a stemmatic text. Then, we can tell it to link the words to the schematized action schema and it will actually know what the words mean after training it with a lot of stemmas.

b.     This could also provide a process of elimination in the genetic algorithm. The better a certain type of robot is at associating: action schema -> stemma structure node -> word will be the most fit in the system we design. In order to get the food, they would have to pass a test. It could even be a textually based test. We could give it words and if it knew what they meant, how to enact them or understand them in some way, it would get the food. The ones that are the best at getting the food will propagate and succeed.

 

 

1)      Image Schemas: Image schemas are schematic, embodied representation within the brain that is pre-conceptual, non-representational, non-propositional entities that accumulate to provide a base for concepts.

a.     The Image Schema BALANCE is created by the embodied experience of balancing for an individual. The experience of balancing provides a pre-conceptual notion of balance that can be used later, after many trials of balance, to provide a polysemic ( multiple meaning) understanding of the notion of balance in the world. Such as the balance of good and evil. Or a fair trade, the situation has balanced giving and taking and it is dictated as fair. This concept derived from a bodily experience of balancing.

b.     Schematic: Image Schemas are not holistic image representations of the event experienced. They are schematic, partly constructed, patterns of interactions of the experience within the world. After many schematic representations have been accrued a solid base for the concept of balance will be formed.

c.     Embodied: Image Schemas are body based. They are dependant upon how the body interacts with the world. The schema BALANCE depends upon how the body controls its muscles in response to a feedback from the nervous system and input from the environment. They cannot be separated from the environment because they stem from it.

d.     Pre-Conceptual: Image Schemas are present before concepts are formed. They provide a base for concepts to be built upon later. They are not conscious images because this would imply a concept. But rather they are patterns of interactions of embodied interaction with the world that predates concepts. Hence, they are preconceptual.

e.     Non-representational: There is no duality between the subject and the activity. The activity is not being represented within the mind of the individual. Image schemas are patterns of interaction that include both the activity and the subject in an interwoven experience that gives rise to schematic structures.

f.        Non-propositional- they are non-linguistic, as in pre-conceptual. They also do not employ any language of thought.

g.     Image schemas are formed through a process called perceptual analysis. This is the process that the mind goes through in order to take sensory input and construct a schematic representation of the object that includes the multimodal/kinesthetic response that the object evoked. The representation does not resemble the object that it encoded, but the individual is able to apply the perceptually processed object in order to have an understanding of it.

                                                             i.      The information processed is transformed from a perceptual input to a non-perceptual form that represents a meaning. It differs from normal perception because it only occurs with the attention of the individual. These representations are crude analogues to the information it stems from.

                                                           ii.      Mandler puts forth this theory and this theory of concept formation is the opposite of what Piaget proposes. Piaget thinks that imitation produces imagery that enables thought. Mandler, with perceptual analysis, thinks that image schemas stemming from the perceptual analysis of perceptual information provide the meaning that enables imitation to take place.

 

 

The theory will best be explained in terms of an example. If I am looking at a car for the first time, I see the car, the doors, and the wheels, all of which are physical aspects of the car represented in the visual system. Then, I go to get in the car and feel the handle and the weight of the door which are both tactile representations. Next, I start the car and hear the engine, auditory stimuli. Maybe I will smell the new car scent, olfactory stimulus. In Barsalou’s theory, all of these different aspects of car are connected under the umbrella term (a kind of container of various aspects) of car; a perceptual symbol system of car.[3] The next time I see a car, the umbrella term will activate and all of the corresponding terms, all of which he claims have neural constructs that are hence connected, will activate.[4] When seeing a different car, maybe this time I will learn something different and it will be integrated into the perceptual symbol system.[5] Say, if I got into a crash. This will be integrated into my perceptual symbol system of car and may be activated if I am in a car driving in a similar situation. [6]

 

 

2)      Perceptual Symbol System- P.S’s are schematic neural representations that are multimodal, across multiple sensory modalities, sensorimotor, proprioceptive, and introspective.

a.     Perceptual symbols activate when the body performs some function or interacts with the environment in any way. They are the schematic neural representation of the embodied action. These perceptual symbols then combine when they are related to form a simulator.

b.     A simulator is a concept. A simulator contains frames which organize the perceptual symbols into groups within the simulator.

                                                             i.      For example: The simulator car can have frames for wheels which contain the perceptual symbols of hub-cap and lug nuts. The Hubcap frame can be further specialized to contain the perceptual symbols for lug nuts, spaces between spokes etc.

c.     They are not holistic symbols that represent the whole neural activation of a certain activity. They are schematic representations. Selective attention likely plays a role in the selection of certain traits that become stored in long term memory.

d.     P.S.S are not amodal. They resemble the thing that they represent in space. Amodal symbols, like those of cognitivism, do not have any basis in reality.

e.     They are multimodal in that they can have a schematic representation of multiple sensory modalities. They can have a representation of the sound a car has, the feeling of excitement one has when inside, or any other feeling that is associated with car. These p.s.’s all come together to form the simulator, concept, of the perceived event car. Later this simulator is used to run an infinite amount of simulations of the car.

f.        A simulation is a physical experience or imagined experience of a simulator. Such as my physical encounter with car. I draw upon my base of p.s. to interpret this encounter. New p.s. are created upon further inspection of a new car and the simulator for car is updated. Simulators are dynamic. They evolve with each encounter for the simulation.

g.     These p.s. are largely unconscious. They are not like mental images. They lie in the unconscious of the mind that provides a base for the conscious representation of concepts. This side steps a lot of problems that arise when dealing with imagism. Imagism holds that cognition is the manipulation of images within the brain, but they fail to explain how these images are encoded and what system is utilized during there representation within the brain.

h.      Shortcomings: Barsalou’s theory of applying p.s. to abstract concept falls short of coherency. His concept of truth in his paper on p.s. is less of truth and more of a type of matching or something of the sort.

                                                             i.      Also, Barsalou does not state explicitly how the certain features of perceptual symbols are selected for schematic representation.

 

Barsalou’s Image Schema: http://cogsci.ucsd.edu/~coulson/203/goldstone.pdf

Barsalou’s Perceptual Symbol System: http://psychology.emory.edu/cognition/barsalou/papers/Barsalou_BBS_1999_perceptual_symbol_systems.pdf

Paul Cohen’s Site: http://www.isi.edu/~cohen/

 

---