Nick Davis

The aim of this paper is to extend current research into perceptual symbols and meshing into the computational domain. The model proposed may be used as a catalogue of mental states for both a cognitive robot and a human. The motivation and logic for the model can be found in another document, Dynamically Meshing Perceptual Symbols. The current paper will use this as a ground and continue from there.

    Looking at the integrated dynamic system, we see two major loops. First, the sensory loop in the top right, and next, the intentional loop in the bottom left. The sensory loop is scanning the environment, while the intention loop in creating a template with which to interpret environmental input. There is only one source of information coming in from the environment and that is the sensory arrow coming from conceptualization. This activates neurons and then the simulator, which in turn activates the subset of proprioceptive information in the simulator in order to determine if the entity under examination can be a part of the current simulator. Next, the information either does one of two things, there is a bifurcation in the diagram, namely to higher order processes or to conceptualization. The sensory data will either be clamped, and reality will be the surface level qualities, like color and form, or one concentrates and elaborates on certain elements of reality, making it more meaningful by attributing a narrative history or future. The sensory loop will be more active when a lot of sensory information is considered in a coarse grained way, while the intention loop will be active when analyzing a small subset of the sensory data with respect to planning a situation, creating some narrative structure to be executed in reality. 

    This model can be implemented in a cognitive robot to determine the allocation of computational resources. It can do this by considering each loop as happening in real time, with a certain frequency. For example, we have been talking in class about the agent scanning its environment. The sensory loop would be precisely this: each scan would be one complete sensory loop. This scanning is variable based on the  agent’s attention and relevance of sensory input for the current goal. Which brings us to the next loop, the intentional loop.

    In order for environmental input to have any meaning, the robot must have some purpose. This narrative structure would be determined by going through the intentional loop. For example, as seen in Dr. Brandt’s robotic note, the robot would set up some ideal representation of the world. Maybe the ball in the virtual environment needs to be on the other side of the room in order to get energy. In order to create an action routine for this outcome, an ideal representation would be created, namely a third person representation of the ball on the other side of the room,  and the execution of this goal would be mentally simulated based on the motor capabilities of the robot and the desired outcome. Upon completion, the intentional loop is completed. However, keep in mind that this model is happening in real time and it may take many intention loop cycles in order to create a coherent action plan. After this state of concentration, the robot has created a plan and can now navigate through the environment. The sensory loop once again regains dominance of the system, let’s say with a 4 to 1 ration of sensory scans to intentional loop scans. This 1 intentional loop would serve to verify if the current action trajectory will yield satisfactory results.  If so, then continue at the current 4 to 1 rate, if not, concentrate on the goal in order to augment the plan.

    In this scenario, the scanning frequencies would change, to keep things simple, the ratio could be reversed, 4 intentional loops, for every 1 sensory loop. This will focus more computational resources on creating a plan and using the third person representation in order to reason, while allocating minimal resources to scanning the environment.

    Thinking about this system with respect to Max’s latest contribution on stemmatic robotics may yield some interesting thoughts. His paper explores stemmatizing the agent’s experience. Each action the robot does denotes the verb of the stemma while the content of that action would fall into the respective place on the stemmatic tree. He notes that in order to implement this system, we will have to provide the agent with a basic ungrounded vocabulary. This is true for ‘to be’ verbs, but for action verbs, it may not be necessary. For example, the verb could just be the motion the agent is currently undergoing: wheels turning at x rate, or touch sensor compressed x amount. Of course, this input would be strings of numbers without the words I inserted, but it is tricky for me to write in 1’s and 0’s.

    The stemma could represent the intentional state of the robot. It could describe the destination or the motivation for action. For example, I turn wheels at rate x, to destination y, for pushing z.

In this example, the only terms that need to be pre-encoded would be closed class terms like ‘to’ or ‘for.’ Destination x would be a numerical value corresponding to some point in the third person representation. Pushing z would be some desired actuator activation of the touch sensor.

    In this context, I am thinking about a constantly changing stemmatic tree that describes the current motivation of the robot. This stemmatic tree would be initially constructed in the intentional loop of the robot and updated as the environment dictates. For example, if there is some obstacle, then the agent must update its plan and create a new stemmatic representation of the goal. If the agent had this stemmatic representation, then it could communicate this to another agent in order to inform agent2 its goals.

    Additionally, we need not necessarily define the names of objects, but the agent could refer to them by their features, both visual and affordance based. Depending on previous interaction with objects, the label could be affordance based. For example, a circle could be ‘thing that moved with x force’ (with numbers, again) and this could be activated based on a feature analysis and comparison with objects previously encountered.

    Bringing all of this back to Dr. Brandt’s robotic note, the narrative structure that the agent constructs could be stemma based as Max and I have been describing. An algorithm for allocating resources in order to 1) create  a plan or 2) scan the environment in order to verify the outcome based on the ideal state  could be determined by the dynamical meshing system introduced here and elaborated on in the dynamical meshing perceptual symbols essay.