A link to the seminars given last year (2004-2005) is provided so that the program is still available.
Mark Payne, Thesis proposal: Integrating Phonotaxis and Visuomotor Behaviour in the Cricket Robot
Biorobots have proved useful tools for investigating sensorimotor processing, but examples with multiple sensory modalities are few. In natural environments, animal behaviour is rarely governed by a single sensory modality, and if the robotic approach is to be used to its full potential then the issue of biologically plausible sensory integration needs to be addressed.
The existing robotic model of cricket phonotaxis has already been expanded with hardware which models visual motion detection in the fly. This set-up was used to investigate the integration of the phonotaxis with the optomotor response. The low quality of original cricket behavioural data prevents any firm conclusions being drawn about the different schemes tried. The aim of my PhD project is to collect new behavioural data from the insects to produce an improved model of auditory / visuomotor integration. This will involve refining the knowledge about the optomotor response, coupling this with new findings about the reactive control of phonotaxis, and hopefully expanding the behavioural repertoire of the robot to include object fixation or responses to polarised light. The finished robot will be evaluated by direct comparison with the insects.
Sethu Vijayakumar, Can kernel methods be useful for reinforcement learning?
Kernel methods are powerful function approximation schemes and encompass a huge variety of popular machine learning methods like Support Vector Machines, Gaussian Processes etc. However, their use in Reinforcement Learning has been quite limited. I will introduce a novel way of looking at function approximation using Reproducing Kernel Hilbert Spaces (RKHS) and suggest possible ways of formulating temporal differnce (TD) reinforcement learning in the framework of kernel based methods based on the formulation.
Barbara Webb, The problem of representation
I will discuss ideas and distinctions in the use of the term "representation". My aim is not to be too philosophical but rather to ground the discussion in examples of behavioural control in robots and animals, and to illustrate the practical consequences for explanation and design that arise from making these distinctions.
Ernesto Andrade, Modelling Crowd Activity for Surveillance
Trying to recognize the behaviour of a crowd is not an easy task. To simplify this task one can apply domain knowledge and statistical methods to model specific activities and behaviours. Following this road I will discuss different aspects of crowd modelling and monitoring on the surveillance domain. Concentrating on results obtained with Hidden Markov Models discussing their applicability, limitations and extensions.
Yaakov Engel, Learning to Control an Octopus Arm with Gaussian Process Temporal Difference Learning
The Octopus arm is a highly versatile and complex limb. How the Octopus controls such a hyper-redundant arm (not to mention eight of them!) is as yet unknown. Robotic arms based on the same mechanical principles may render present day robotic arms obsolete. In this talk, I will describe how we tackle this problem using an online reinforcement learning algorithm, based on a Bayesian approach to policy evaluation known as Gaussian process temporal difference (GPTD) learning. For our experiments, we use computer simulation of a 2-dimensional model of an Octopus arm. Even with the simplifications inherent in this model, the state space we face is a very high-dimensional one, for any arm of reasonable size. We apply our algorithm to this domain, and demonstrate its operation on several tasks of various degrees of difficulty.
Marc Toussaint, An EM-algorithm for solving MDPs - and its analogy to policy iteration
Solving a Markov-Decision Process means finding a policy that maximizes the expected future reward, assuming that the MDP is given (which is a subtask of model-based Reinforcement Learning). The standard approach to solving MDPs is based on computing value functions via value iteration or policy iteration. I'll discuss an alternative approach that is based on probabilistic inference. r a passage. This judgment is based on "mentally" simulating an obstacle-avoidance algorithm and predicting the outcome.
All these experiments have in common that a robot observes a single camera image and understands properties of the outside world solely by associating action or by predicting the effect of action on the image.
Nicolai Petkov, Contour detection by surround suppression of texture
Various effects show that the visual perception of an edge or line can be influenced by other such stimuli in the surroundings. Such effects can be related to non-classical receptive field (non-CRF) inhibition, also called surround inhibition, that is found in 80% of the orientation selective neurons in the primary visual cortex.
A mathematical model of non-CRF inhibition is presented. Non-CRF inhibition acts as a feature contrast computation for oriented stimuli: the response to an edge at a given position is suppressed by other edges in the surround. Consequently, it strongly reduces the responses to texture edges while scarcely affecting the responses to isolated contours. The biological utility of this neural mechanism might thus be that of contour (vs. texture) detection.
The results of computer simulations based on the proposed model explain perceptual effects, such as orientation contrast pop-out, "social conformity" of lines embedded in gratings, reduced saliency of cof the sensory neuron for an incoming stimulus. The biological neural network of the wind-mediated escape is also described and compared to two neural network models which will be the basic design of our model. The aims of my PhD are to find online adapting rules for the neural network through which the robot will be able to adapt to malfunction of some artificial hairs or electronic sensory neurons and to investigate how the invertebrate is able to discriminate, when its hairs are moving, between a predator wind flow and self-induced movement. This also raises questions about proprioreceptor information and internal efferent copy. Finally, I am expecting to complete the system by combining other sensors to produce complex behaviour.
Jafreezal Jaafar, Autonomous Virtual Agent Navigation in Unknown Virtual Environment using Fuzzy Logic
The aim of the study is to improve the performance of the autonomous navigation of a virtual agent using a fuzzy logic approach. We focus on developing a fuzzy controller to control and coordinate the navigation behaviour of a virtual agent which is able to take decisions autonomously. The two behaviours are goal seeking and obstacle avoidance. Two main problems had been identified: (i) Behaviour coordination problems, which can be split into two main sub-problems: how to decide which behaviour should be activated at each instant, and how to combine the results from different behaviours into one command to be sent to the virtual agent; (ii) The navigation animation problem is to produce smooth motion of the virtual agent. The system requires producing a navigation path; a safe path will be chosen and executed in real-time.
In this talk, I will introduce problems in behavioural animation for autonomous virtual agent navigation, discuss some related work and present a proposed solution.
Jay Bradley, Blunderings in Function Approximation With Reinforcement Learning
Simple reinforcement learning uses a state-action lookup table to determine the value of performing a particular action from a particular state. With any reasonably complex environment and especially when other agents are included in the state representation the size of the state space becomes too large to practically keep in tabular form.
When this is the case the state-action-value table can be approximated using a number of methods. I shall briefly discuss using a weighted linear combination of the state-action vector and a feed-forward neural network to approximate the state-action-value table.
Paul Crook, Active Perception and Reinforcement Learning
I will present an overview of my PhD thesis which (i) looked at what the addition of active perception brings to reactive, reinforcement learning agents, (ii) considered the issue of convergence of reinforcement learning to satisfying policies on non Markovian tasks. I will also briefly outline the reinforcement learning code that I developed as part of my research, and which I have made freely available in the hope that it will be of use to others working in this area.
Scott Blunsden, Recognition of Coordinated Multi Agent Activities
The problem of identifying coordinated group activity has received relatively little attention compared to the many approaches of recognising individual actions. Within this talk a method for classifying coordinated team activity within the sports domain is presented. Specifically the game of Handball is used as a dataset for identifying team activities (such as attacking/defending).
Looking at the group as a whole provides a different approach to tackle the problem then many previous attempts. This talk will present results and hopefully suggest some future enhancements to this method.
Narayanan Unny E, Regression using independent local models
Traditionally function approximation has been considered in static environments. Conventional regression techniques are inadequate when dealing with dynamic environments. In this context, I will be discussing how independent local models can adapt to such conditions. I will also discuss the probabilistic model that has been developed adhering to the principle of independent localised learning. Apart from discussing the workings of the model called "Randomly Varying Coefficient" model, I will also present some experiments to demonstrate its performance and compare it with conventional learners.
Tim Hospedales, Integration and Segregation in Multi-Modal Perceptual Inference
Statistically optimal multi-modal perception is a hot research topic in the context of human perception as well as that of building effective machine perception systems. I will show some results from learning in my unsupervised model for multi-modal integration. Next, I will discuss some motivation for multi-modal segregation, ways to achieve this in theory, and show some results for toy models to illustrate where I hope to go next.
Mykel Kochenderfer, Adaptive Modelling and Planning for Learning Intelligent Behaviour
In reinforcement learning problems an agent receives reward while interacting with the world. The objective of the agent is to maximise its expected accumulation of reward. Typically, the agent does not possess a complete and accurate model of the world, requiring the agent to generalise from its own experience to produce a competent plan.
This talk introduces the Adaptive Modelling and Planning System (AMPS) as a novel framework for integrating modelling and planning in problems with large state and action spaces. The approach involves dynamically partitioning the state and action spaces by splitting and merging clusters as the agent accumulates experience. AMPS maintains an abstract model over the clusters and uses this model to produce a reactive plan. Because the agent must make decisions in real time with limited processing power, AMPS prioritises revisions of the model and plan.
The first part of the talk begins with a brieworking in these scenarios need to build a representation of the environment as they explore it looking for potential hazards (e.g. mine removal, search and rescue, surveillance, etc.). In these scenarios, typically the communication between robots is possible only within a limited range.
In this talk I will introduce the BERODE architecture. BERODE is a novel distributed approach to explore an environment in a coordinated fashion while maintaining a team of robots as a fully connected network. The approach integrates the scalability and robustness from distributed architectures and the efficiency from centralized approaches. BERODE is based on behavioural roles. These roles reactively adapt to the dynamic conditions of the communication network formed by the robots as they explore an environment. The communication network is maintained as a single network by building and maintaining a control network. The control network is a sub network of the communication networkervous system.
Georgios Petkos, Learning multiple models of non-linear dynamics for control under varying context
Learning dynamics for control is a well studied problem. However, the dynamics of the environment that the system has to interact with or even of the system itself are often changing in a rapid or discrete way. Classic adaptive control techniques may not be appropriate in such cases. A reasonable solution is the use of multiple models.
In this talk, I will discuss the challenges that an ideal multiple model paradigm for control presents and briefly present existing multiple model paradigms. I will also present a variant of one of the existing multiple models paradigms that is able to learn online multiple models of non-linear dynamics for control. Finally, I will sketch future directions for this work.
Mohammed Bennamoun, 3D Model-based Object Recognition and Segmentation in Cluttered Scenes
Viewpoint independent recognition of free-form objects and their segmentation in the presence of clutter and occlusions is a challengl/image processing and computer vision.
Rowland Sillito, Towards Semi-supervised Multiscale Trajectory Classification
My project seeks to investigate the benefits of incorporating labelled data into procedures for learning to identify anomalous behaviour from motion trajectories, using a multiscale trajectory representation approach. I shall describe a few preliminary results obtained in the rather short time-period elapsed since my proposal talk. Firstly, I shall show how classification performance changes as a function of the quantity of labelled training data provided. Secondly, using a semi-supervised self-training procedure for incorporating unlabelled data into the original classifier, the number of labelled examples (and therefore user interventions) required to achieve the same level of performance is quantified, and found to be relatively low. Some potential pitfalls of the approach will then be discussed, along with the future trajectory of this work.
Matthew Szenher, Range-only Homing
Typical robot navigation algorithms seek to determine the position of the robot in its environment in order to calculate the direction and distance to a goal position. Robotic homing avoids explicit location information, answering the question "How do I get to the goal?" without answering "Where am I now?" Most robot homing schemes use visual input, deriving goal direction from changes in landmark bearing. Here, I investigate homing using only landmark range information. I cast the problem as one of function minimisation and leverage knowledge from the vast body of optimisation literature.
Darren Smith, Context Generalisation in a Simulated Mushroom Body
The mushroom body, a central neuropil of the fruit fly, is strongly associated with olfactory learning and memory. Other evidence suggests a role for visual context generalisation - a behaviour ability which allows the fly to identify the cues in its environment which predicpredict arrival of punishment/reward. In this talk I'll be describing progress made on implementing a simplified model of the mushroom body for achieving this learning behaviour.
Adrian Haith, Adaptive Control Architectures for Gaze Stabilization
Oculomotor control has proved to be a good test-bed for theories of vertebrate motor control. The underlying physiology has been extensively studied and the control problem is relatively simple with few degrees of freedom and neglible inertia. Even so, the system must deal with significant non-linearities and large processing delays. I will compare two models of gaze stabilization (the vestibulo-ocular reflex). Each consists of a quick but coarse brainstem pathway augmented by a slow but more versatile adaptive pathway via the cerebellum. The conventional model employs a feedforward architecture for the cerebellar pathway, but recent work by Porrill and Dean has highlighted the advantages of an alternative, recurrent architecture. I will compare performance of the two and discuss the possible implications.
Finlay Stewart, Sensory integration in free-flying Drosophila
When fruit flies are put in an arena containing a concealed attractive odour source, they are only able to locate the source if the walls offer vertical visual contrast. In this seminar I shall outline my plans to investigate this curious finding through a combination of flight-tracking experiments and modelling. Focusing first on the issue of visual course guidance, I'll discuss how the fly is thought to detect and avoid obstacles by computing optic flow information, and how certain visual environments can upset this behaviour. I shall go on to present hypotheses as to how olfactory input may modulate this collision-avoidance response.
Thor List, Simulating a Vision System
To enable testing of diverse control strategies in a computer vision system, a generic reference architecture must be created, which adequately simulates common functionalities found in vision systems. The application chosen is one of automated surveillance, which will attempt to acquire, track, classify and recognise moving objects in the scene. Each module will simulate the computation of either features or output results, and each must be monitored in terms of time, performance, stability, quality and error during increasing system load. Quality and errors will be simulated probabilistically.
Matthew Howard, Learning Utility Landscapes for Humanoid Control and Planning
Humanoid robots are by nature highly redundant systems, both at the kinematic control level and the task execution/planning level. In order to resolve this redundancy it is frequently required that some cost-function is defined by which movements can be optimised. In this talk I will discuss how the cost-based approach can be applied both as a null-space constraint on the movements of redundant manipulators as well as tool for trajectory generation/planning in the manipulator's task-space. I will then present some preliminary work on incremental learning of utility landscapes for use on the humanoid robot ASIMO.
Neil Robertson, Intelligent Video Surveillance
Our work is concerned with producing high-level descriptions and explanations of human activity in video from a single, static camera. The scenarios we focus on are urban surveillance and sports video where the imaged person is medium/low resolution. The final output is text descriptions which not only describe, in human-readable terms, what is happening but also explain the interactions which take place. The input to this reasoning process is the information obtained from action/behaviour recognition algorithms, which is an abstraction from the image data to qualitative descriptions. Causal explanations of global scene activity, particularly where interesting events have occurred, is thus achieved using an extensible, rule-based method. The complete system represents a general technique for video understanding.
Matthew Whitaker, Evolving Morphology and Control for Modulr vision and content-based image retrieval
Recent work in this area has produced a range of shape matching algorithms which can be used to assess shape similarity. However, few techniques can handle occlusion or operate on discontinuous boundaries - both of which arise when dealing with real images. I will describe some probabilistic techniques which do work in these cases. The basic approach can be seen as a probabilistic iterative closest point algorithm for matching point sets. This can extended in a variety of ways to handle more interesting problems. Specifically, I will discuss how part-based shapes can be compared, and also, how to proceed when one of the shapes is represented by a set of line segments (e.g. when matching a line drawing to real images in a "query by example" scenario).
Hugo L. Rosano, The Control of Turning in Real and Simulated Stick Insects
A non-trivial problem even for movement on flat surfaces is how insects manage to turn in such a way that almost all body trajectories are possible. Collaboration of all legs for turning has been suggested for the cockroach; however, difference in walking dynamics and morphologies between insect species might result in different leg roles. We observed the turning response of stick insects presented with an attractive visual target. We show that the front legs play an importanis work, and suggest what advances in understanding will be needed to build successful learners in real-life environments.
Short Biography: Michael Littman is director of the Rutgers Laboratory for Real-Life Reinforcement Learning (RL3) and his research in machine learning examines algorithms for decision making under uncertainty. After earning his Ph.D. from Brown University in 1996, Michael worked as an assistant professor at Duke University, a member of technical staff in ATandT's AI Principles Research Department, and is now an associate professor of computer science at Rutgers. He is on the executive council of the American Association for AI, the advisory board of the Journal of AI Research, and serves as an action editor of the Journal of Machine Learning Research.
Craig Robertson, Dynamic body model fitting using particle swarm optimization
This talk presents a novel evolutionary approach for fitting skeletons to 3D data acquired from moving human models. We apply the particle swarm optimization technique to fitting a 24 parameter skeleton model to acquired 3D data from a human model. Although model fitting to a moving sequence is known to be a difficult problem, we show that because knowledge is preserved in the system between successive frames, this technique can also be used for robust tracking.
Toby Collins, Facial Dynamics for Identity Recognition
The aim of this Ph.D project in a nutshell is to reresearch and develop algorithms for automatically recognising human identity using observed 3D facial motion. Specifically, we aim to validate the following hypothesis: Using a video-rate 3D system capturing a person's face undergoing some set of facial motions, it is possible to extract a 'facial motion signature' from the data which is both unique to that person and reproducible. Furthermore, this signature will be more robust and discriminatory than previously attempted facial motion-based identity systems. The human face has been the subject of tremendous scrutiny in the fields of human cognition, computer vision, image processing and computer graphics, and has been motivated by some very desirable applications, which consider both its static nature, such as identity recognition, and its dynamic nature, such as expression recognition, visual speech recognition and the realistic animation of virtual humans. Its study from a computational perspective has resulted in increasingly more sophisticated tools in non-rigid object modelling and tracking, object parametrisation and recognition, occlusion handling and extracting invariant features in real-world settings.
Much of facial analysis has been conducted on static 2D or 3D images or short 2D image sequences. However there has been very little work in investigating facial dynamics in video-rate 3D data. The advantages of 3D over 2D data in pattern recognition tasks have been largely considered as a means to overcome variations in pose and illumination. However, 3D information over time also provides us with a complete description of how an object deforms in 4D spatiotemporal space without the loss of information which is incurred as part of the 2D image projection process. One important use for this is to study the ways in which individuals can, or are able to deform their face while performing expression or speech. In this thesis presentation, we propose how the similarities and idiosyncrasies of facial motion across individuals can be analysed, and show how individuals can be characterised based on their facial motion. We will state how we can quantify the similarly between two peoples' smiles, for example, and determine which expressions best discriminate individuals. We will show how it is possible to build a prototypical model of how people perform expressions and how individual differences are reflected as a deviation from this model. These ideas have received a very small amount of attention in 2D image analysis, and none in 3D video.
