Connectionist Structure 2

Inference Using RAAM

• Representations from RAAM network as inputs to a separate auxiliary feedforward network which transforms and analyzes them (Chalmers, Blank et al.)
• Combining RAAM and auxiliary network (Chrisman)
  • Two RAAM networks share a single hidden layer
  • Subtree training on separate networks
  • Training on "inference" task with entire trees
  • Applied to translation: hidden layer as "interlingua"

Convolution Associative Memory (Role Binding)

• Encoding (binding): 2 item vectors (e.g., role, filler) -> memory trace
• Decoding (unbinding): memory trace and single item (cue) -> item associated with cue
• Composition: combining associations in a single trace
• Capacity: number of associations that can be represented
• Tensor product approach (Smolensky)
  • Encoding (binding): tensor product (generalized outer product)
  • Decoding (unbinding): inner product of cue and trace
  • Composition: tensor addition
  • Size of trace increases exponentially with the depth of the structure being represented
• Holographic reduced representations (Plate)
  • Encoding (binding): circular convolution
  • Decoding (unbinding): circular correlation

    

  • Composition: vector addition
  • Size of trace is constant as depth of structure increases
  • For correlation to decode convolution, elements of each vector must be independently distributed with mean zero and variance 1/n
  • Convolution trace stores pairs only with enough information to discriminate item from others; a cleanup memory (e.g., a Hopfield net) is required if an accurate output is needed
  • (Some problems: types and tokens, microfeature representation)

Elman

• The technique: differs from RAAM and convolution approaches in that it does not assume the structure in the input initially but instead discovers it
• The larger claims
  • Context-independent meaning is an ideal which may only be approached
  • Processing a sentence means following a trajectory through a multi-dimensional state space with specifying the path.

Synchrony Approaches

• In addition to activation, each unit has another property (phase angle, firing time, etc.)
• Units which are "synchronized" on this second property are "bound" together
• Localized models (Shastri and Ajjanagadde, Hummel and Holyoak, etc.)
  • One node per concept
  • Spreading activation, constraint satisfaction, settling
  • Hard-wired
  • Separate role and filler units, bound together through synchronization
  • Instantiation: activation across existing concept nodes (problem of multiple instantiations) or new network structure
• Distributed model (Gasser and Colunga)
  • Relations are patterns of activation across micro-relation units, each with its own micro-roles bound to object feature units through synchronization

[L645 Syllabus]