It is often desirable to share resources (hardware) because this
permits generalization.
Combination is by superposition, rather than concatenation.
Connections between units are relatively fixed.
Challenges
Which features are bound together as objects?
Which objects are bound together by relations?
How are representations of larger objects built from
representations of smaller objects?
How are more complex relations built from simpler ones?
How are instances of the same concept which are being considered
simultaneously kept track of?
How are relations or objects at different levels of complexity kept in
short-term memory?
How can the same hardware be used to represent different levels
in a part-whole hierarchy? (Hinton)
Given a representation for a relation and a role in it, how
is the filler of the role in the relation extracted?
Given a representation for an object and a feature, how is the value
of the feature in the object extracted?
Given representations for facts, how are they to be
transformed systematically into other ("inferred") facts?
Approaches
Assume a fixed-valence tree,
treat each of the terminals as a vector, and build a
machine that can encode and decode vectors representing the tree
structure.
Assume a frame-like structure,
bind roles and fillers by "multiplying" the corresponding vectors,
add these products to get a representation for the whole frame.
Unbind using the inverse of the "multiplication" operation.
Implement feature binding by treating units with the same "phase"
as representing features associated with a single object.
Present an input sequence embodying some sort of structure to a
trainable sequential network with an output task which requires it to
derive the structure. Hope for the best.