Decision Making and Reward

• Dualism


  Mind and body as two separate entities
• Monism


  Mental processes can be identified with physical properties of the nervous system
• Phenomenalism


  Ultimately, only mental objects exist
• Cartesian theater


  The homunculus argument
• Models of Simple Decisions

  • Sensory processing: oddball detections
  • Sensory processing: motion discrimination
  • Nonsensory information (decision variables)
• Saccade generation and the control of eye movement

  1. Perceptual decision making (Newsome)

  2. Integrative sensorimotor processing (Shalden)

  3. Target selection and reaction times (Schall)

  4. Decision variables and subjective choice (Glimcher)
• Middle temporal area (V5/MT)

  • Extrastriate visual area, connections primarily from V1 and V2
  • Motion sensitive neurons with large receptive fields and preferred velocities (direction and speed)
  • Neurons encode for instantaneous strength of motion in preferred direction
• Lateral Intraparietal area (LIP)


  • Region of lateral bank of the intraparietal sulcus
  • Receives input from extrastriate cortices, including MT; major projections to FEF and SC
  • Topographically organized code for direction and amplitude of saccades
  • Pre-saccadic and peri-saccadic neural activity associated with preferred saccades proposed covert psychological processes in sensorimotor processing: attention, motor planning, and decision
• Frontal Eye field (FEF)

  • Located in posterior to the arcuate sulcus in prefrontal cortex
  • Motor function: microstimulation elicits saccades, pre- and peri- saccadic activity
  • Direction innervation of superior colliculus and other brainstem eye movement centers
  • Visual function: convergence of extrastriate input (MT, LIP, TEO)
  • Stereotyped activity with saccade to target in response field: initial burst of activity at target onset, then low rate into slow ramping activity to a pre-saccadic burst
• Superior colliculus (SC)


  • Locate in the dorsal midbrain
  • Inputs include retinal, striate, and extrastriate, and motor inputs from FEF and LIP
  • Outputs to multiple brainstem areas involved in oculomotor control
  • Retinotopic map of visual space organized into response fields, coding for saccade amplitude and directions
  • Electrical stimulation produces conjugate contralateral saccades
• What should the neural correlate of decision look like?
• What about post-sensory decision-related processing?
• How does decision activity affect action selection?
• What about the role of nonsensory information in decision making?
• Integrative post-sensory processing


  • Integrators should how gradual increase in activity during motion stimulus if that eye movement or 'choice' is ultimately made
  • Rate of increase in activity should be a function of sensory signal (motion coherence): higher coherence → faster rate of rise
  • If a decision is made – even in the absence of a true signal (0% coherence) – activity in the integrator should still reflect the upcoming choice
• Integrative post-sensory processing in LIP

  • Gradually rising activity predicts subsequent saccade towards or away from MF
  • Rate of rice of decision-coding activity dependent on strength of motion stimulus
  • At 0% coherence, despite no sensory signal above noise, neural activity is correlated with the monkey's subsequent choice
• Drift Diffusion Models - evidence accumulation for decision making

  • Accumulation involves both maintaining a memory of evidence accrued so far and adding new evidence to the memory
  • In our task the accumulator's memory was noiseless, for both rats and humans. In contrast, the addition of new sensory evidence was the primary source of variability
• What about more complex decisions?
• What happens if the eye movement has to reflect a more complex decision?
• Two classes of variable that affect Decisions

  • Current sensory information
  • Stored representation of environmental contingencies (expected gains/losses)
• Reward


  • A pleasurable event that follows a specific behavior
  • Can be primary (e.g., food) or secondary (e.g., money), or abstract (music)
  • The brain uses rewards to learn, choose and prepare/execute goals. A pleasurable event that follows a specific behaviors
• The function of reward


  • Elicit approach behavior (either through innate mechanisms or leaning)
  • Increase the frequency and intensity of a behavior that leads to a reward (learning)
  • Induce subjective feelings of pleasure (hedonia) and positive emotional states
• Wanting


  Elicit approach behavior. Motivation for reward, conscious, or unconscious desires for incentives or cognitive goals
• Learning


  Learning the associations, representations, and predictions about future rewards based on past experiences (reward prediction error)
• Liking


  Induce subjective feelings of pleasure (hedonia) and positive emotional states
• Dopamine System


  Stimulation of dopamine pathways is "rewarding (motivating)"
• Dopamine and Reward


  • Many rewards seem to lead to release of dopamine (DA) in the striatum
  • Dopamine is released during basic drives (i.e., hunger)
  • DA is released in the rat Nacc right before and during copulation, but not afterwards
  • DA is released in the human caudate nucleus when presented with food stimulation in a food-deprived state
  • DA is released in both NAcc and caudate when participants are playing video games for money
• Dopamine receptor binding


  • DA binding is lower in addiction
  • Massive and repeated relapse of dopamine makes the system less sensitive to "typical" rewards
• Reward Prediction Error


  • Dopamine neurons are active when drop of liquid is delivered outside any behavioral task
  • Earliest predictor of reward signals dopamine response instead of fully predicted reward
  • Dopamine neurons will be depressed at the time of the predicted reward if it fails to occur
  • DA provides an error prediction signal to aid in goal-directed behavior
  • DA response = reward occurred - reward predicted
• Rewards serve various functions, primarily they serve to drive behavior
• Rewards can be primary or secondary in nature
• Primary = innate in the environment (liquids, food)
• Secondary = learned through associations (money)
• Dopamine is important for reward processing
• Dopamine is released in the striatum during rewarding situations
• One hypothesis about DA function suggests that dopamine codes for prediction errors during learning, helping us understand how to change our behaviors to attain a goal
• BOLD signal in stratum correlates parametrically, trial-by-trial with prediction error (O'Doherty et al. 2003)
• This signal modulated up & down by dopaminergic drugs (Pessiglione et al 2006)
• Prediction error hypothesis of dopamine

  The idea: Dopamine encodes a reward prediction error