Volume 14: pp. 19–24

All Hail Suboptimal Choice! Now, Can We “Fix” It?

Michael J. Beran

Georgia State University

Reading Options

Continue reading below, or:
Read/Download PDF | Add to Endnote


Abstract

Zentall (2019) describes cases in which nonhuman animals show interesting failures in some kinds of choice tests. The failures are particularly valuable, he argues, for understanding the nature of choice behavior and why it may be adaptive in some contexts but then necessarily may look suboptimal in others. I agree that these are interesting test cases. I discuss some of the ways in which the presented results converge among themselves, and with some other choice tasks, on the idea that choices are made in contexts, and those contexts play as large a role as do the actual choice options themselves. Framing effects, temporal discounting, and motivation levels of choosers all lead to choice behavior that reflects bounded rationality, just as is true for humans. In this way, suboptimal choice is natural to expect in some instances, and potentially can be offset by manipulations to the environment in which the choice is made.

Keywords: suboptimal choice, delayed reward, less is more, impulsivity, ephemeral reward

Author Note: Michael J. Beran, Department of Psychology, Georgia State University, P.O. Box 5010, Atlanta, GA 30302-5010.

Correspondence concerning this article should be addressed to Michael J. Beran at mberan1@gsu.edu.

“Sometimes, you understand a thing best when you figure out why it is broken.” There is no famous scientist, or writer, or inventor (that I know of) whom I learned that from. That was from my dad. He said something like that when I was about 15 years old (goes to show you that, like most teenagers, I was probably only half paying attention). I imagine he was trying to teach me that when things fail, and you figure out why they failed, that actually tells you how they otherwise normally work. It may have taken another 15 to 20 years before I saw the application of that in my research, but I finally did. In my case, it happened when I watched chimpanzees fail at something that should have been so easy, at least in my mind. And I was not alone, as this was the reverse-reward contingency test (i.e., the “Boysen inhibition test”). You present two choices, and your subject gets whatever they do not point toward. Simple enough, one would assume, but it turns out that chimpanzees have a very hard time with this (Boysen, Berntson, Hannan, & Cacioppo, 1996). Even if you can show that they can do a lot of interesting things that one would imagine are required to pass this test (such as choosing to point at one item to add it to a bigger set of items rather than pointing at the bigger set), they still fail this basic choice test in which they have to point at a smaller amount (Beran, James, Whitham, & Parrish, 2016). Other animals fail, too, except in some specific situations that punish failed responses or that involve extensive training (see Shifferman, 2009, for an overview; see Beran, 2018, for a longer discussion on this and other tests of inhibitory control and self-control in animals).

Zentall (2019) has compiled a list of other suboptimal behaviors in nonhuman animals, and he has pointed out that these errors are a window into understanding the evolution of behavior and cognition in the species that make these errors. It is an interesting article for bringing together these different kinds of tests under the “umbrella” of looking closely at suboptimal choice (“errors”) as being just as valuable a topic for study as are reports of new successes in nonhuman species. This article continued an earlier contribution (Zentall, 2016) that looked at three other errors in choice behavior, and as someone who has also studied some of these suboptimal choices in nonhuman primates, I find these two articles to be a nice introduction and summary of that research. Cognitive psychology has made a major shift to seeing heuristics and cognitive biases as important and necessary topics for research, and this is true in comparative psychological research, as well.

As Zentall (2019) notes, it is not always easy to know whether a choice is rational/optimal. But there are some cases in which choice behavior, at face value, certainly looks at odds with any definition of what rational or optimal responders should do. For example, if I offered you a candy bar or the same candy bar plus an apple, and you chose the candy bar only, just about anyone would ask, “Why choose that? You could have had the candy plus an extra apple.” But people will make choices like this, leading to what has been called the “less-is-more” bias. I was certain that chimpanzees would not also do this, but as with the reverse-reward task, I was again wrong (Beran, Ratliff, & Evans, 2009). When we offered chimpanzees the choice of a piece of banana or an identically sized piece of banana plus a slice of apple, they did not show the strong bias for the latter option that they should have if they were solely concerned with maximizing food reward. This intrigued us, and frankly surprised me a lot, given that we know that chimpanzees are excellent at telling apart even small differences in food items (Menzel, 1960, 1961) and telling apart small differences in quantities of discrete items (Beran, 2004; Rumbaugh, Savage-Rumbaugh, & Hegel, 1987). Yet, in this test, some chimpanzees preferred the single food item over two food items. However, this pattern of choosing can be fairly easily manipulated depending on other aspects of the testing environment. In our case, we considered the basic preferences of the chimpanzees. Banana was always preferred over apple, although they would eat apple when they had it. In our tests, chimpanzees had to eat everything they had before we would set up a new trial, and so we hypothesized that the choice of the large amount of food actually slowed the pace at which the chimpanzees could get their next bite of banana, because they had to eat the lower preference apple pieces. When we inserted a delay between trials, so that now there was plenty of time to eat one or two pieces of food and choosing the set with apple did not have any effect on trial pacing, responding become more “rational.” This result aligns very nicely with Zentall’s discussion of motivation in pigeons. When pigeons were more food deprived, they responded more optimally versus when they were less food deprived (Zentall, Laude, Case, & Daniels, 2014). Together, these results suggest that the choice options, viewed in a vacuum, should lead to clear predictions for what should be chosen on the basis of only maximizing reward, but in the real world, other factors frame those choice differently. We know this is true for many choice situations for humans, where framing effects and the individual’s perspective at the time of that one choice can radically affect the choice that is made (Kahneman, 2011; Tversky & Kahneman, 1974).

This idea of context playing a role in choice comes from another study with great apes. Sánchez-Amaro, Peretó, and Call (2016) attempted to replicate our findings (Beran, Ratliff, & Evans, 2009) but largely failed to do so. Instead, they found that their apes were not able to overcome less-is-more errors even with longer delays between trials. Their apes were more likely to select the greater overall amount of food (i.e., chose optimally) when the mixed option was composed of items that were closer in value compared to when items were more discrepant in value. This perhaps made the task closer to a true quantity judgment task, allowing the apes to choose in a way that maximized their reward. It is also possible that these apes, unlike the apes we worked with and who had very different testing histories, did not recognize (or perhaps care) that they could control trial timing, but they did recognize (and care) that higher preference food items were being made available. The point here is that the test itself is informative only when it is viewed in the context of the animal being tested—and in terms of that animal’s own “frame” from which it engages the task. To me (and, I think, to Zentall), this is a convergence of human and nonhuman animal choice behavior that reflects a kind of heuristic-based, bounded rationality. As Zentall notes, there is not a lot of research on the less-is-more effect in humans, and it would be interesting to modify the typical tests used (e.g., Hsee, 1998) by placing the choices into different contexts. For example, when people choose a smaller overall number of unbroken dishes compared to a larger number of unbroken dishes plus some broken ones, it is not clear how they are anticipating what they will do with the broken dishes. If that choice was made with a trash can or recycle bin available to immediately discard the broken items and then take the remaining items home, I suspect choices would be different. But that remains to be seen.

Another interesting convergence in the discussion by Zentall (2019) comes from the role that temporal delay plays in the ephemeral reward task. He notes that improved success on this task (defined as more choices of the ephemeral reward over the stable reward) comes when one inserts a delay between choice and outcomes (Zentall, Case, & Berry, 2017a, 2017b). As Zentall (2019) noted, these delays have a long history of “improving” choice behavior in areas such as intertemporal choice, where one chooses between a smaller-sooner and larger-later reward (Ainslie, 1974). When both outcomes are relatively close in time (e.g., one reward in 1 s, two rewards in 10 s), many individuals of many species (including humans) prefer the more immediate reward. But if you shift both further into the future (e.g., one reward in 31 s, two rewards in 40 s), choice of the larger-later reward increases. Time matters for intertemporal choices in ways that it should not if it was always experienced in an objective way by the chooser. But it is experienced subjectively, and that means that different contexts lead to different choices. A person with no money for lunch will more likely choose $10 now versus $20 tomorrow, but a person with $200 in his or her pocket may choose the more delayed payment. In both cases, however, pushing the two outcomes into the future (e.g., $10 in a month vs. $20 in a month and a day) seems to increase choices of the larger, more delayed reward. This is an interesting connection to the role that temporal delay may play in mastering the ephemeral reward task (Zentall et al., 2017a, 2017b), just as temporal delay appeared to aid the chimpanzees in the less-is-more test of Beran, Ratliff, and Evans (2009; but also see Sánchez-Amaro et al., 2016).

As Zentall (2019) notes, animals should have a complicated relationship with time. The present is typically much clearer to evaluate than the future. Delay discounting can therefore be seen as adaptive, given that future reward cannot be truly guaranteed, even in laboratory settings. Therefore, impulsivity may reflect some of the rational aspects of minimizing or avoiding delay. Reaching toward more food rather than less food makes sense if one anticipates only what one next wants in one’s hand. Choosing only banana versus banana and apple makes sense if that speeds up the next chance to get banana (i.e., reduces the time to when more banana will come). And if the present is more certain than the future, going toward a stimulus that has a higher immediate probability of reward (the permanent stimulus in the ephemeral reward task) in the present makes sense. When those outcomes are delayed, however, responding changes to a more optimal pattern.

To my knowledge, there have not been many efforts made to manipulate timing factors in the reverse-reward task, but it might be the case that inserting a delay between choice and outcome would improve performance in pointing at the smaller amount. We showed some evidence of a related result when apes had to point to single items (and not sets of items) to accumulate larger rewards (Beran et al., 2016). This is an indirect way of forcing inserted delays into a choice task, but more work is needed on this question. It is interesting that chimpanzees do solve the reverse-reward task when it is modified so that, rather than pointing at foods, the chimpanzees point at Arabic numerals (Boysen et al., 1996; Boysen, Mukobi, & Berntson, 1999). This use of symbolic stimuli seems to produce more optimal responding, even though that success does not transfer to optimal responding when food items are again used. It is also interesting that this result aligns with newer work with nonhuman primates on the ephemeral reward task. Whereas nonhuman primates were originally found to be poor at the ephemeral reward task, modifications that highlighted differences between rewards (by coloring them differently) or making more symbolic versions of the test (through use of computerized tasks) led to better performance (Prétôt, Bshary, & Brosnan, 2016a, 2016b). Here, too, we see a convergence toward optimal responding based on changing task context.

The tasks that Zentall (2019) describes need to be used more broadly, with more individuals and more species. The comparative approach that has been taken with the ephemeral task, in particular, is exciting because one can directly measure choice behavior and make predictions based on other aspects of a species ecology, or on the basis of contextual factors (e.g., Prétôt et al., 2016a, 2016b; Salwiczek et al., 2012). Included in this expanded study of optimal and suboptimal choice should be more emphasis on perceptual factors that may influence choices. We know that nonhumans share with humans a number of illusory experiences that may impact how they even represent the choices presented to them, and this would add another layer to better understanding suboptimal choice behavior. For example, we have also seen that chimpanzees undervalue sets with separated food items compared to stacked food items (Beran, Evans, & Ratliff, 2009); they undervalue food items that are broken compared to those that are whole, despite no difference in how long it takes to eat either type of food (Parrish, Evans, & Beran, 2015); and they misperceive food amounts based on the containers holding those items (Parrish & Beran, 2014a, 2014b). Ultimately, studying a variety of choices presented in different contexts will let us begin to appreciate why a chooser is compelled to choose the way that it does, and from there we can design and test environmental “nudges” that might improve choice behavior in nonhuman animals and in humans.

References

  1. Ainslie, G. W. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, 485–489. doi:10.1901/jeab.1974.21-485

  2. Beran, M. J. (2004). Chimpanzees (Pan troglodytes) respond to nonvisible sets after one-by-one addition and removal of items. Journal of Comparative Psychology, 118, 25–36. doi:10.1037/0735-7036.118.1.25

  3. Beran, M. J. (2018). Self-control in animals and people. New York: Academic Press.

  4. Beran, M. J., Evans, T. A., & Ratliff, C. L. (2009). Perception of food amounts by chimpanzees (Pan troglodytes): The role of magnitude, contiguity, and wholeness. Journal of Experimental Psychology: Animal Behavior Processes, 35, 516–524. doi:10.1037/a0015488

  5. Beran, M. J., James, B. T., Whitham, W., & Parrish, A. E. (2016). Chimpanzees can point to smaller amounts of food to accumulate larger amounts but they still fail the reverse-reward contingency task. Journal of Experimental Psychology: Animal Learning and Cognition, 42, 347–358. doi:10.1037/xan0000115

  6. Beran, M. J., Ratliff, C. L., & Evans, T. A. (2009). Natural choice in chimpanzees (Pan troglodytes): Perceptual and temporal effects on selective value. Learning and Motivation, 40, 186–196. doi:10.1016/j.lmot.2008.11.002

  7. Boysen, S. T., Berntson, G. G., Hannan, M. B., & Cacioppo, J. T. (1996). Quantity-based interference and symbolic representations in chimpanzees (Pan troglodytes). Journal of Experimental Psychology: Animal Behavior Processes, 22, 76–86. doi:10.1037/0097-7403.22.1.76

  8. Boysen, S. T., Mukobi, K. L., & Berntson, G. G. (1999). Overcoming response bias using symbolic representations of number by chimpanzees (Pan troglodytes). Animal Learning and Behavior, 27, 229–235. doi:10.3758/BF03199679

  9. Hsee, C. K. (1998). Less is better: When low-value options are valued more highly than high-value options. Journal of Behavioral Decision Making, 11, 107–121. doi:10.1002/(SICI)1099-0771(199806)11:2<107
    ::AID-BDM292>3.0.CO;2-Y

  10. Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux.

  11. Menzel, E. W. Jr. (1960). Selection of food by size in the chimpanzee and comparison with human judgments. Science, 131, 1527–1528. doi:10.1126/science.131.3412.1527

  12. Menzel, E. W. Jr. (1961). Perception of food size in the chimpanzee. Journal of Comparative and Physiological Psychology, 54, 588-591. doi:10.1037/h0044421

  13. Parrish, A. E., & Beran, M. J. (2014a). Chimpanzees sometimes see fuller as better: Judgments of food quantities based on container size and fullness. Behavioural Processes, 103, 184–191. doi:10.1016/j.beproc.2013.12.011

  14. Parrish, A. E., & Beran, M. J. (2014b). When less is more: Like humans, chimpanzees (Pan troglodytes) misperceive food amounts based on plate size. Animal Cognition, 17, 427–434. doi:10.1007/s10071-013-0674-3

  15. Parrish, A. E., Evans, T. A., & Beran, M. J. (2015). Defining value through quantity and quality—Chimpanzees (Pan troglodytes) undervalue food quantities when items are broken. Behavioural Processes, 111, 118–126. doi:10.1016/j.beproc.2014.11.004

  16. Prétôt, L., Bshary, R., & Brosnan, S. F. (2016a). Comparing species decisions in a dichotomous choice task: Adjusting task parameters improves performance in monkeys. Animal Cognition, 19, 819–834. doi:10.1007/s10071-016-0981-6

  17. Prétôt, L., Bshary, R., & Brosnan, S. F. (2016b). Factors influencing the different performance of fish and primates on a dichotomous choice task. Animal Behaviour, 119, 189–199. doi:10.1016/j.anbehav.2016.06.023

  18. Rumbaugh, D. M., Savage-Rumbaugh, E. S., & Hegel, M. T. (1987). Summation in the chimpanzee (Pan troglodytes). Journal of Experimental Psychology: Animal Behavior Processes, 13, 107–115. doi:10.1037/0097-7403.13.2.107

  19. Salwiczek, L. H., Prétôt, L., Demarta, L., Proctor, D., Essler, J., Pinto, A. I., … Bshary, R. (2012). Adult cleaner wrasse outperform capuchin monkeys, chimpanzees, and orang-utans in a complex foraging task derived from cleaner-client reef fish cooperation. PLoS One, 7, e49068. doi:10.1371/journal.pone.0049068

  20. Sánchez-Amaro, A., Peretó, M., & Call, J. (2016). Differences in between-reinforcer value modulate the selective-value effect in great apes (Pan troglodytes, P. paniscus, Gorilla gorilla, Pongo abelii). Journal of Comparative Psychology, 130, 1–12. doi:10.1037/com0000014

  21. Shifferman, E. M. (2009). Its own reward: Lessons to be drawn from the reversed-reward contingency paradigm. Animal Cognition, 12, 547–558. doi:10.1007/s10071-009-0215-2

  22. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131. doi:10.1126/science.185.4157.1124

  23. Zentall, T. R. (2016). When humans and other animals behave irrationally. Comparative Cognition & Behavior Reviews, 11, 25–48. doi:10.3819/ccbr.2016.110002

  24. Zentall, T. R. (2019). What suboptimal choice tells us about the control of behavior. Comparative Cognition & Behavior Reviews, 14, 1–17. doi:10.3819/CCBR.2019.140001

  25. Zentall, T. R., Case, J. P., & Berry, J. R. (2017a). Early commitment facilitates optimal choice by pigeons. Psychonomic Bulletin & Review, 24, 957–963. doi:10.3758/s13423-016-1173-8

  26. Zentall, T. R., Case, J. P., & Berry, J. R. (2017b). Rats’ acquisition of the ephemeral reward task. Animal Cognition, 20, 419–425. doi:10.1007/s10071-016-1065-3

  27. Zentall, T. R., Laude, J. R., Case, J. P., & Daniels, C. W. (2014). Less means more for pigeons but not always. Psychonomic Bulletin & Review, 21, 1623–1628. doi:10.3758/s13423-014-0626-1