You are here: Home Cognitive Modelling … Cognitive Modelling Challenge

Cognitive Modelling Challenge

Currently, there are at least twelve theories describing syllogistic reasoning (Khemlani & Johnson-Laird, 2012). These theories entail predictions which are partly mutually contradicting. Still, there is no established measurement to evaluate the theories in terms of criteria like explanatory power, response prediction and simplicity. Cognitive modelling is a recent field of computer science aiming at formally modelling human's responses to reasoning tasks. From these models, we expect to eventually gain insights into the actual cognitive mechanisms of human cognition. 

That is why we have established a benchmark incorporating key results from psychological experiments such as the percentage of the given response. In this year's challenge, we want you to program a model which predicts the response data from human participants. To evaluate different cognitive models several accepted methods from mathematical psychology and artificial intelligence exist. Theory predictions can be modeled by multinomial process trees (e.g., Oberauer 2006; Klauer et al., 2007; Ragni et al., 2014). The model itself can be modeled by using AIC, BIC, and Fishers Information Approximation. These can be with small effort evaluated using packages in R like MPTinRWe will evaluate the models and determine a winner. 


Conditional Reasoning

Syllogistic Reasoning

Relational Reasoning



  • Anderson, J. R. (2007). How can the human mind occur in the physical universe?. Oxford University Press, USA.

  • Byrne, R. M. (1989). Suppressing valid inferences with conditionals. Cognition31(1), 61-83.

  • Goodwin, G. P., & Johnson-Laird, P. N. (2006). Reasoning about the relations between relations. The Quarterly journal of experimental psychology59(6), 1047-1069.

  • Goodwin, G. P., & Johnson-Laird, P. N. (2008). Transitive and pseudo-transitive inferences. Cognition108(2), 320-352.

  • Klauer, K. C., Stahl, C., & Erdfelder, E. (2007). The abstract selection task: new data and an almost comprehensive model. Journal of Experimental Psychology: Learning, Memory, and Cognition33(4), 680.

  • Oaksford, M., & Chater, N. (2007). Bayesian rationality: The probabilistic approach to human reasoning. Oxford University Press.

  • Khemlani, S., & Johnson-Laird, P. N. (2012). Theories of the syllogism: A meta-analysis. Psychological bulletin138(3), 427.
  • Khemlani, S., Lotstein, M., Trafton, J. G., & Johnson-Laird, P. N. (2015). Immediate inferences from quantified assertions. The Quarterly Journal of Experimental Psychology68(10), 2073-2096.
  • Klauer, K. C., Musch, J., & Naumer, B. (2000). On belief bias in syllogistic reasoning. Psychological Review, (4), 852-884.
  • Ragni, M., Khemlani, S., & Johnson-Laird, P. N. (2014). The evaluation of the consistency of quantified assertions. Memory & cognition42(1), 53-66.
  • Ragni, M., & Knauff, M. (2013). A theory and a computational model of spatial reasoning with preferred mental models. Psychological review120(3), 561.