You are here: Home › Teaching › Cognitive Reasoning: Methods, …

Cognitive Reasoning: Methods, Algorithms, and Statistics to Discern Human from Artificially Generated Data (WS 2019/20)

Organizer: Marco Ragni
Assistants: Nicolas Riesterer, Daniel Brand

Objective

The ability to interpret data based on one glance helps computational modeling by allowing results to be inspected easily. For example in image classification, we can quickly check if a model's predictions are suitable just by looking at some example images.

In a lot of domains, inspecting data is not as easy, if possible at all. For instance in cognitive science, data is usually collected via questionnaire-based experiments. The resulting datasets consist mostly of categorical features with no apparent relationships. This has the unfortunate consequence that direct interpretation of participant responses (e.g., to determine unconcentrated participants jeopardizing the quality of the data) is impossible, even for experts in the domain.

The seminar investigates the interpretability of data in the field of cognitive science. In particular, the seminar participants will attempt to solve a classification task in which given datasets true human data is to be differentiated from artificially generated data. To this end we provide data from the field of human syllogistic reasoning. Syllogisms are quantified categorical assertions (using quantifiers All, Some, Some ... not, and No) relating three terms via two premises. Consider an example:

All A are B
Some B are C
What, if anything, follows?

The goal in the domain of syllogistic reasoning is to select one of nine possible responses which follows from the premises (including "No Valid Conclusion" indicating that no logically valid conclusion can be inferred from the premises).

By investigating profiles of reasoning behavior, the goal of the seminar is to investigate different ways to interpret this kind of behavioral data in order to determine whether given data was artificially generated or obtained from human reasoners. The results are to be discussed with respect to the question how interpretability in categorical data modeling can be enhanced.

Background Literature

  • Khemlani, S., & Johnson-Laird, P. N. (2012). Theories of the syllogism: A meta-analysis. Psychological bulletin, 138(3), 427.

Important Dates

  • October 21st, 13:00-14:00, building 052, room 02-017: Introductory meeting
  • October 30th: HisInOne registration deadline
  • November 19th, 15:00-16:00, building 051, room 00-031: Midterm presentation
  • December 15th, 23:59: Deadline for final models & written report
  • December 20th-21st, 10:00-17:00, building 51, room 00-031: Blockseminary

Requirements

  • Presentation of your preliminary & final results
    • Theoretical and computational foundation
    • Predictive performance
    • Ideas for improvement
  • Written report of your work (~6 pages, CogSci format)
    • Introduction/Motivation
    • Theoretical Foundation
    • Method/Model
    • Results
    • Conclusion/Discussion

Materials