Group Decision-Making in a Signal Detection Task

 

I wanted to see how groups of humans made decisions, so I set up a collaboration with Professor Miguel Eckstein from the Psychology Department. We analyzed data from an actual experiment in which groups of three human observers first performed the same signal detection task individually, then conferred to arrive at a group decision.

 

We investigated five main subjects. First, we were interested in comparing the performance of the individual observers with the performance of the group, to see if the group out-performed its best member. Second, we investigated different ways of visualizing the data, to see what additional information could be gleaned. Third, we wanted to know what strategy or strategies the groups used to arrive at a group decision given the individual decisions. We investigated this in both a top-down and a bottom-up fashion: we regressed the weights actually applied to each observer in different windows of time, and also applied eleven different group rules in a fusion center-based setup to the individual data to see which rule produced fusion center responses that were most similar to the human group’s responses. Fourth, we were interested in seeing if the results found in our original study (600 trials) held over a longer period of time, and addressed this by analyzing the data in a continuation study in which one group returned for an additional 5,400 trials, for a total of 6,000 trials. Finally, we explored some common assumptions taken in modeling, such as that the observers have a fixed sensitivity over the course of the experiment, through a series of simulations.

 

 

M. Kimura, 2011

Sequential Probability Ratio Test (SPRT)-based Group Decision-Making Models

From law-making to the research publication process, we rely on groups to make our most important decisions. Our goal is to gain a better understanding of the performance of a group of cybernetic (human or device) decision-makers (DMs) in a simple decision-making task.

 

In this project, our goal was to derive the performance of a group of DMs in a sequential two-alternative forced-choice task, given only the performance of the individual group members. Our setup is shown below. We developed intuitive mathematical formulas for the group's performance under two previously-proposed group rules, the Race and Majority Total schemes, and a novel rule, the Majority First scheme. In our examples, we modeled the individual DMs with the Sequential Probability Ratio Test (SPRT)-based Drift Diffusion Model (DDM), an optimal and biologically relevant method for choosing between two hypotheses. We then verified our analytical results with simulation. We find our models to be more general than previous works because we derive explicit formulas for the full probability distribution function of group decision times and group error rate for a group of any size using each rule, while only requiring that the group members be independent. We also compared the relative merits of the different rules, demonstrated that our models can be used to accurately predict the performance of a non-identical group, and used a simple example to show the robustness of each group rule to errors in the group members' parameters.

 

Our models are flexible enough to accommodate numerous hierarchical group topologies and group rules, and can be applied to groups of devices, groups of human or monkey observers, or cybernetic groups. Additionally, our models and analysis establish a way to objectively and quantitatively compare different group decision rules and model group performance in an intuitive manner that is accessible to a wide range of communities.

research

In modifying a known coupled oscillator control law for three vehicles, we found a multitude of interesting Spirograph-like trajectories. In my Master’s thesis, we used a dynamical systems-style bifurcation-based analysis to characterize the vehicles’ behavior and gain an intuitive understanding of the underlying mechanism driving the trajectories. We investigated the dynamics of the system for both all-to-all coupling and the “Arbiter” heterogeneous coupling topology shown at right. Some examples of the types of interesting trajectories we found are shown below, for one vehicle. In each case, all three vehicles trace out the same pattern.

Novel Vehicular Trajectories for Collective Motion from Coupled Oscillator Steering Control  

Resume [pdf]

CV [html] [pdf]

Illustration of how our group models are organized. Each DM takes and processes independent and identically distributed (iid) observations of the source, which represents the correct hypothesis. The observations are iid both within and across DMs. Once a DM makes a decision, it sends that decision to the Fusion Center, which then applies the group decision rule and issues the group’s decision once the group rule’s end condition has been satisfied.

An example stimulus that was presented to the observers in the individual phase of the signal detection task. The observers viewed the stimulus for 1.5 seconds. The signal, if present, is a white Gaussian in the center of the cue (black box). In this case, the signal is present. A reference image of the signal is shown at right.