In 1907, Francis Galton, half-cousin of Charles Darwin, who was an anthropologist and a statistician, observed a curious phenomenon that is called the “Wisdom of the crowd”. During this year, he assisted a game where the players were supposed to guess the weight of an ox just by looking at it. Individually, the responses were generally wrong and far from the right answer. But when he took all the 787 estimations, he observed that the median of those (it means that 50% of the participants answered above and 50% under) was the right answer with only 1% of error.
With this observation he published a paper in Nature in 1907 which was the first about the Wisdom of the crowd.
In the context of the second Biosensors week that focused on force sensing, our team, ScalingHuman, decided to compare the accuracy power of the crowd to an electronic scale to determine the weight of an object.
To do so, we designed a protocol that enabled us to collect the crowd measures as well as the scale ones. We first prepared three different weighed boxes that looked exactly the same, so that their appearance would not interfere in our measures. We then masked the eyes of the person we were testing in order to reduce even more biais. After that, we handed the person a 1kg box, and told her it weighed 1kg in order to “tare” her, just like an electronic scale. At that point the experiment could start: we handed her the other boxes, asked how much she estimated their weight and wrote her response down. We handed 4 times the same box in a random order in order to test people’s precision. In order to reach a so called crowd, we reproduced the protocol on 35 different persons.
Finally, we compared both results and observed that the median and the average of the estimations are relatively close to the scale measurement. The results are shown in the graph below :
In those two graphics , the curves represent the average/median of the estimation made by the scale (red) and the crowd (blue).
We can clearly observe an evolution of the difference between the 2 estimations, it increases as the objects get heavier. We can observe that the average and the median of the estimations are quite close to the measurement made by the scale for light object. As for the 3kg object, the estimation were in average and with the median overestimated (of about 2kg). Our results allow us to make the assumption that the crowd manages to estimate light weights (<1kg) easier than heavy weights (>1kg) .
However, our results could heavily be contested because of the following bias:
The size of our crowd was for sure too small, Galton tested almost 800 persons we tested only 33, and that was not a sufficient sample. Furthermore, we tested only 3 random people on the street, the others being scientific students. That makes a rather homogeneous crowd, while the crowd should be heterogeneous to be relevant. Then, we also observed that the large volume of boxes could have led to biased weight perceptions.
The most significant bias in our experiment seems to stem from the way we collected our data. In fact, for each tested person we reproduced the experiment 4 times for each box, which may be too much. Then, we gave them the different weights randomly, and alternating heavy and light boxes may have altered people’s perception. A study on the influence of the order we follow to give our boxes could also be really interesting.
https://www.youtube.com/watch?v=xtuh5zTa7mQ the video that introduced us to the concept of wisdom of the crowd
: https://www.youtube.com/watch?v=iOucwX7Z1HU : Explanation to the concept and application of wisdom of the crowd
Resources: F. Galton, Vox Populi, Nature, no1949 75, March 7, 1907.
- Weight perception and the haptic size–weight illusion are functions of the inertia tensor. Amazeen, Eric L.; Turvey, M. T. Journal of Experimental Psychology: Human Perception and Performance, Vol 22(1), Feb 1996, 213-232.
Follow us on twitter : @SyllogikiS
Written by : Noémie Paillon, Lucile Szpiro, Léon Grillet and Aurélien DIEHL.
No comments:
Post a Comment