Agile Research

I was lucky to have the chance to participate in the XP2012 conference which was held last month at Malmo Sweden. One thing that makes this conference special is that the conference is organized in a joint effort with universities. The basic idea (which is great) is to bring together practitioners from the industry along with academicals representatives in order to combine theoretical knowledge with real life experience. and make both share their knowledge.
The thing that did strike me weird however was that all the professors and PHD students I met, came from Exact Science departments, that is all of them were majoring/researching in some field related to computer science, software engineering or something similar.
Why is that weird you might ask. Well it was weird to me because most the things worth studying in agile are not technical in nature, they relate to human behavior, interactions team dynamics and general communications. I would expect that in order for these studies to be useful, they should be assisted by some people specializing in this kind of research. I would expect such research to be done in collaboration with phycology departments with business school with behavior specialist and even social experts. Computer science researchers (for most) simply don’t posses enough background to actually conduct this kind of research.
For example:
I attended one of the session which introduced such the following study:

Impact of Test Design Technique Knowledge on Test Driven Development: a controlled experiment

as the name suggests the goal of the study is to judge if concrete background of testing theory will change how TDD is practiced. Now don’t get me wrong, the assumption checked is very interesting and the experiment was done according to proper standards. However there was at least one little problem. The subject of the experiment were under graduate student undergoing a programming course as part of their studies.
Which raises the following question:
Under what conditions would someone consider a group of students to be a fair representatives of the general developer community?
Are we seriously going to take undergraduate students (which at most had a course about TDD and most likely just an intro session of 2 hours) and treat anything that might happen to them as useful evidence applicable to the general group of developers?
I think this is probably a very big mistake.
so how does this example relates?
During the conference I heard a nice story about some behavior studies conducted in the US sometime in the 70’s. It appears that those studies which later became the foundation of the field, suffered from a similar symptom. Most of them were done on subjects which were mainly young adult white males in the age range of 20-30. Naturally it was later found out that conclusions drawn from that research group were not very applicable when facing the wider general population and they were a little bias in their finding.
Are we sure its wise to repeat that mistake yet again?

Posted in: Research