As law professors, most of us don't have much (if any) training in traditional social science research methodology like surveys. On the few occasions that I've dipped a toe into those waters (e.g., here), I've teamed with other researchers who have experience with those methods. Apart from lack of familiarity and training, another wrinkle for many law profs is that survey work typically requires approval of the applicable institutional review board. Law faculties are far less accustomed to dealing with IRB approval than, say, medical school faculties.
I was interested to read (here) in today's Inside Higher Education about two researchers at the London School of Economics, Friedrich Geiecke (Computational Social Science) and Xavier Jaravel (Economics), who have created an open access tool with promising survey capabilities: Conversations at Scale: Robust AI-led Interviews with a Simple Open-Source Platform.
Here's a descriptive excerpt from the IHE article by Juliette Rowsell:
Rather than having a standard set of multiple-choice and open text questions, as has typically been the case with online surveys, the chat bot takes a conversational approach, collecting interviewees’ responses and using them to generate new questions within a broad set of parameters. ***
In trials, it demonstrated some impressive results. A team of sociology Ph.D. students from Harvard University and the London School of Economics that was asked to assess the quality of the interviews based on transcripts rated them as being broadly comparable to interviews conducted by human experts.
And when the almost 1,000 study participants were asked to evaluate their interaction with the chat bot, the majority said they had enjoyed it and preferred this mode of interview over open text fields. Only 15 percent of respondents said they would have preferred the interview to have been conducted by a person.
I will be following this technology with interest.
Comments