The Faculty Lounge

More Online Teaching Resources

To add to the current compilation of materials on how to move to remote teaching, I wanted to share the following SSRN links.

The first was posted this week by Prof Agnieszka McPeak (Duquesne) and is a straightforward, simple best practices guide to moving things online. Six pages, easy to read and great advice:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3553094

The second is a piece I wrote last year which is longer and focuses more on the costs and benefits of law schools moving to online teaching, but it also includes some practical observations on larger and smaller group teaching online:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3491427

Hopefully some of those struggling to get online quickly may make use of these pieces. I highly recommend Prof McPeak's thoughts.

 

The Great Mass Online Teaching Experiment

For many years, I've been fascinated by the possibilities of online teaching. Things like MOOCs and flipped classrooms seem to offer real opportunities to teach in new and better ways. All the same, I've been busy with other things, and actually making the leap to offering online instruction never quite made the list.

Then came Covid-19. My students are sequestered in apartments all across China, and the Peking University School of Transnational Law campus in Shenzhen is empty. I've chosen not to return to China after the spring festival, but have been hunkered down in Chiang Mai, Thailand, waiting for the situation to gain some clarity. The clarity that has gradually emerged is that we aren't going to be teaching face-to-face for at least a while, and we're taking all our classes online.

In going online, we're joining all kinds of schools all over China. I've got a cousin who has retired from K12 teaching in Canada and is running the international section of a large school on the east coast. They've taken all high school classes online, along with most similar schools. Teachers report that students locked down in house arrest tend to be highly prepared. At an expat event in Chiang Mai, I met two kindergarten teachers from an international school in another part of China. Although the ministry of education doesn't support online teaching for kids that young, in response to parental demands they have been teaching children as young as three years old online. While hugs are hard to deliver, they can record sing-a-long songs or record readings of English books and lessons, and comment on the responses the children's parents sent back to them. Possibly hundreds of millions of students have been shifted to online instruction in China, something that has not happened at that scale in the traditional educational environment anywhere before.

My students, fortunately, are a lot more self-sufficient and able than three-year-olds. It's been a challenge to move everything online, but it seems to be working. Here are some quick thoughts. As my experience deepens, I might be back with more thoughts.

Platforms. There seem to be a lot of good platforms out there. We are using Zoom, in both its webinar (better for classes over 100 in size) and meeting (better for smaller classes) sizes. I find Zoom pretty easy to use, although with each class I find some features and shortcuts I hadn't noticed before that make it still easier. I am able to call on students by name, put them on camera, share their video with everyone, and in general carry on pretty much as we do in a regular classroom. Our budgets tend to be pretty limited, and we aren't using some of the complementary software that enables things like seamlessly moving to pre-recorded snippets. So far, that seems to be ok.

Bandwidth. As you would expect, bandwidth is an issue, both for the students and the teachers. This week in order to renew my Thai visa I took a side trip to Luang Prabang in Laos, where the internet just isn't robust enough for online teaching. To deal with that, I went asynchronous for one class, recording a lecture before I came over. Students are also differently positioned, and some students experience issues with audio quality or dropped connections. To ameliorate, we are recording all classes, and students can download and listen to any difficult portions at their leisure. I find 20 megabits per second connections more than adequate and five megabits per second too low for full functionality. (Type internet speed test into Google to get a test that will tell you your connection speed).

Disadvantages. Some of the issues involved in moving online are obvious, but others less so. For example, in our situation, library reserve - which I use heavily - is not available. In some courses, we have copies of the basic text or core readings for everyone available in the library. That doesn't work when students are scattered. For example, in my Law and Innovation class we have enough copies of the Susskind's The Future of the Professions to go around when we are all in Shenzhen, as well as enough copies of Thomas Morgan's PR book to go around when we cover specific ethical issues in the legal profession. We have no way of getting to those to students all over China. That requires adjustments in the course materials as a small subset of my students can buy whatever book I mention and others are financially constrained, so simply telling them to go to Amazon or JD is not a good solution. In some cases, the only solution has been for me to write substitute materials, which has been a huge time commitment.

You can't see the classroom as you talk in the same way you see them in a live classroom, so the skills you've developed in reading those cues aren't on point. 

There are lead time and set up issues. I use PowerPoint slides, and I've found it helpful to the students to circulate those before class so they can refer to them in a separate window. That requires having them locked down a day or so ahead of time. Online meeting links also need to be generated and circulated. I've also been designating a list of on-call students, something I don't always do in person, so that the students can be sure their cameras and microphones are ready to go. 

One thing I've not sorted out is assessment in my Civ Pro class. I typically give an exam with multiple choice and essay components. I'm not sure how I am going to do that if we are not back on campus by exam time. There are software solutions that will lock student's computers down, but we don't have them. For self-assessment, I'm using the same online quizzes on TWEN that I've used for years.

Advantages. Some of the advantages have been surprising as well. Zoom allows students to ask questions anonymously in the Q&A. This allows students to put things out there without losing face or seeming stupid, and it actually generates way more student commentary and questions than I get in a live class. I get so many comments in my Civ Pro class that keeping the flow of class moving ahead has become a bit of an issue.

Another advantage that I hadn't thought about in advance is that making the classroom non-physical opens up the world of guest panelists and speakers. I'm busy trying to pull in thought leaders and people with on-point experience to take a few minutes in both my Civ Pro and Law and Innovation classes. For example, in Law and Innovation, which has a large PR component, I'm bringing in a lawyer from Kentucky to talk about the devastation that was visited on the local community around Butcher's Hollow (yes, that Butcher's Hollow) when a corrupt local lawyer entered into a bribery deal with a social security judge to have his clients all approved as disabled. The upshot was that the Social Security Administration cut off benefits for all the clients of the corrupt lawyer, most of whom were in fact disabled, and who were neither aware of nor involved in the fraud, leaving many of them in perilous straits. I'm hoping that having live in the classroom someone who was involved in cleaning up a tragic situation caused by lawyer dishonesty will make the real-life consequences of lawyer misconduct more palpable. (By the way - feel free to volunteer to be a guest speaker).

My students, I think, are enjoying having all classes recorded and available for download. Setting aside technical problems, if they drift or lose focus for a moment, they can replay the section that did not quite make sense. If it does not make sense because I did not make sense, they can bring it up in online office hours. (Having every word recorded and distributed might be an issue for those of you with aspirations to sit on the Supreme Court or even just good reputations to protect, and if that's the case you can choose not to enable the recording feature.)

One obvious benefit of teaching online is that you can become a digital nomad and be someplace new and different. The attached photo is from my temporary office at a riverside cafe a few days ago.

As this goes forward, and I am only four classes in, I may post more. Comments on online teaching techniques are welcome. Comments reflecting your personal conspiracy theories on the origins and dispersion of the virus that causes Covid-19 will be deleted.

@SSRN and the (Arbitrary) Determination of "Scholarly" Merit

Yesterday Brian Frye (@brianlfrye) (Kentucky) tweeted:

I am sad. @SSRN has decided that my article about Gremlins (1984), "In re Patentability of the Peltzer Inventions," does not qualify for "public" status because it is "opinion, advocacy, or satire." Why judge? Oh well.  You can still download it here: http://www.ssrn.com/abstrac_id=3371989.

I followed Brian's direct link to the piece. The abstract refers refers to the many inventions of the movie's Randall Peltzer character, and explains, "This essay takes the form of an opinion letter valuating the patentability of Peltizer's inventions." I don't teach IP, but I like Brian's work and so I downloaded the essay. It struck me as funny and as an excellent teaching tool.  But if you go to Brian's author page on SSRN, you wouldn't be able to access the paper.  You wouldn't even see it.

I myself have posted material that apparently doesn't meet SSRN's criteria for a "scholarly paper," including this interview with the principal drafter of some important state trust legislation (and the interview itself has been cited in subsequent scholarship) and this short piece for Tax Notes reviewing estate and gift tax law review articles published in 2016 (even though SSRN published my similar pieces reviewing scholarship for the years 2015, 2014, 2013, 2012, 2011, 2010).

When I posted my Information for Submitting to Online Law Review Companions (a submission guide modeled after Nancy Levit and Allen Rostron's useful Law Review Submission Guide), SSRN reviewer "TyD" responded:

We have accepted your paper to appear in the "Other Papers" section of your Author Page, which is where opinion, advocacy and satirical papers are displayed as per SSRN policy. It will not be searchable by the SSRN eLibrary search engine but will be searchable by public search engines, and you may share the URL.

I asked SSRN for reconsideration, and got this from SSRN reviewer "Katie M":

We have recently reviewed your submission to SSRN. Our classifiers have determined that this submission is very useful information, however would still be considered non-scholarly. Because of our acceptance of your previous paper 1019029 [the Levit/Rostron paper, which isn't "my" paper -- BC], we have made the decision to allow the public viewing of this paper. In the future, similar submissions will be processed to appear in the "Other Papers" section of your Author Page, which is where submissions that are not full scholarly research papers (including submissions such as data tables, summary book reviews, opinion, advocacy and satirical papers) are displayed as per SSRN policy.

I don't even bother thinking that Information for Submitting to Specialty Law Reviews and Journals in Gender, Women & Sexuality will be a searchable public paper. 

So what exactly are SSRN's rules for what is "publicly available" and what is not? What I could find appears below the fold.

That "Cocky" Trademark

For anyone interested in the use of trademarks in the publishing industry, an interesting situation is emerging in the romance writers' area. This week romance author Faleena Hopkins was granted a registered trademark in a stylized version of the word "Cocky" for her romance series which includes the word in the title of each of her books. Apparently she then sent takedown notices to Amazon to ask them to remove from sale other romance books with the word in their titles and also sent letters to independent romance authors asking them to change their titles. The Romance Writers of America has apparently engaged an IP attorney to investigate the matter and there is at least one cancellation petition floating around at the moment. A number of legal issues are raised here including whether the registration should have been granted in the first place, although, in theory, there's no objection to registering a mark that relates to a book series, as opposed to a title of a single book. It's also important to understand that registration is not the same as owning the mark - it gives benefits to the registrant but it's of course possible to claim an unregistered mark as valid. So even if the registration is cancelled, there may be questions as to whether there's a valid unregistered mark in play. There's a useful summary of the situation and relevant legal issues here.

Lecture Recordings in Online Teaching

In recent years, I've been delving (both in law and other disciplines) into strategies and platforms for effective online teaching/learning. I was never a strong proponent of online law teaching in particular because, like many others, I've always felt there's something about classroom interactions that's hard to replace online. And, yes, I know there are all sorts of ABA Standards and other rules that preclude significant amounts of online J.D. instruction, so my experience in the recent past has been with online LL.M. and other similar programs.

While I was always leery of online law teaching, I have taken a lot of online courses, and started to teach several, in other disciplines. The technologies have advanced significantly since I first raised my concerns in the mid to late 1990s when law profs in Australia began to worry we'd soon all be replaced by talking heads on computers.

I still haven't come to any definitive conclusions about when, and in what circumstances, online teaching can be as good as, better than, or a useful supplement to classroom instruction, but I've recently started thinking about one particular aspect of online teaching: lecture recordings, and how best to make use of them.

I've taken, and also instructed, online classes that have no "lecture" component. There may be preset readings or short YouTube videos, or blog posts gathered together as part of the course materials, but no actual formal lectures as such. I've also taken, and instructed, classes where the instruction has utilized either interactive video classes with students (via Skype, Zoom etc) or pre-recorded lectures, or a combination of both.

Pre-recorded lectures can be particularly useful in some circumstances: for example, where the course needs to be asynchronous because students are in different places/time zones etc. 

I've been teaching a few courses recently that require pre-recorded lectures for this purpose. Because these courses are law-related and because I want discussions to be as interactive as possible (even though they're largely asynchronous and use text-based discussion boards), I record new lectures weekly, both presenting new material and reacting to/commenting on discussions from previous weeks. I do this, rather than pre-recording everything upfront, because I want to simulate, as best I can within the constraints of the technology, the feeling of an interactive classroom.

However, some people have suggested to me that this is a waste of time and it's easier to pre-record everything because this can give you the benefit of re-using lectures in later iterations of a course in subsequent semesters, and it also gives you the benefit of having everything prepared upfront and not having to fit in lecture-recording at what may become a busy time of the semester.

I'm wondering if others have faced this issue in teaching or learning in an online setting and how you feel about it. I think pre-recorded lectures that are recyclable can be very useful in many contexts including things like Bar Prep courses where there is a lot of material that doesn't change much from year to year, a lot of students in a lot of different places, and perhaps less need to create an environment that simulates a classroom setting. However, does it always make sense to save time and recycle lectures if you can? (I'm aware too that a lot of folks teaching online courses are adjuncts and visitors and have less time to spend on course prep because of, say, other practice obligations, than full time faculty members and that might make a difference too.)

So, yes, this is a finicky point and not many people are probably thinking about it, but if anyone is, I'd be interested in your thoughts.

 

New Book Announcement: Electronic Health Records and Medical Big Data (Hoffman)

Congratulations to Sharona Hoffman (CWRU Law) on the recent publication of her new book on Electronic Health Records and Medical Big Data with CUP.

Here's a quick summary for anyone interested in taking a look: This book helps readers gain an in-depth understanding of electronic health record (EHR) systems, medical big data, and the regulations that govern them. It analyzes both the shortcomings and benefits of EHR systems, exploring the law's response to the adoption of these systems and highlighting gaps in the current legal framework.  It also develops detailed recommendations for regulatory, policy, and technological improvements to enhance EHR systems’ safety, security, and efficacy for both clinical and secondary (such as research) uses. Electronic Health Records and Medical Big Data addresses not only privacy and security concerns but also other important challenges, such as those associated with data quality and data analysis. The book can serve as a primer for students and a resource for knowledgeable professionals and academics.

More info here.

The (Video) Gamification of Law

No one’s immune from the (video) gamification of life.  Games populate our Facebook pages, home screens, and even (still) once per week family gatherings.  You can be a marine, a real estate tycoon, an environmentalist, a football coach, or even a fruit ninja.  Recently, I downloaded a game to my iPhone called Landlord 2.  It’s a basic buy property and get rich game.  In this game there are several “skills” the player may purchase to improve their money-making: Tycoon (increase property slots), Lawyer (increase properties waiting for paperwork), Innovator (reduce property cost), Landlord (increase property rent), Accountant (reduce taxes), and Speculator (increase property valuations). 

All of these skills are interesting in their own right; they all simplify the complexities of property acquisition.  But, what might be of most interest to readers of this blog is the Lawyer skill.  There are two problems with this skill attribute: 1) the lawyer only functions transactionally, increasing the amount of space available for the processing of land deals, and, relatedly, 2) that the lawyer’s primary function is “paperwork.”  Both of those points cut deeply into the importance of lawyers, law, and legal knowledge. 

Practitioners and scholars alike will note that while lawyers function in transactional roles they also do much more on a variety of issues.  Lawyers are often involved in the beginning, middle, and end of legal agreements, provide counsel and engage in negotiations, and represent clients of all types in court and other proceedings.  Simply put, there’s a lot more to law than moving things through a nebulous system from one’s desk chair.  Even if transactional lawyering is understood as synonymous with corporate or business lawyering, that still denies the complexities of that wide-ranging practice area.  The only business of business law isn't sitting at one's desk.  

Secondly, the abstract notion of paperwork seems to demean just how important legal documentation is.  Paperwork would seem to include contracts, memos, briefs, and maybe emails, professional publications, etc.  Rather than discuss contracts or filing motions, Landlord 2 gives the Lawyer the job of paperwork.  This risks positioning lawyers as paperwork pushers rather than advocates, allies, and important participants in legal actions. 

Of course, television, movies, and music have all had their problems in reasonably representing law and lawyering, but now that our students, friends, family, and colleagues are on their iPads, iPhones, and other devices, we should turn our attention to the video gamification of law.  I’d be interested in readers adding their experience with lawyers and law firms in video games or apps.  What role do these organizations and characters serve?  Do they represent law firms, prosecutors, defense attorneys, etc. fairly or as best they can given the constraints of the game or app?  What games or apps seem particularly successful or particularly disastrous?  Have representations in video games changed over time? 

Other folks have written about gamification, but not video gamification.  Gamification describes a broader concept of incentivizing action through rewards.  Imagine leveling up for turning in a report to your dean or memo to a senior partner.  Of course, leveling up might involve more free time, better case assignments, an extra day off, or increased research budgets, and not a Super Mario Bros. 3 Raccoon tail. 

Readers might find these stories interesting on the debate about gamification, generally: “Improve your legal practice through gamification,” “What can ‘gamification’ do for lawyers?,” “Gamification of law: Good idea or game over?,” and for a scholarly take from Austria, “Gamification and law: On the legal implications of using gamified elements.”

When we start to address these issues, we’ll be in a better place to assess what I’ve called “coming to the law,” or how people because of their social experiences construct the legal world and their interactions with it.  If our clients and students are learning from video games and apps, then we’ll need to work with them to not only understand them and their ideas, but also to better understand law and lawyering ourselves. 

The Accidental Entrepreneur

Thanks to Al for the introduction and to Dan Filler and the rest of the Faculty Lounge crew for the kind invitation to guest post.

It's a familiar tale: you're an eccentric law professor in your mid-40s, and one day you look around to discover your children are getting older, your hair is turning grey, and you accidentally founded a high tech legal ed start-up.  

It's that last bit I thought I'd write about.  

When I began teaching, I found myself drawing charts to help me organize my courses -- charts that visually represented, and therefore helped me understand, how the concepts I was teaching were related to each other within a system.  Eventually it occurred to me that if the charts aided my understanding, they might help my students too.  So I shared them.  The effect on student comprehension was immediate.  What had seemed to them like a meteor shower of disparate concepts became useful constellations instead.  We even built some charts together in class.

Intrigued, I dug into the educational literature and found that the research was pretty unequivocal: concept maps were pedagogically superior to text alone for understanding the relationships between ideas within systems.  Books are wonderful for lots of things, but it turns out they are not the best tool available to us for teaching systemic relationships between concepts.  

The next year the casebook I had been using came out with yet another new edition (its 3rd in 6 years).  The changes were trivial -- just enough to alter the pagination so that using a mix of editions was unworkable.  I reluctantly assigned the new edition, but for the 1st time, I asked about its price (after all, I always got my copy for free).  $250, for one book, for one class.  I considered switching to an e-casebook version, but I found that the e-book was really just a pdf'd version of the book with a few hyperlinks and the publisher was still charging $180 for it.

It occurred to me that in the best of all worlds, courses could be formatted as concept maps, and the maps would contain within them all of the material for the course -- cases, statutes, practice problems, background readings, powerpoints, decision trees, even videos.  That would replace the casebook with something that was pedagogically better, more comprehensive, more flexible, and less expensive.

But I'm a technophobe.  Not only couldn't I create such a thing, it took me weeks to figure out what the people who possibly could are called (software developers, it turns out . . . but you probably knew that).  Still, the idea gnawed at me, until I took my pencil-and-paper drawings to a bunch of software development companies and asked them if they could make something like my drawings, except interactive and on one of those computer thingees.  Their answer?  Yup -- for a price.  As your typically business-savvy academic, I then tried to give the idea away. In this case, to several major publishers.  But my pitch to them -- 'just think of how much less you could charge your customers!' -- somehow did not generate the enthusiasm I imagined.

So, faced with adversity, I gave up.  Except I couldn't quite.  Because my students were swaying under a mountain of debt.  Because I knew I could teach them better with a tool designed to show the structure of relationships between concepts.  

So, I gave the developers my kids' college fund (for real -- don't tell them) and formed a company with two professor colleagues.  We called the company ChartaCourse.  We simply asked ourselves: what do we, as law professors, want in our teaching materials? We came up with 4 fundamental goals:

(1) A new format that constantly reveals and reinforces the structure of our courses.  True to the original vision, we wanted concept maps that charted entire law school courses, with all of the content for the course actually embedded within the map itself, in the right spots.  Students should constantly see the 'forest' of the course and the particular 'tree' of each concept we are studying.  And all of the material for that concept should be available with one click.  Here's what it looks like.  Notice that the dark blue color creates a trail from the general to the specific as we move from left to right:

 (click on the image to expand it)

And here's what happens when the user clicks on Pierson v. Post from the card of the left:

(click on the image to expand it)

(2) Complete customization.  Once a chart was created for a course by one of our authors, we wanted every adopting professor to be able to customize it completely.  She should be able to easily add material, delete material, edit material, re-arrange the chart, embed her syllabus, etc., and those changes should be visible only to her own students. For example, in my personal version of the Crim Pro chart, I have added a video of Freddie Gray's arrest:

 (click on the image to expand it)

The coming arrived casebook revolution

This morning, I received a survey from Wolters Kluwer (Aspen), publisher of the IP Survey textbook I use. The survey sought out what I find important in a book, and even asked if I knew any young IP professors I was impressed with (who could presumably write a new book). Why? The casebook authors (Profs. Merges, Menell & Lemley) somehow worked out a deal to take the book "private." That is, Aspen won't be publishing the book anymore, and they will self publish the book for a mere fraction of the former cost.

This is just one example of the casebook revolution. Contrary to what students might think, many professors actually do care about casebook prices. Modern technology is finally allowing them to do something about it.

The straw that broke the camel's back, I think, came last year, when Aspen offered Dukeminier, et al., as a rental only. That is, students paid for an electronic version and a paper version, could mark up the paper version, but then were required to send the paper version back. The cost was less than the prior full price of a casebook, but not by much. The rationale for this move was obvious: cut out the secondary market. Professors rebelled, and many said they would switch books before they would allow this, and Aspen relented, offering a print version (and claiming that its prior emails saying there would be no such version were misinterpreted).

But the handwriting was on the wall: book publishers are looking for a way to extract more and limit competition. Some of my colleagues get angry about that. My view is more sanguine: entry costs are low, so someone should be able to capture the market by simply competing with a lower price. If we hate this setup so much, we should just do casebooks another way. 

Perhaps not surprisingly, IP and internet law professors have led the way to start doing just that. Independent presses have arisen, open source casebooks have been created, and materials have been shared. You can now get two IP Survey texts (with a third on the way) for around $30. You can get statutory supplements for free in PDF (or cost of printing in paper). These things sell for $60-$90 from publishers. There are specialty casebooks (e.g. trademark or internet law), as well as highly focused reading materials (for example, you can get free access to my internet law readings online, or you can buy it in paper for about $35, if you want my exact reading choices for internet law).

Another benefit of the revolution is that new platforms allow for benefits not previously possible. Unlocked PDFs that allow for easy search, cut, and paste. An "open source" feel whereby edits by one professor's edits can be recombined into other books (though standardization of format would be helpful for this). More images, quotes, video links, etc., by professors willing to take fair use positions that publishers might not be willing to take.

In other words, the revolution has arrived, and is waiting to spread. Non-IP professors have not picked up on this as much (yet), though the H2O platform at Harvard has several reading lists in non-IP courses. As self-publishing platforms grow and more professors get on board, casebook publishers will see significant pressure.

I'll end this post with a couple caveats, however:

1. Publishers add value. For the most part, the cheaply available works are not nearly as polished as casebooks. They also contain far less material (which is good and bad - depending on whether the book matches what you want to teach). There are definitely exceptions to both of these.

2. Authors that have more potential upside may spend more time. If you like "notes" that sum up history, cases in other jurisdictions, etc., you are less likely to see those in a self-published book - certainly in the first editions. That work takes time, and simple incentives may limit it. On the other hand, if the author keeps 100% of the profit, then maybe such work will be worthwhile. Creating my own set of materials took a huge amount of time, and I simply had nothing left in the tank to write long notes, introductory text, hypotheticals, etc. But not all works are the same. I have colleagues who have written books with some fine problems in them, for example.

3. Sometimes the published book is just better. I like my patent law book. I like my contracts book. It's going to take a lot to convince me that another book is better. But that's what competition is all about.

4. New systems lack standardization, and thus may not offer all the features we want. Converting my fully online materials to printed materials was time consuming and, frankly, pretty ugly. There are other systems that may work better, and eventually the best ones will shake out.

In conclusion, be on the lookout for opportunities to collaborate on new textbooks in new areas, and the next time you complain about high prices, remember that you can do something about it!

[updated to incorporate and respond to some good comments]

Call for Papers: Designing Ethical Review Processes for Big Data Research

The Future of Privacy Forum is hosting an academic workshop supported by the National Science Foundation to discuss ethical, legal, and technical guidance for organizations conducting research on personal information. Authors are invited to submit papers for presentation at a full-day program to take place on December 10, 2015. Papers for presentation will be selected by an academic advisory board and published in the online edition of the Washington and Lee Law Review. Four papers will be selected to serve as “firestarters” for the December workshop, awarding each author with a $1000 stipend. Submissions, which are due by October 25, 2015, at 11:59 PM ET, must be 2,500 to 3,500 words, with minimal footnotes and in a readable style accessible to a wide audience. Publication decisions and workshop invitations will be sent in November. Details here.

More on the A/B Illusion: IRB Review, Debriefing, Power Asymmetries & A Challenge for Critics

Here are a few clarifications and further thoughts arising out of my co-written Sunday NYT op-ed based on my forthcoming law review article (here, I speak only for myself, and not for my op-ed co-author).

First, as is usually the case with op-eds, we didn’t write, or even have any advance notice of, the headline. It’s easy to see why the Times chose it, as it’s clickbaity, but “Corporations, Please, Experiment On Us. You don’t even need our consent” is not very reflective of the nuances of our position. Now on to some of those nuances.

A Clarification About the Op-Ed’s Position on IRB Review and Debriefing

On Twitter, James Grimmelmann worries that New York Times readers will interpret one part of the op-ed, as he did, to be making a problematic claim. In the piece, we ask readers to imagine a hypothetical CEO who worries that some of her employees aren't taking optimal advantage of the company's generous 401(k) matching program. She has a hunch that adding to the usual matching letter information telling each employee what his or her similarly-aged peers are saving would nudge the low-savers to save more. We explain that the CEO could implement this peer information letter policy in the next enrollment cycle and observe the result, but that, regardless of whether savings went up or down, she'd never really know what effect, if any, the policy had without running a randomized, controlled experiment in which half the employees receive letters with no peer information and half receive letters with peer information. We also explain that telling employees in advance about that the company would be sending out different letters and why would badly bias the results by altering employees’ behavior. We then explain that our hypothetical CEO isn’t so hypothetical:

Economists do help corporations run such experiments, but many managers chafe at debriefing their employees afterward, fearing that they will be outraged that they were experimented on without their consent. A company’s unwillingness to debrief, in turn, can be a deal-breaker for the ethics boards that authorize research. So those C.E.O.s do what powerful people usually do: Pick the policy that their intuition tells them will work best, and apply it to everyone.

It is this paragraph that has James worried:

Your argument appears to run: (1) people will be outraged about being experimented on if they're debriefed; (2) managers "chafe" at de-briefing because of the outrage, (3) IRBs would require debriefing; so (4) it’s reasonable for companies not to go through IRBs at all [and to conduct the nonconsensual A/B test of 401(k) letters with no debriefing afterward].

(The above is three tweets strung together but otherwise verbatim; the bits in brackets are my words, but I think they fairly reflect James’s other tweets and the logic of his argument. I trust he’ll tell me if I’m wrong.)

To be clear: No, this is not the position we intended to take in the op-ed. I thank James for bringing this possible misreading to my attention so that I can correct it here, at least. More after the jump.

New York Times Op-Ed on the A/B Illusion

I have an op-ed with Christopher Chabris that will appear in this Sunday's New York Times on one theme in my recent law review article on corporate experimentation. Despite the rather provocative headline that the Times gave it, our basic argument, made as clearly as we could in 800 words, is this: sometimes, it is more ethical to conduct a nonconsensual A/B experiment than to simply go with one's intuition and impose A on everyone. Our contrary tendency to see experiments—but not untested innovations foisted on us by powerful people—as involving risk, uncertainty, and power asymmetries is what I call the A/B illusion in my article. Here is how the op-ed begins:

Can it ever be ethical for companies or governments to experiment on their employees, customers or citizens without their consent?

The conventional answer — of course not! — animated public outrage last year after Facebook published a study in which it manipulated how much emotional content more than half a million of its users saw. Similar indignation followed the revelation by the dating site OkCupid that, as an experiment, it briefly told some pairs of users that they were good matches when its algorithm had predicted otherwise.

But this outrage is misguided. Indeed, we believe that it is based on a kind of moral illusion.

Two Cheers for Corporate Experimentation

I have a new law review article out, Two Cheers for Corporate Experimentation: The A/B Illusion and the Virtues of Data-Driven Innovation, arising out of last year's terrific Silicon Flatirons annual tech/privacy conference at Colorado Law, the theme of which was "When Companies Study Their Customers."

This article builds on, but goes well beyond, my prior work on the Facebook experiment in Wired (mostly a wonky regulatory explainer of the Common Rule and OHRP engagement guidance as applied to the Facebook-Cornell experiment, albeit with hints of things to come in later work) and Nature (a brief mostly-defense of the ethics of the experiment co-authored with 5 ethicists and signed by an additional 28, which was necessarily limited in breadth and depth by both space constraints and the need to achieve overlapping consensus).

Although I once again turn to the Facebook experiment as a case study (and also to new discussions of the OkCupid matching algorithm experiment and of 401(k) experiments), the new article aims at answering a much broader question than whether any particular experiment was legal or ethical. Here is how the abstract begins:

“Practitioners”—whether business managers, lawmakers, clinicians, or other actors—are constantly innovating, in the broad sense of introducing new products, services, policies, or practices. In some cases (e.g., new drugs and medical devices), we’ve decided that the risks of such innovations require that they be carefully introduced into small populations, and their safety and efficacy measured, before they’re introduced into the general population. But for the vast majority of innovations, ex ante regulation requiring evidence of safety and efficacy neither does—nor feasibly could—exist. In these cases, how should practitioners responsibly innovate?

My short answer to this question is that responsible innovators should inculcate a culture of continuous testing of their products, services, policies, and practices, and that it is a kind of moral-cognitive mistake (which I dub the "A/B illusion") for the rest of us to respond to these laudable (and sometimes morally obligatory) experimental efforts by viewing it as more morally suspicious for innovators to randomize users to one of two (or more) conditions than to simply roll out one of those conditions, untested, for everybody. The long answer, of course, is in the article. (The full abstract, incidentally, explains the relevance of the image that accompanies this post.)

Thanks to Paul Ohm and conference co-sponsor Ryan Calo for inviting me to participate, to the editors of the Colorado Technology Law Journal, and to James Grimmelmann for being a worthy interlocutor over the past almost-year and for generously unfailingly tweeting my work on Facebook despite our sometimes divergent perspectives. James's contribution to the symposium issue is here; I don't know how many other conference participants chose to write, but issue 13.2 will appear fully online here at some point.

If you would rather hear, than read, me drone on about the Facebook and OkCupid experiments (and some other recent digital research, including Apple's ResearchKit and the University of Michigan's Facebook app-based GWAS, "Genes for Good," as well as learning healthcare systems and the future of human subjects research) you may do so by listening to episode 9 of Nic Terry's and Frank Pasquale terrific new weekly podcast, This Week in Health Law.

Reference Management Software – If you’re not using it, you probably should be.

After spending 4 years doing other sorts of graduate coursework before starting my JD, I was pretty surprised when I learned that many (most?) legal academics don’t use reference management software. However, after an introduction to the Bluebook, I realized why this is probably the case. One of the best features these sorts of programs provide is the ability to automatically generate and dynamically update the citations in a paper-in-progress. Unfortunately, the Bluebook is so (needlessly?) complex that until recently there wasn’t a good implementation of BB style citations by any of the major citation management packages. 

Enter MLZ, a branch of the open source Zotero program aimed at implementing a number of multi-language features (hence the name “MultiLingual Zotero”). But the “killer feature” as far as legal writing goes is the MLZ Bluebook style. While MLZ’s bluebooking isn’t perfect, I’ve seen many hundreds of submitted law review articles and I’d wager that a paper written solely relying on MLZ’s automatic citation generation would be in better bluebooking shape than most of the articles that student reviewers receive. And, for better or worse, the state of one’s citations is probably an important factor in determining how your article is perceived by reviewers. 

The Law School Network & Legal Network Analysis

Because it was a series of network-related posts that led to my invite to guest-blog here at the FacultyLounge, I thought I’d start things off with another interactive network and a discussion of networks more generally. The network below (click-through the image for an interactive version) is an alternate projection of the law prof twitter network. I’ve taken the law prof follower relations and changed the unit of analysis to the school that each prof teaches at. This shows us an electronic social network of law schools. Schools that are strongly linked in this network have a lot of profs following one another on twitter. The larger a school’s node, the more influential it is (as gauged by twitter followers) in the law prof social media world.

This network shows three distinct communities. I’m not convinced the distinction between the red/blue communities is really very meaningful, but the green community is quite clearly mostly made up of European schools, and we see Canadian schools bridging the Atlantic divide. This shows that while geography probably only matters to a small degree in forming these relationships, national borders appear to be very important.

Clickers, Anyone?

I've used clickers since I started teaching. In some classes/lessons, I use them extensively. For example, my novelty and statutory bars sections of patent law are almost all clicker exercises, where I vary the hypotheticals just slightly to test the outer limits of what constitutes, say, "public" use. I think of it as lightning-round Socratic method, except that everyone participates. I also like to use them because it forces the students to take a position. I routinely get many more votes than I do hands raised when I just ask.

I also find that clickers embolden some to speak up, because when I show the totals and that many people agreed, students are more likely to raise their hands and explain their answers. This is helpful, because it allows me to figure out whether the class is understanding the material and, if not, to find out why. The clickers can also scare students off - if only one picked an answer, he or she is less likely to volunteer why. It frequently happens, though, that the minority of one was correct or recognized some nuance in the question. Thus, I use a belt and suspenders approach - get clicks, ask for hands, and then show the results and ask for more volunteers if need be.

I recommend clickers to anyone not using them. I find them helpful and my students have responded positively for years.

But as the ABA moves toward requiring formative assessments (Standard 314), I wonder whether the clicker slides are enough. On the one hand, the students answer questions and we discuss right, wrong, and iffy answers and they can judge for themselves if their responses were correct. But maybe that's just me being lazy and not wanting to do more work to change my methods. Perhaps it would be better if I required everyone to answer (my participation rate runs at about 2/3) and graded answers (I have colleagues that do this).

I don't know the answer to this question. On the one hand, the students are responsible for their own education and should participate as they wish to their benefit or detriment. On the other hand, maybe it would be better to have more grades spread out throughout the year. I'm not sure my students would like that so much, but the educational results may end up improved.

More on Twitter and Information Sharing

For those who followed my posts on law professor use of Twitter, I thought this recent article, called "The Unbearable Lightness of Tweeting," at The Atlantic was illuminating. The gist of the article was an author's self-study of the rate that users click through to read an article. The results leave a lot of questions about the information sharing value of Twitter, not just for law professors, but for everyone.

Bioethx Chat on Reverse Transplant Tourism Tonight

Those of you on twitter should feel free to join me tonight as I host a Bioethx chat on Reverse Transplant Tourism. Given that I am a twitter moron, still don't understand hashtags, and find it hard to even understand much tweet shorthand, this could be fun.  

LinkedIn...to what?

For this final post in my series on social media, I'd like to mention LinkedIn. Much about the site is either creepy or bizarre. For example, alleged spamming from email information has led to a class action complaint. The skill recommendations make lawyers feel a bit uneasy.  Job updates that are essentially resume cleaning lead to awkward congratulations/questions about non-existent job changes. In short, I've always believed that LinkedIn is useless, especially to law professors who don't get hired through that network. However, I have recently upgraded my assessment to partially useless.

Facebook and the Law Professor

Continuing my series of posts on professors and social media, I thought I would write a few words about Facebook. I have less to say about Facebook than Twitter, in part because I don't have data and in part because I think that most professors use Facebook like any other such user, to connect with family and friends, share photos, etc.

There are, of course, a few professors who use Facebook to only share their own work or other information specifically related to their areas of interest. I tend to tune those users out, for the same reason I tune out much of my Twitter feed. I just don't have time to absorb all of it.

So, what is there to say about Facebook and Academia, then? I think there are a few relevant points, which I categorize into "everyone" and "career sensitive" (e.g. junior) professors.

Everyone

1.  I find that academic life can be very insular. Facebook can fight this tendency to the extent that Facebook friends are not part of the academy. For example, my high school classmates are a diverse group and the stuff they talk about and articles they share give me a sense of what's happening outside of my own social group and outside media portrayal of particular events and debates. Indeed, the list of "trending" shared news stories that Facebook provides is interesting, if only because I almost never see those news stories being shared by my Facebook friends. This serves as a good reminder that what's important to me is not important to everyone.

2. I've found that stories shared by friends on Facebook are more diverse than stories shared on Twitter, even by those same folks. I'm not sure if this is really true, and if so, why.  I know it is for how I use the two platforms. My guess is that people see their Facebook viewers as a more diverse group, which is a bit ironic considering that Twitter is public. But it may be the limited audience that makes a difference - by not sharing with the world, you can be free to share information outside of your normal professional zone.

Career Sensitive Users

But it may be that not everyone should share outside their professional zone. Career sensitive users might want to consider whether Facebook use harms them in any way, and if there is a way to manage it. Note that I offer these thoughts as an observer and not a practitioner. I don't separate sharing lists, so my professional colleagues get pictures of my kids and groans about football losses. Similarly, family and non-academic friends are treated to my wonky ramblings about policy issues. It turns out that there's some overlap in each group, so YMMV.

So, what are some career sensitive concerns? I discuss a few below; some are more cynical than others, but while I'm plumbing the depths of social media by law professors, even the cynical views deserve an airing.

1. Your true personality may shine through. Now, ordinarily this is a good thing, but we all have flaws. I've heard more than one person (in academia and otherwise) say that an interesting thing about Facebook is you often find things out about people that you really wish you hadn't known.  And, so, this can affect how people think about you.

2. Your true political leanings may shine through. Similarly, the things one says and posts on Facebook will help others form a more complete picture of where you stand politically. Indeed, this is one reason why I hate election cycles on Facebook. But if you are career sensitive, then (I am sad to say) your political leanings might make a difference to others who read your posts.

3. Your work habits may be misinterpreted. To the extent that professors use Facebook often, but only post about non-professor things, there is a chance that others might wonder whether you are actually doing any work. This, of course, is a cruel judgment. After all, shameless marketing is frowned upon, but it is so prevalent that when you are not doing it, it stands out.

4. You might just rub some people the wrong way. It happens, and there's nothing you can do about it, but it's a risk to consider. I'll give a couple of my own examples. First, on more than one occasion I've offended senior colleagues/mentors with snarky comments about the academy. These were comments I'd never make seriously or "publicly," but that I stupidly felt appropriate as a Facebook comment. Second, I had a colleague - one I communicate with regularly in other ways - unfriend me after one too many poopy diaper stories. I haven't told a poopy diaper story in years, but I tell plenty of other stories about things my kids say and do and to this day we still aren't Facebook friends.

Those are just a few thoughts; most of them are probably obvious, but I thought it might be helpful to put them in one place.