Despite our reservations described in yesterday’s post about limiting the consideration of value to income differentials, here we follow Simkovic’s lead and focus solely on income as the outcome. Boiled down to its essence, Simkovic’s goal is to identify the ‘value’ of a US LLM by comparing the income of international LLM graduates (i) “to those from their countries who stay home” -- that is, comparing the incomes of foreign law graduates with and without a US LLM, and (ii) to other “immigrants . . . by education level” -- that is, comparing the incomes of foreign law graduates who’ve earned a US LLM and are working in the US to the incomes of foreign law graduates who have not earned an LLM but nevertheless are working in the US.
Based upon our review, Simkovic
- is not actually analyzing data for the population about which he wishes to draw conclusions, and
- is not actually making comparisons that reflect the differences in which he is interested.
The Sample Does Not Represent the Population of Interest
Limitations in the available data and some of Simkovic’s analytic choices result in his analysis of a sample that does not represent the population he wishes to study.
Simkovic acknowledged that his choice of dataset excludes people who “permanently leave the U.S. upon completing their degrees,” as the American Community Survey is a survey of households in the United States. This choice is problematic in two respects. First, it excludes the vast majority of international LLM graduates: those who return home or move to a third country. In earlier research, Silver found that slightly more than 18% of the international LLM graduates she studied, who had graduated between 1996 and 2000, remained in the US for at least several years after graduation; today, we might expect the stay rate for international LLM graduates to be lower because of the combination of current conditions in the overall economy and in the US legal market, not to mention the changed circumstances of US law schools after 2008. For those international LLM graduates who leave the US, research suggests that the benefits gained from a US LLM with regard to career opportunities may actually be greater than for those who remain in the US. By building on and extending the work begun by Silver in the early 2000s as well as the After the JD, which has provided a gold standard for research on the US legal profession for more than a decade, we hope to learn more about the careers of these graduates through new and appropriately designed survey research and in-depth interviews.
Second, research of scholars who study international student mobility apart from the law school context, as well as Silver’s earlier research on international law students, demonstrates clearly that dividing the universe of international LLM graduates into two groups -- those who leave and those who stay -- results in an overly simplistic categorization of the career trajectories of international graduates -- both for LLMs and for graduates of any type of US university degree program.
Still, we do not discount the importance of drawing insight from existing data sources. But US Census data are not up to the task: they are ill-suited for the study of LLM graduates because they do not directly identify LLM graduates. Rather, respondents to the ACS are asked to indicate the highest degree they obtained in broad categories: these include ‘bachelor’s degree,’ ‘master’s degree,’ ‘professional degree beyond a bachelor's degree’, and ‘doctoral degree.’ Further, the question about highest degree earned does not restrict respondents to reporting degrees earned in the US. Thus, we cannot know whether a US LLM will be reported as a master’s degree or a professional degree, and we also cannot know whether a reported degree was earned in the US or abroad. Because of these imprecisions, the use of ACS data fails to capture with any reasonable degree of certainty the population of international LLM graduates, even within the limited population that remains in the US.
In addition, Simkovic’s sample selection criteria undermined his analysis. For example, he included as ‘likely LLM holders’ individuals who were children when they entered the US as well as those who were children when they became naturalized US citizens. But it doesn’t make sense in this instance to categorize as ‘international’ people who have been in the US most of their lives, because they would not have earned an LLM as international students -- that is, after earning a first degree in law outside of the US. Rather, if people who immigrated or became naturalized as children wished to study law in the United States as adults, their most likely path for doing so would be in a JD program after graduating from college in the US, just like others who grew up in the US.
Simkovic also excludes Hispanics from the sample he analyzes. We aren’t sure why he made this decision because data from LSAC indicates that approximately 17.6% of matriculating international LLM students in 2012 were from Latin America. Data from the Institute of International Education’s Open Doors project indicates that in 2016, 8.1% of all international students enrolled in US colleges and universities (at any level of study) are from Latin America and the Caribbean, compared to 8.8% from Europe.
The occupational classification system used by the Census and Simkovic’s choices about which occupations to consider also muddy the findings of his analysis. Simkovic’s analysis includes respondents working in occupations that require neither bar passage nor a law degree. While Silver’s earlier research identified a small number of LLM graduates who stayed in the US and work as paralegals and law firm project assistants, by including these occupational groups in his analysis Simkovic amplifies the messiness of the data he draws on because it includes individuals without any law degree, much less an LLM.
Finally, the Census data that Simkovic analyzed included individuals who reported being unemployed or outside the labor force, despite also indicating their occupation as being in the legal services field. This raises questions about whether individuals in this group reported their occupation as in the legal services field only because that was their field of work before arriving in the US (a real possibility given that unemployed survey respondents were instructed "to give their most recent occupation"). Without additional explanation, including these respondents adds to the confusion regarding what his analysis shows.
Tomorrow's post will consider the conclusions that can be drawn from this analysis of Census data.
 See Carole Silver, “States Side Story: ‘I like to be in America:’ Career Paths of International LLM Students,” 80 Fordham Law Review 2383 (2012) (finding a stay rate of slightly more than 18% for a sample of international law graduates who earned a US LLM between 1996 and 2000); Carole Silver, “Agents of Globalization in Law: Phase I,” LSAC Research Report (3.2009), avail. at http://www.lsac.org/docs/default-source/research-(lsac-resources)/gr-09-01.pdf). While it is not possible to know whether this stay rate is reflective of the LLM population today, law schools are relentless in warning international students that the LLM is not a path to a US job. See, for example, University of Pennsylvania LLM Career Services (“We also want to take a minute to talk to you, frankly, about the great difficulty LLMs have in finding legal employment here in the United States due to significant structural barriers. This is in contrast to the excellent opportunities many of our LLM graduates secure in their home countries or sometimes in a third country. . . . Unfortunately, it is very difficult for LLM graduates from any LLM program in the United States to find a law-related job in the United States today.”); Columbia University Law School LLM Eligibility and Admission Standards (“International lawyers who intend to reside and practice law in the United States permanently should apply for the J.D. degree, the first degree in law, rather than the LL.M.”).
 See Carole Silver, “The Variable Value of US Legal Education in the Global Legal Services Market,” 24 Georgetown Journal of Legal Ethics 1 (2010).
 See Ronit Dinovitzer et al., After the JD: First Results of a National Study of Legal Careers (2004); Ronit Dinovitzer et al., After the JD II: Second Results from a National Study of Legal Careers (2009); Gabriele Plickert, After the JD III: Third Results from a National Study of Legal Careers. For a description of the After the JD project, see http://www.americanbarfoundation.org/publications/afterthejd.html; for a list of certain of the publications drawing on AJD data, see http://www.americanbarfoundation.org/publications/AftertheJD/AJD_Publications.html.
 Personal email correspondence dated September 30, 2014 between Ethan Michelson and Josiah Evans, who at the time was Assistant Director of Social Science Research at the Law School Admissions Council. The estimate is approximate because countries with fewer than 5 applicants or matriculants were not reported. The same data also reveal that the largest group of LLM matriculants by home country -- approximately 20.5% -- are not international students, but rather report their home country as “United States of America/Territories.”