(Second in a four-part series) When Elizabeth Warren, Teresa A. Sullivan, and Jay Westbrook, co-authors of the 1989 book As We Forgive Our Debtors: Bankruptcy and Consumer Credit in America, were charged by Rutgers Law School Professor Philip Shuchman with scientific misconduct in 1990, they quickly asked for an investigation to clear their names. Sullivan, who resigned earlier this month as President of the University of Virginia, “immediately asked [her] employer, The University of Texas, to investigate the charge,” as she told me in a letter I received by email from her on June 5, 2012.
By 1990, all three co-authors–Sullivan, Warren, and Westbrook–were well established fixtures in University of Texas academic and social circles. Sullivan was the Chairman of the Sociology Department at the University of Texas, and would soon become the Dean of Graduate Studies. Westbrook was a long-tenured member of the faculty at the University of Texas Law School. Warren, who taught at the University of Texas Law School from 1981 to 1987, had by this time moved on to the University of Pennsylvania Law School, a top ten school with great prestige.
Warren’s former colleagues at the University of Texas took pride in her emergence as a rising legal star of the left, a reputation she carefully cultivated. According to some who have followed her career, she was considered a master practitioner of hard-nosed academic political tactics, and her critics in the genteel world of higher education were increasingly wary of her.
It was in that environment that current President of the University of California system Mark Yudof (then Dean of the University of Texas Law School) and current University of Texas Dean of Graduate Studies, ad interim Judith Langlois, wrote the April 1991 report for the investigation that Sullivan requested. It was titled University of Texas Preliminary Inquiry Report: Scientific Misconduct Against Teresa A. Sullivan, Elizabeth Warren, and Jay Westbrook. You can read the full report, provided to Breitbart News by Teresa A. Sullivan here.
The document, though riddled with errors, was accepted by University of Texas President William Cunningham on April 22, 1991, as conclusive evidence exonerating Sullivan, Warren, and Westbrook from charges of scientific misconduct.
Mark Yudof, co-author of the 1991 University of Texas Preliminary Inquiry Report: Scientific Misconduct Against Teresa A. Sullivan, Elizabeth Warren, and Jay Westbrook, is now the President of the University of California system
The lack of serious scrutiny of Ms. Warren’s academic research has continued for the subsequent two decades. Questions about Ms. Warren’s empirical studies have not been fully explored, and specific policies she has promoted–in particular those that led in 2010 to the passage of the Dodd-Frank Act and its onerous Consumer Financial Protection Bureau–have been imposed upon the public.
Breitbart News is not alleging that Sullivan, Warren, and Westbrook engaged in scientific misconduct. We are, however, presenting evidence that suggests the 1991 investigation conducted by the University of Texas into the allegations brought by Philip Shuchman in his scathing sixty page review of the book Sullivan, Warren, and Westbrook co-authored in 1989 was neither thorough nor exculpatory. In that review, Shuchman charged:
This book contains so much exaggeration, so many questionable ploys, and so many incorrect statements that it would be well to check the accuracy of their raw data, as old as it is. But the authors arranged matters so that they could not provide access to the computer printouts by case, with the corresponding bankruptcy court file numbers, this preventing any independent check of the raw data in the files from which they took their information.
The 1991 University of Texas Preliminary Inquiry Report fails to answer Shuchman’s charge that “the authors arranged matters…preventing any independent check of the raw data.” Though Shuchman implies this “arrangement” may have been deliberate, he has no specific basis for the suggestion, other than the suspicious nature of the facts. The effect of the decision by the co-authors was clear, however. It became virtually impossible for other scholars to check their data.
In addition, the 1991 University of Texas Preliminary Inquiry Report fails because there are at least five specific factual errors contained in this brief three page report that purportedly “exonerates” Sullivan, Warren, and Westbrook.
The National Science Foundation (NSF) plays a role in this investigation because it funded the research upon which Sullivan, Warren, and Westbrook’s 1989 book was based with a grant in the amount of $109,999 to principal investigator Teresa A. Sullivan, and co-principal investigators Jay Westbrook and Elizabeth Warren for a study of “Consumer Choices in Bankruptcy: Statutory Intentions and Statutory Consequences.” The data in this study formed the basis for the 1989 book. Research began around September 1, 1983 and continued for about three years thereafter. The University of Texas at Austin was listed as the sponsor of the grant.
Factual Error 1: “NSF has subsequently dismissed Shuchman’s complaint (see letter of March 6 from NSF Office of Inspector General)
The Facts: The March 6, 1991 letter from NSF investigator Montgomery Fisher to Teresa A. Sullivan did not constitute dismissal of Shuchman’s complaint of scientific misconduct by the NSF.
March 6, 1991 letter from Montgomery Fisher at NSF to Teresa A. Sullivan
Instead, it focused narrowly on Shuchman’s request for “access to the original data” used in the study. NSF investigator Montgomery K. Fisher simply confirmed in this letter that he had communicated to Professor Shuchman that Elizabeth Warren and her co-authors were willing to provide a list of the cases they sampled:
“Prof. Sullivan also says that the names and court file numbers are not, and have never been, part of the file. (There was apparently some misunderstanding regarding the applicability of HHS’s Human Subjects regulation; as you observed in note 249 of your article, this project was probably exempt from the regulation.) They are willing, however, to provide you a complete list of the cases they sampled, should you desire to verify each datum. In these circumstances, I believe Profs. Sullivan, Westbrook, and Warren have demonstrated sufficient willingness to comply with NSF’s policy in favor of openness of scientific communication.” [emphasis added]
It was not clear, however, why the authors had not simply documented and archived their data so that others might replicate their results.
Factual Error 2: “Many of the bankruptcy cases were more than ten years old.”
The Facts: The data set was from 1981, which was exactly ten years prior to the 1991 University of Texas Preliminary Investigation Report. None of the data set was “more than ten years old.” The cases were not “resolved” in some instances until several years after 1981.
Factual Error 3: “We conclude, along with NSF, that Sullivan, Warren, and Westbrook did not violate any policy of scientific openness, nor did they engage in scientific misconduct.”
The Facts: At the time the April 1991 University of Texas Preliminary Inquiry Report was written, NSF had come to no conclusions on either the matter of scientific openness or the matter of scientific misconduct. NSF did not come to any conclusions regarding the Shuchman allegations until three months later when a sole investigator wrote a closeout memo on July 2, 1991.
The closeout memo, like the March 6, 1991 letter before it, did not address allegations of scientific misconduct. Instead, it focused on the narrow question of data sharing and scientific openness.
Curiously, in circumstances that will be examined in more detail later in this series, the author of the memo notes that in his personal judgment Shuchman’s allegations did not meet the NSF definition of misconduct:
My examination of [redacted, most likely ‘Shuchman’s’] review (which was published at [redacted] (1990)) and letters made it clear that the allegations regarding the substance in the book do not meet the NSF definition of misconduct.
(As subsequent articles in this series will show, it was NSF policy at the time to undertake investigations of allegations of scientific misconduct on the basis of an investigation conducted by a team of experts rather than an examination conducted by an individual investigator.)
Factual Error 4: “To have provided Shuchman with identifying information would have violated the investigators agreement with our IRB [Institutional Review Board] and with their granting agency, NSF.”
The Facts: To the contrary, the NSF closeout memorandum dated July 2, 1991 stated clearly “[i]n her letters [redacted, but probably ‘Teresa A. Sullivan’] had given the impression that the identifiers for the data base against the [redacted] file it had been coded from, existed but that she could not provide the database to [redacted, but probably ‘Philip Shuchman’] with those identifiers included because of HHS’s Human Subjects Regulation (45 CFR part 46).”
The NSF called Sullivan’s argument on this matter “bogus, because (A) on the cover sheet of the proposal the [Principal Investigators–Sullivan, Warren, and Westbrook] had stated the proposal was exempt from the HSR under 45 CFR section 46.101 (b) (3) and (B) it was in fact exempt under 45 CFR section 46.101 (b )(5).” (Emphasis added.) Sullivan’s failure to disclose this significant change in the methodology to the funding agency, NSF, was problematic at best, and may possibly have been a violation of the original terms of the grant.
Indeed, the NSF closeout memo is quite critical of the authors’ approach to verification of the data used in the study:
When I talked to [redacted but most likely ‘Teresa A. Sullivan’] however, she said that the identifiers were excluded from the database from the start, because of the interpretation of the HSR by [redacted but most likely ‘The University of Texas’] Institutional Review Board. Thus, they were not refusing to provide the database with the identifiers: they couldn’t because the identifiers weren’t put in to begin with…It is unfortunate that their IRB erroneously compelled them to omit useful information from the database, rendering verification of their data exceedingly difficult.”[emphasis added]
One scholar who reviewed this excerpt from the NSF closeout memo, and who spoke exclusively to Breitbart News, noted the impact Institutional Review Boards can have on academic research, suggesting that they have significant power to protect information on human subjects–even when dealing with public records.
Factual Error 5: “[T]hey [the authors] had an agreement with IRB (and therefore, the National Science Foundation) to treat the information as confidential.”
The Facts: Sullivan, Westbrook, and Warren may have had an agreement with the IRB (the University of Texas Institutional Review Board), but the funding agency–NSF–was not party to that agreement. NSF did not learn of it until after Shuchman’s complaint was filed and the NSF inquiry was initiated.
Besides these five factual errors made in the report, this 1991 University of Texas “investigation” was also flawed for the critical and obvious question it failed to ask: Did Sullivan, Warren, and Westbrook intentionally misinterpret HPP federal regulations on “protection of human subjects” in order to persuade the Institutional Review Board of the University of Texas to impose what they knew to be uncalled-for constraints on the use of primary data for the purpose of making the data set unverifiable by other researchers?
A more thorough report would have interviewed the members of the University of Texas Institutional Review Board and questioned them as to why no one had apparently bothered to reconcile the University of Texas’s regulations with the conditions of the NSF grant.
Here’s how Sullivan, Warren, and Westbrook described their methodology, which sampled data from 1981 bankruptcy filings from the 15 federal district courts in the three states of Pennsylvania, Texas, and Illinois in their 1989 book:
“Because existing data were inadequate, we had to collect data on the characteristics of bankrupt debtors, their assets, their liabilities, their jobs, their marital status, whether they were homeowners, and so forth. This decision meant we would have to gather an enormous amount of information. We coded over 200 variables for each of 1,529 debtors, generating over 300,000 pieces of information.” (page 342)
Sullivan, Warren, and Westbrook went on to describe their crucial decision–a decision that was unknown to the funding organization, NSF, until after it began to investigate Professor Shuchman’s allegations of scientific misconduct:
“[A]ll the data were coded by someone with at least a basic understanding of bankruptcy law and procedure. Every coder–indeed every person associated with the project–signed an agreement promising to hold confidential any information acquired about any individual. Although the files we used are public documents, the petitioners, lawyers, and judges had become, for the social scientist, human subjects, and we used normal safeguards for the protection of human subjects. These safeguards included the use of identifying numbers rather than names on all the data tapes prepared, and careful storage, under lock, of any data with identifying information. The names of all debtors and some creditors in this book are pseudonyms.” (page 350)
Despite an invitation to Warren and her co-authors from officials at Rutgers University and the editors of the Rutgers Law Review to defend themselves publicly against charges of scientific misconduct Shuchman made in his 1990 Rutgers Law Review article, neither Warren nor co-authors Teresa A. Sullivan and Jay Westbrook have ever publicly defended themselves against Shuchman’s charges in an academic journal.
They did, however, as subsequent articles in this series will demonstrate, use the cover provided by the University of Texas whitewash report to encourage the unsuccessful attempts by the University of Texas and the University of Pennsylvania to put pressure on Rutgers University to force Professor Shuchman to recant.
June 23, 1992 letter from Rutgers Provost Samuels to Texas Provost Monti
Both Teresa A. Sullivan and Jay Westbrook continue to defend their 1989 work vigorously, while Ms. Warren has failed to respond publicly in any way on this matter. In a recent email to me, Professor Westbrook stated:
Concerning the Shuchman article. The allegation of scientific misconduct was false and was found to be false in two independent investigations supported in substance by two universities and the National Science Foundation. The specific claims made by Professor Shuchman about the validity of our data conclusions were obviously wrong, but were part of the usual academic debates about emerging research. Any suggestion the claims were more than typical academic disagreements would be factually false. The many positive reviews of our book in leading legal and scientific publications speak for themselves as does the fact that our work is generally regarded as part of the bedrock of contemporary study of consumer bankruptcy in America.
Professor Westbrook’s claim that “the many positive reviews of our book in leading legal and scientific publications speak for themselves” doesn’t tell the entire story of the book’s reception by academic peers. It is worth noting that most of these positive reviews were written by academics with little or no expertise in bankruptcy law. One positive review in Science Magazine was only a page and a half long. The author, Ramona Heck, specializes in home based employment and the family. Another favorable review, mentioned specifically in Teresa A. Sullivan’s letter to me received by email on June 5, 2012, was written by Dr. David Caplovitz, who was described in his 1992 New York Times obituary as a “a sociologist and an authority on American spending habits and misleading sales practices.”
All three nationally recognized experts on bankruptcy who reviewed the book–Shuchman of Rutgers Law School, Marjorie Girth of SUNY Buffalo Law School, and economist Michelle White of the University of Michigan–gave it negative reviews. In the final two articles in this series, we will explore the unanswered questions posed by those reviews, as well as the institutional failings of the National Science Foundation and Harvard University and the roles they played in this unfolding scandal.
Michael Patrick Leahy is a Breitbart News contributor, Editor of Broadside Books’ Voices of the Tea Party e-book series, and author of Covenant of Liberty: The Ideological Origins of the Tea Party Movement.