A Pilot Rasch Scaling of Lawyers' Perceptions of Expert Bias
http://www.100md.com
《美国精神病学与法律学报》
Dr. Dattilio is Instructor in Psychiatry, Harvard Medical School, Boston, MA, and Clinical Associate in Psychiatry, University of Pennsylvania School of Medicine, Philadelphia,PA. Dr. Commons is Assistant Clinical Professor of Psychiatry, Harvard Medical School, Boston,MA. At the time of this study, Ms. Adams was Research Associate, Program in Psychiatry and the Law, Massachusetts Mental Health Center, Boston, MA.
Dr. Gutheil is Professor of Psychiatry and Co-Founder, Program in Psychiatry and the Law, BIDMC Department of Psychiatry, Harvard Medical School, Boston,MA. Dr. Sadoff is Clinical Professor of Psychiatry and Director, Center for Studies in Socio-Legal Psychiatry, University of Pennsylvania School of Medicine, Philadelphia, PA. This study was funded by a general grant awarded through the Pennsylvania Bar Institute to the first author.
Abstract
How seriously do attorneys consider the biases of their retained mental health experts Participants in this pilot study included 40 attorneys, randomly selected from a pool of members of the Pennsylvania Bar Institute, who rated—for their biasing potential—several situations that might affect the behavior of an expert. A Rasch analysis produced a linear scale as to the perceived biasing potential of these different items from most to least biasing. Among other results, the study suggests that attorneys do view mental health experts who work on both sides of cases as being more balanced in their testimony. However, they also indicated that they have a preference for using individuals who repeatedly testify for one side. Working for only one side in both civil and criminal cases yielded large scaled values. Additional comments offered by respondents indicated that: (1) an opposing expert also serving as the litigant's treater and (2) an opposing expert being viewed as a "hired gun" (supplying an opinion only for money) were viewed by subjects as not being very biased. A discussion of the results raises the need for future research in this area.
The demand for psychiatric and psychological expert testimony has multiplied over the past three decades, in both criminal and civil cases, because of the increasing need to translate for the court in cases involving mental health.1 The role of any forensic expert is to educate the court or the fact finder about matters that are beyond the lay person's knowledge.2 Based on the active discussions that appear in the forensic literature3–6 regarding the expert witness's role, certain questions have yet to be addressed. Among those are problems pertaining to the notion of being "typecast" by attorneys as appearing consistently biased for one side or the other. Are experts who have been retained over time by both sides of legal cases viewed by attorneys as being more credible than those who seem to "favor" only one side Do attorneys seek out experts for their presumed biases or do they genuinely value the objectivity of experts to "call it as they see it" What do attorneys view as biasing factors affecting experts Does a "history as an expert" (one of six important factors for experts identified by Kennedy3) influence an attorney's choice as to whom to retain
In their respective professions, experts typically subscribe to ethics codes that urge them to serve the court as impartial providers of reliable information within their fields of competence.4,5 Expert witnesses also propose that the objectivity they bring to the legal system is regarded as one of their most valued qualities, together with honesty and neutrality. A challenging but necessary task for experts, therefore, would be to deal with potential "expert bias." A looming question that remains, however, is whether retaining attorneys, who are explicitly and appropriately partisan rather than neutral, subscribe to the same values
In a previous survey on the perceived bias of forensic experts, Commons et al.6 presented data showing that expert witnesses themselves perceive the existence of a good deal of such bias in their own professions. In those studies, some of the potentially biasing situations had higher significance values and larger effect sizes. In other words, some situations were perceived as more biasing than others. For example, it was found that experts believed that working for only one side (either the prosecution/plaintiff or the defense) in both civil and criminal cases is very biasing. Also, in an interesting contrast, it was found that (1) an opposing expert also serving as the litigant's treater and (2) an opposing expert who is acting as a "hired gun" (supplying an opinion only for money) were two situations viewed as not very biasing. The results of this study created a template from which to construct questions directed to other groups involved in the legal system, regarding expert bias. In particular, the purpose of the current study is to determine how attorneys perceive the potentially biasing effect of the model situations proposed in the aforementioned study.
The Present Study
The results were measured on a ruler-like scale that facilitated more convenient responding, using a technique known as the Rasch analysis.7 It was our hope that forensic experts, attorneys, and scholars would benefit by being informed about how attorneys view the seriousness of various biasing situations. These findings may enable more rational decision-making by those three groups and shed light on an area that has not been previously explored from an empirical standpoint.
In this pilot study, a Rasch analysis was used to show the degree of perceived bias in an objective, empirical manner. To understand the results, a basic knowledge of Rasch scales is described.
The Rasch Model of Analysis
The Rasch Model of Analysis produces an objective, additive scale that is independent of the particular items used and of the particular participants tested.7 The Rasch method can be used to analyze a large variety of human sciences data.8–12 For example, through the use of probabilistic equations, this model converts raw ratings of items into scales that have equal intervals. This analysis is particularly effective when the raw data are entered as values on a continuous scale. (Either participants were asked to rate an item on a scale, or nonscale answers were coded with continuous whole number integers.) Once the raw data input is coded in a uniform manner (percentages, words, and decimals are all entered, or coded, as whole numbers), the Rasch analysis converts these codes into small numeric values (generally between –4 and +4), according to an order of magnitude. A scale is then produced, on which each item (that was coded for and entered as a raw data point) is placed according to its Rasch "rating " or scaled score. Such a scale can then be used as a type of objective ruler against which to measure the data on items as well as on respondents' ratings. The ruler-like properties of this scale are what provide its advantage over other scaling techniques. For example, the scale is made up of equally spaced, continuous intervals. Also, from a statistical standpoint, this scale provides a linear interval measure. As a result, a change in severity of a perceived bias of 1 carries the same weight from –2 to –1 as it does from 0 to +1. As with a ruler, a change in length of one inch, either from two inches to three inches, or three to four is the same. Regardless of the locations of the starting and finishing places along the ruler, the magnitude of the distance change is equal in both instances. Furthermore, doubling on the Rasch scale means the same change in severity anywhere along its linear axis. Again, using the figurative ruler example, doubling the distance from one inch to two inches results in a magnitude of change equivalent to doubling the distance from two inches to four inches. In this case, a perceived bias with a value of 2.3 is half as severe as a perceived bias of 4.6, just as two inches is half as long as four inches on a standard ruler.
This relationship can be further corroborated by examining the distances between item difficulties anywhere along the scale's linear axis. On doing so, a zero point must first be determined. For items, we can choose the mean item bias, the mean person "bias," or another reference point. Let us choose the mean person bias, M. If we identify an item of bias M, participants of bias M should succeed at it about 50 percent of the time.13 For this study, we could equate this to an item with M level of bias, which participants of M amount of bias should agree with about 50 percent of the time. Next, we can identify an item of bias A, so that its height relative to M is A – M. Then, we can ask what proportion of participants of bias M succeeded at item A. Say this proportion was 25 percent. Keep in mind, the proportion of participants with bias A who succeeded on or agreed with item A should be about 50 percent, since this is how we know they are of ability A. Next, we can find item B, which people of bias A succeed at (or agree with) 25 percent of the time. We will find that its level of bias is approximately 2(A – M) + M. This item must, therefore, be twice as high (severe, or biased) as item A (Michael Linacre, personal communication, January 2003). This mathematical relationship gives further evidence for the notion that doubling on the Rasch scale means the same change in severity anywhere along its linear axis.
In analyzing data using a Rasch model, several questions can easily be answered. For example, this model indicates where on the scale each item falls (e.g., in this case, how severe the perceived biases were for any given item). This is a question that cannot be answered through use of other scaling techniques and will therefore provide a novel approach to information seeking by systematic, objective scaling. Second, the Rasch model aids in denoting the range of scaled values that exist between all variables for all participants. Third, the scaled value for each participant can also be measured with regard to the overall severity of these biases.
The rating scale provided respondents with a measure of how biasing each situation appears to them. Such a measure allows respondents to point to empirical data when confronted with some of these situations. This method also allows us to determine how much of a difference a change in the score will make. The smaller the range of scaled perceived biases, the larger the difference a change in score of a particular unit, such as 1, makes.
Finally, the extent to which the measured items fit on the scale was also addressed by the infit mean square (MNSQ) values.10 In the Rasch analysis output, both infit and outfit mean square statistics are reported. These mean squares are the unstandardized form of the fit statistic (generally t) and merely the average value of the squared residuals for that item.10 According to Bond and Fox, "the residual values represent the differences between the Rasch model's theoretical expectation of item performance and the performance actually encountered for that item in the data matrix" (Ref. 10, p. 43). In other words, the larger the residual value (and subsequently, the square of this value), the larger the difference between how the item should have performed and how it actually did perform (on the Rasch scale),10 and the fit statistics (both infit and outfit) are representative of the squares of these residual values (since they are merely an average value).
Although there is some controversy as to whether the infit or the outfit MNSQ should be used to determine how well items fit the Rasch scale, we will use infit MNSQ for our purposes. According to Bond and Fox,10 the infit and outfit statistics adopt slightly different techniques for assessing an item's fit to the Rasch model. The infit statistic gives relatively more weight to the performances of persons closer to the item value. The argument is that a person whose ability is close to the item's difficulty should give a more sensitive insight into the item's performance. The outfit statistic is not weighted and therefore is more sensitive to the influence of outlying scores. It is for this reason that users of the Rasch model routinely pay more attention to infit scores than to outfit scores. Aberrant infit scores usually cause more concern than large outfit statistics.10 Furthermore, Linacre (personal communication, January 2003) developed a criterion for rejecting items with infit errors larger than 2.00. Therefore, it is possible that items with an infit score greater than 2.00 have characteristics that are sensitive to issues not reflective of the scale: they may not have fit because they are too extreme for the scale or because they lie on another dimension.
This pilot study was designed as a prelude to further studies and analyses of data related to the views of attorneys about mental health experts. One of the aims of this pilot was to sample whether the Rasch model would accurately measure attorneys' biases regarding mental health experts on a number of variables.
Methods
Participants
Because this was a survey, the Massachusetts Mental Health Center human studies committee approved its exempt status. Participants in this study were a random sample of 40 attorneys (20 men and 20 women, a coincidentally even distribution) who are members of the Pennsylvania Bar Institute. The mean age of the respondents was 48.58 (SD 9.27), with a mean number of years in practice of 19.55 (SD 9.37). An analysis of area of professional practice showed that 42.5 percent of respondents practiced private law, 25 percent practiced public law, and 12.5 percent were jurists. All respondents reported working with mental health experts over an average of 17.74 years (SD 8.76). All participants were mailed the instrument that appears in the Appendix and were asked to respond anonymously, stating only age, gender, and other demographic information.
Procedure
The instruments were mailed to the attorneys in hard copy form and returned in enclosed stamped envelopes to one of the authors (M.L.C.) for data analysis. The return rate was 100 percent. This atypical return rate may have occurred because the attorneys were willing to participate in the study due to its nature and content.
Instrument
The Appendix contains the relevant items from the instrument used. Note that the questionnaire did not use the word "bias" in its prologue. Respondents were asked to think of recent cases in which they had retained mental health expert witnesses as they answered the questions. As shown, the queries characterized experts in various ways that an attorney might be aware of. The final series of queries focused on attorney attitudes toward bias and biasing factors.
Results
There were two types of questions regarding respondents' views of experts on this instrument. One type asked about the quality of expert witnesses in general. The second type asked for views toward expert witnesses, given certain circumstances. For each of these items, a one-sample t test was conducted, assessing the mean rating against a fixed value of 3.5 (on a scale of 1 to 6, 3.5 is the mean or neutral value). Of 17 sample means tested, 15 differed significantly from that mean or neutral value. The level of significance against which each of the items was tested was corrected using the Bonferroni method, which allowed for multiplicity of statistical tests, as referred to in Rosenthal and Rosnow,14 and resulted in a more conservative criterion of significance. The new value was obtained by dividing .05 (the usual criterion of significance) by the number of items that were tested against 3.5.
The first three questions asked for respondents' view of an expert witness who works for both sides (namely, prosecution and defense, and plaintiff and defense), in terms of credibility, trustworthiness, and loyalty. For the question addressing credibility, respondents viewed these expert witnesses as significantly more credible than not (M = 5.13, SD = 1.005, t(38) = 10.120, p < .0005). It is important to note that, because of the number of comparisons to the value of 3.5, a criterion of significance was used that is more conservative than the typical .05. This was corrected to .0029, using the Bonferroni method, and each of the items that are reported as significant was compared with this value of .0029. Furthermore, the effect size,15 (d' = 1.62), for this item was large, therefore accounting for a large portion of the item's variability. (Effect size was calculated using: d' = m – c/, where m is the estimated population mean, c is the expected population mean, and is the estimated standard deviation of the population.) Participants also found such experts to be significantly more trustworthy (M = 5.08, SD = 1.010, t(38) = 9.750, p < .0005) and loyal (M = 4.50, SD = 1.285, t(33) = 4.537, p < .0005). The effect size for the item pertaining to trustworthiness was also large (d' = 1.56), therefore accounting for a large portion of the item's variability. The item pertaining to loyalty had a large effect size as well (d' = .778), accounting for a large amount of variability.
Consider that, as suggested by responses to the three previous questions, experts who routinely work for both the defense and the prosecution are seen as generally credible, trustworthy, and loyal. Then, would an expert who typically worked for one side and then changed sides, be viewed as more or less desirable It was found that respondents were relatively varied (or neutral) about whether or not this would affect their opinions of the expert. The mean rating (on a scale of 1 to 6, where 1 represents "does not affect my opinion " and 6 represents "seriously affects my opinion") did not differ significantly from 3.5 (M = 2.97, SD = 1.739, t(38) = –1.187, p = .067). Because the standard deviation was 1.739 and the mean value was not significantly different from 3.5, one could infer that some respondents would change their opinions and others would not, therefore causing the net effect to be close to neutral (3.5). In other words, the mean rating was close to the midpoint of the scale, therefore inferring either mixed feelings or a "slight" change of opinion (since the extremes of the scale are "no effect" and "serious effect"). However, there was a significant tendency for participants to say that they would rehire such an expert, despite this factor (M = 5.11, SD = 0.906, t(38) = 10.793, p < .0005). The effect size for this item was large (d' =1.78), therefore explaining most of the variance.
The second type of question addressed the general matter of the appropriateness of expert witnesses' (1) testifying to what they are told to say by retaining attorneys versus (2) testifying to what they believe to be true. For example, respondents believed that it was untrue that "expert witnesses who say whatever they are paid to say are doing their jobs" (M = 1.40, SD = 1.033, t(39) = –12.860, p < .0005) and that "those who say what they believe do not know how to work with attorneys" (M = 1.30, SD = 0.564, t(39) = –24.676, p < .0005). The effect sizes for these items were both very large (d' = –2.03 and d' = –3.90, respectively), therefore explaining much of the items' variance.
It was also found that respondents believed it to be true that "those who say whatever they are paid to say are prostituting themselves" (M = 5.48, SD = 1.154, t(39) = 10.820, p < .0005), and that "those who say what they believe to be true are objective and well-balanced" (M = 5.49, SD = 0.942, t(38) = 13.169, p < .0005). The effect sizes for these items were both very large (d' = 1.72 and d' = 2.11, respectively), therefore explaining much of the variance. It is important to note, however, that the item stating, "those who say whatever they are paid to say are prostituting themselves" had an infit MNSQ of 2.16, which indicates that the item may have had characteristics that were sensitive to issues not reflective of the scale. Yet, since this item and its answer scale were similar to that of other items, the reason for its anomalous infit value may have been extreme ratings, which could cause it to not fit on the scale.
A related set of questions addressed respondents' opinions of the degree to which expert witnesses offer objective opinions in given situations. For example, they believed that "those who are paid to do what they are told," in fact say what they are told to say (M = 2.23, SD = 1.693, t(38) = –4.628, p < .0005). The effect size for this item was large (d' = –75), which means that a large amount of this item's variance was explained. Likewise, respondents believed that "those who prostitute themselves," say whatever they are told to say (M = 1.10, SD = .304, t(39) = –49.960, p < .0005, d' = –7.89).
In contrast, respondents believed that "those expert witnesses who say what they believe and are objective and well-balanced" offer objective opinions (M = 5.56, SD = 0641, t(38) = 20.125, p < .0005, d' = 3.21). Respondents were essentially neutral in rating "experts who do not know how to work with attorneys" as neither offering objective opinions, nor in saying whatever they are told to say (M = 3.12, SD = 1.871, t(33) = –7.105, p = .242). It is important to note, however, that this item had a relatively large standard deviation (1.871) and therefore a wide range of responses. This large standard deviation might result in the mean's being closer to neutral, as we have seen here. Also, the large standard deviation reduces the chances that whatever difference (from the fixed value of 3.5) there was would be found significant.
The next set of questions inquired into whether experts who testified in various ways were in fact desirable to attorneys or convincing to jurors. For example, respondents believed that expert witnesses who testify in the direction that the retaining attorney desires were not necessarily convincing to jurors (M = 2.21, SD = 1.119, t(37) = –7.105, p < .0005, d' = –1.15), yet those who testify to what they believe to be true were, in fact, more convincing to jurors (M = 5.08, SD = 0.722, t(36) = 13.324, p < .0005, d' = 2.19).
Another interesting finding was that respondents had more positive regard for expert witnesses who were repeatedly court appointed, as opposed to those hired by the opposition (M = 4.65, SD = 0.949, t(39) = 7.667, p < .0005). The effect size was large (d' = 1.21), indicating that much of this item's variance was explained.
The Rasch analysis in this study linearly ordered how severe the perceived bias was for each item. On the right-hand side of the Rasch map (Fig. 1 ) are the item-scaled scores. Each item label represents a questionnaire item (see Table 1for label key). At the top of the map are the items that display the most perceived bias. At the bottom are the items with the least perceived bias. It is more difficult to be perceived as less biased, which is what makes the item negative. On the left-hand side of the Rasch map are the respondent ratings. Each X represents one respondent. These ratings were determined according to each respondent's perception of bias. Notice that the X's form a near normal distribution in the center of the map, indicating that most respondents rated the items in a similar manner and were able to recognize the items of moderate bias most of the time. Also, note the variables M, S, and T on the Rasch map. The variable M on the right side of the map indicates the mean rating for the items tested and gives a reference point against which to compare the items. The variable M on the left side of the map represents the mean rating for respondents. The variable S is located one standard deviation from the mean for both items and respondents, whereas the variable T is located two standard deviations from the mean. These variables allow one to determine the distribution of items, as well as to compare item ratings with the mean rating.
Not only does the Rasch analysis order the items according to severity of perceived bias, but it also allows one to view the actual linear magnitude by which each item differs from the other items in severity. This is the advantage provided by the Rasch analysis, as opposed to other analyses. Instead of simply ordering items, it gives the magnitude of differences between items so that we have a clear idea of how much of a difference those magnitudes of perceived bias make. This can be useful to both attorneys and experts, when either presenting an argument or cross-examining, because they will have an empirical scale from which to guide their examinations/testimonies, based on degree of severity for different types of perceived bias.
Furthermore, the ruler-like nature of the Rasch scale allows one to compare items with equally spaced perceived biases. In other words, it becomes clear, based on each item's scaled score, how much each item differs from other items. For example, the item that asked participants whether "those who prostitute themselves" say what they are told to say (1) or say what they believe to be true (6), had a scaled score of 2.46. A second item, which asked participants to assess the truth value of the statement: "those expert witnesses who say whatever they are paid to say are doing their jobs" (where 1 denotes untrue and 6 true), had a scaled score of 1.47, approximately one unit away from first item. A third item, which asked participants to rate the likelihood of effect on their opinion of an "expert witness who has worked primarily for one side or the other in the past now working for the opposite side," had a scaled score of 0.38, approximately one unit away from the second item. A fourth item, which asked whether they looked more positively or negatively on "an expert witness who is repeatedly appointed by the court, as opposed to one usually hired by the legal adversaries," had a scaled score of –0.42, which is approximately one unit away from the third item. Finally, a fifth item, which asked participants to assess the expert witnesses who "say what they believe are objective and well-balanced," had a scaled score of –1.44, which is approximately one unit away from the fourth item. This illustrates the ruler-like properties of the Rasch scale, in which items are equally spaced by a single unit.
Representative Comments
The statistical findings in this study may be supplemented by a review of the written comments provided by some respondents. It should be noted that while a space was provided for such comments, not all respondents offered commentary. Comments that were volunteered reveal attorney attitudes in more detail:
Respondent 002:
Generally, most experts understand that their job is to provide an opinion that is as positive as possible about the client. Believe most experts do not give opinions that go beyond supported data, though most experts would tend to minimize or overlook problematic aspects or overstate the positives.
Respondent 010:
...I don't care what an expert "believes,"—I want to know what they have concluded after competent objective evaluation.
Respondent 016:
Experts who do not stay true to their opinions (honest opinions) are of little help to either the prosecution or defense. I would still like to believe that the truth will usually prevail provided the expert is honest. I also would like to believe that a dishonest expert always goes over like a lead balloon.
Respondent 018:
...I think the best expert is the one who speaks from the heart [and] juries can usually detect that. If an expert does not honestly believe in the position of the hiring atty, do not take the case.
Respondent 026:
The proper use of experts has become the Rosetta Stone of criminal defense. To obtain any leeway at all in sentencing discretion, the defendant's pathology must be explained and a roadmap for rehabilitation shown. Also, to gain empathy of the jury, the compulsions and phobias driving criminal obsession need to be de-mythologized and humanized. Experts give a helluva framework for doing this.
Respondent 030:
Experts' credentials and courtroom demeanor are critical factors for consideration.
Discussion
The results of the study shed light on a topic that has been a point of discussion among attorneys and experts for many years. In an adversarial courtroom atmosphere, one might expect that suspicions of bias would exist strongly among experts. Consequently, experts who work for only one side may clearly be perceived to be evincing a bias, simply by nature of their one-sided role. However, attorneys may also harbor conflicting feelings about experts who do work for both sides and may believe that such experts prostitute themselves by working for whoever pays them without allegiance to one side or the other; such experts may be perceived as saying what they are paid to say.
This brief pilot study represents an assessment of just what these perceptions actually are. The study reveals the very telling finding that the majority of respondents in this study agreed that experts who work for both sides are perceived to be "more balanced" and "more credible" than those who consistently work for only one side. Those who work for both sides of cases were also found to be generally more trustworthy and loyal. Hence, trustworthiness was an important aspect of an expert's integrity, next to loyalty. Experts who were found to testify always for the same side were considered less desirable. Even more telling is the result that, if an expert were subsequently to shift to another side, the move would not negatively affect the respondents' view of the expert; in fact, the respondents would be inclined to rehire that expert for future cases because the expert was more objective and was perceived so by the court.
The results also indicated that respondents did not believe that experts who say what they are told to or are paid to say are actually doing their job. In fact, respondents tended to believe that such experts are perceived as being biased. Respondents did not indicate that such experts were merely naive and did not know how to work with attorneys. It is not surprising that respondents also thought that experts who say what they are paid to say are considered to be prostituting themselves. It may be argued that the item addressing this question could be characterized as a "truism," and respondents had no choice but to agree with it (due to the working definition of the term "prostitute" in the survey item). However, this item served as a type of control, against which to compare the mean responses to items such as: "those who say what they believe to be true and objective are considered to be better balanced." We expected that mean responses to these two items would be on opposite ends of the spectrum—and indeed, they were just that.
For the most part, the results suggest that respondents were quite traditional in their attitudes toward expert witnesses. For example, they believed that experts who say what they are paid to say are not necessarily convincing to jurors. Respondents tended to perceive experts as being more convincing if they say what they truly believe. What makes the difference in terms of credibility of the expert was not specifically clear and perhaps might be an area for additional study. It was also not surprising that court-appointed experts were viewed by all respondents as being more highly regarded. This view likely derives from the perception that court-appointed experts are not under any pressure to sway their testimony one way or another. They are probably the epitome of the neutral expert and, in a sense, serve as experts to the court itself. It is important to note that a larger sample would increase the reliability of the item estimates to some degree. The items on the Rasch scale (Fig. 1) could be slightly out of order by up to one logit; however, the overall story should stay the same with an increased sample size because of the relatively small infit errors.
Respondents were provided a space for additional written comments that they wanted to offer. In these comments, respondents indicated that they believed that most experts give opinions that do not go beyond supported data; but they cautioned that some experts may tend to minimize or overlook problematic aspects of their opinions or to overstate positives to bolster the side for which they are testifying. Respondents apparently hope that the truth will usually prevail, provided that the expert is honest. Respondents also stated that they would like to believe that the dishonest experts always "go over like a lead balloon" and are seen by juries as being phony. Respondents also believed that juries could detect when an expert is not being honest and that— if experts don't believe that they can be helpful to one side of the case, they won't accept the assignment—an optimistic view, indeed.
The desire of subjects to be politically correct may have had a contaminating influence on the results of the present study. Since respondents were asked to report about themselves (rather than, as in other studies, what others in their profession might do), they may have felt pressure to give more desirable responses. We hope that the fact that these respondents answered anonymously allayed any anxiety that their identities would be revealed and thus allowed them to respond in a genuine and free fashion. Overall, the study did indicate that attorneys lack sensitivity to bias, a finding that could not be concealed by politically correct answers. Also, it should be mentioned that there are other factors that may influence attorneys' perception of expert bias that were not addressed in this questionnaire. However, we were able to come upon important and highly significant findings with the items that we chose to include. Therefore, with the few exceptions noted, our pilot study supports traditional views of attorneys' perceptions of expert witnesses and their role in litigation. Their ratings say they want unbiased experts. Their behavior shows that they want biased experts.
Appendix 1: Questionnaire on Mental Health Expert Witnesses
We would greatly appreciate your completing this questionnaire and returning it in the enclosed stamped envelope. Help us keep this completely anonymous by avoiding any self-identifying data or material. All material will be presented in the aggregate without individual data. Please accept our personal thanks for your help.
This study will ask you questions about various aspects of your hiring of forensic experts. The purpose of this study is to further our understanding of our forensic practices and to facilitate the discussion among members of such professional practice by exploring its empirical basis.
Each X on the left-hand side of the scale represents one participant rating, and the labels on the right-hand side of the scale represent each item's rating (see Table 1 for item label key).
Acknowledgments
The authors acknowledge their indebtedness to members of both Programs and members of the Dare Institute for help with data analysis and critical review and comments. Much appreciation is also owed to Eric Frey of Lehigh University for his assistance with the literature search. Thanks are extended to members of the Pennsylvania Bar Association for their participation in the study, as well as to Michael Linacre who provided consent for the use of his personal communication.
References
Gutheil TG: The presentation of forensic psychiatric evidence in court. Isr J Psychiatry Relat Sci 37:137–44, 2000
Resnick PJ, Knoll JL: The psychiatric expert witness. Isr J Psychiatry Relat Sci 37:145–53, 2000
Kennedy WA: The psychologist as expert witness, in Forensic Psychology and Psychiatry: Perspective and Standards for Interdisciplinary Practice. Edited by Curran WJ, McGarry AL, Shah SA. Philadelphia: FA Davis, 1986
Gutheil TG, Simon RI: Mastering Forensic Psychiatric Practice: Advanced Strategies for the Expert Witness. Washington, DC: American Psychiatric Press, Inc., 2002
American Academy of Psychiatry and the Law: Ethical Guidelines for the Practice of Forensic Psychiatry. Bloomfield, CT: AAPA, 1991
Commons ML, Miller PM, Gutheil TG: Expert witness perceptions of bias in experts. J Am Acad Psychiatry Law 32:70–5, 2004
Rasch G: Probabilistic Models for Some Intelligence and Attainment Tests. Chicago: University of Chicago Press, 1960
Andrich D: Rasch Models for Measurement. Thousand Oaks, CA: Sage Publications, 1988
Wright BD, Stone MH: Best Test Design. Chicago: Mesa Press, 1979
Bond TG, Fox CM: Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Mahwah, NJ: Lawrence Erlbaum, 2001
Commons ML, Miller PM, Gutheil TG: A pilot Rasch scaling of forensic experts' perceptions of expert bias. J Am Acad Psychiatry Law, in press
Linacre JM: Winsteps: Multiple-Choice, Rating Scale and Partial Credit Rasch Analysis. Chicago: Mesa Press, 2003
Wilson M, ed: Objective Measurement: Theory into Practice. Norwood, NJ: Ablex Publishing Corp., 1992
Rosenthal R, Rosnow RL: Contrast Analysis: Focused Comparisons in the Analysis of Variance. New York: Cambridge University Press, 1985
Cohen J: Statistical Power Analysis for the Behavioral Sciences (ed 2). Hillsdale, NJ: Erlbaum, 1988(Frank M. Dattilio, PhD, Michael Lamport )
Dr. Gutheil is Professor of Psychiatry and Co-Founder, Program in Psychiatry and the Law, BIDMC Department of Psychiatry, Harvard Medical School, Boston,MA. Dr. Sadoff is Clinical Professor of Psychiatry and Director, Center for Studies in Socio-Legal Psychiatry, University of Pennsylvania School of Medicine, Philadelphia, PA. This study was funded by a general grant awarded through the Pennsylvania Bar Institute to the first author.
Abstract
How seriously do attorneys consider the biases of their retained mental health experts Participants in this pilot study included 40 attorneys, randomly selected from a pool of members of the Pennsylvania Bar Institute, who rated—for their biasing potential—several situations that might affect the behavior of an expert. A Rasch analysis produced a linear scale as to the perceived biasing potential of these different items from most to least biasing. Among other results, the study suggests that attorneys do view mental health experts who work on both sides of cases as being more balanced in their testimony. However, they also indicated that they have a preference for using individuals who repeatedly testify for one side. Working for only one side in both civil and criminal cases yielded large scaled values. Additional comments offered by respondents indicated that: (1) an opposing expert also serving as the litigant's treater and (2) an opposing expert being viewed as a "hired gun" (supplying an opinion only for money) were viewed by subjects as not being very biased. A discussion of the results raises the need for future research in this area.
The demand for psychiatric and psychological expert testimony has multiplied over the past three decades, in both criminal and civil cases, because of the increasing need to translate for the court in cases involving mental health.1 The role of any forensic expert is to educate the court or the fact finder about matters that are beyond the lay person's knowledge.2 Based on the active discussions that appear in the forensic literature3–6 regarding the expert witness's role, certain questions have yet to be addressed. Among those are problems pertaining to the notion of being "typecast" by attorneys as appearing consistently biased for one side or the other. Are experts who have been retained over time by both sides of legal cases viewed by attorneys as being more credible than those who seem to "favor" only one side Do attorneys seek out experts for their presumed biases or do they genuinely value the objectivity of experts to "call it as they see it" What do attorneys view as biasing factors affecting experts Does a "history as an expert" (one of six important factors for experts identified by Kennedy3) influence an attorney's choice as to whom to retain
In their respective professions, experts typically subscribe to ethics codes that urge them to serve the court as impartial providers of reliable information within their fields of competence.4,5 Expert witnesses also propose that the objectivity they bring to the legal system is regarded as one of their most valued qualities, together with honesty and neutrality. A challenging but necessary task for experts, therefore, would be to deal with potential "expert bias." A looming question that remains, however, is whether retaining attorneys, who are explicitly and appropriately partisan rather than neutral, subscribe to the same values
In a previous survey on the perceived bias of forensic experts, Commons et al.6 presented data showing that expert witnesses themselves perceive the existence of a good deal of such bias in their own professions. In those studies, some of the potentially biasing situations had higher significance values and larger effect sizes. In other words, some situations were perceived as more biasing than others. For example, it was found that experts believed that working for only one side (either the prosecution/plaintiff or the defense) in both civil and criminal cases is very biasing. Also, in an interesting contrast, it was found that (1) an opposing expert also serving as the litigant's treater and (2) an opposing expert who is acting as a "hired gun" (supplying an opinion only for money) were two situations viewed as not very biasing. The results of this study created a template from which to construct questions directed to other groups involved in the legal system, regarding expert bias. In particular, the purpose of the current study is to determine how attorneys perceive the potentially biasing effect of the model situations proposed in the aforementioned study.
The Present Study
The results were measured on a ruler-like scale that facilitated more convenient responding, using a technique known as the Rasch analysis.7 It was our hope that forensic experts, attorneys, and scholars would benefit by being informed about how attorneys view the seriousness of various biasing situations. These findings may enable more rational decision-making by those three groups and shed light on an area that has not been previously explored from an empirical standpoint.
In this pilot study, a Rasch analysis was used to show the degree of perceived bias in an objective, empirical manner. To understand the results, a basic knowledge of Rasch scales is described.
The Rasch Model of Analysis
The Rasch Model of Analysis produces an objective, additive scale that is independent of the particular items used and of the particular participants tested.7 The Rasch method can be used to analyze a large variety of human sciences data.8–12 For example, through the use of probabilistic equations, this model converts raw ratings of items into scales that have equal intervals. This analysis is particularly effective when the raw data are entered as values on a continuous scale. (Either participants were asked to rate an item on a scale, or nonscale answers were coded with continuous whole number integers.) Once the raw data input is coded in a uniform manner (percentages, words, and decimals are all entered, or coded, as whole numbers), the Rasch analysis converts these codes into small numeric values (generally between –4 and +4), according to an order of magnitude. A scale is then produced, on which each item (that was coded for and entered as a raw data point) is placed according to its Rasch "rating " or scaled score. Such a scale can then be used as a type of objective ruler against which to measure the data on items as well as on respondents' ratings. The ruler-like properties of this scale are what provide its advantage over other scaling techniques. For example, the scale is made up of equally spaced, continuous intervals. Also, from a statistical standpoint, this scale provides a linear interval measure. As a result, a change in severity of a perceived bias of 1 carries the same weight from –2 to –1 as it does from 0 to +1. As with a ruler, a change in length of one inch, either from two inches to three inches, or three to four is the same. Regardless of the locations of the starting and finishing places along the ruler, the magnitude of the distance change is equal in both instances. Furthermore, doubling on the Rasch scale means the same change in severity anywhere along its linear axis. Again, using the figurative ruler example, doubling the distance from one inch to two inches results in a magnitude of change equivalent to doubling the distance from two inches to four inches. In this case, a perceived bias with a value of 2.3 is half as severe as a perceived bias of 4.6, just as two inches is half as long as four inches on a standard ruler.
This relationship can be further corroborated by examining the distances between item difficulties anywhere along the scale's linear axis. On doing so, a zero point must first be determined. For items, we can choose the mean item bias, the mean person "bias," or another reference point. Let us choose the mean person bias, M. If we identify an item of bias M, participants of bias M should succeed at it about 50 percent of the time.13 For this study, we could equate this to an item with M level of bias, which participants of M amount of bias should agree with about 50 percent of the time. Next, we can identify an item of bias A, so that its height relative to M is A – M. Then, we can ask what proportion of participants of bias M succeeded at item A. Say this proportion was 25 percent. Keep in mind, the proportion of participants with bias A who succeeded on or agreed with item A should be about 50 percent, since this is how we know they are of ability A. Next, we can find item B, which people of bias A succeed at (or agree with) 25 percent of the time. We will find that its level of bias is approximately 2(A – M) + M. This item must, therefore, be twice as high (severe, or biased) as item A (Michael Linacre, personal communication, January 2003). This mathematical relationship gives further evidence for the notion that doubling on the Rasch scale means the same change in severity anywhere along its linear axis.
In analyzing data using a Rasch model, several questions can easily be answered. For example, this model indicates where on the scale each item falls (e.g., in this case, how severe the perceived biases were for any given item). This is a question that cannot be answered through use of other scaling techniques and will therefore provide a novel approach to information seeking by systematic, objective scaling. Second, the Rasch model aids in denoting the range of scaled values that exist between all variables for all participants. Third, the scaled value for each participant can also be measured with regard to the overall severity of these biases.
The rating scale provided respondents with a measure of how biasing each situation appears to them. Such a measure allows respondents to point to empirical data when confronted with some of these situations. This method also allows us to determine how much of a difference a change in the score will make. The smaller the range of scaled perceived biases, the larger the difference a change in score of a particular unit, such as 1, makes.
Finally, the extent to which the measured items fit on the scale was also addressed by the infit mean square (MNSQ) values.10 In the Rasch analysis output, both infit and outfit mean square statistics are reported. These mean squares are the unstandardized form of the fit statistic (generally t) and merely the average value of the squared residuals for that item.10 According to Bond and Fox, "the residual values represent the differences between the Rasch model's theoretical expectation of item performance and the performance actually encountered for that item in the data matrix" (Ref. 10, p. 43). In other words, the larger the residual value (and subsequently, the square of this value), the larger the difference between how the item should have performed and how it actually did perform (on the Rasch scale),10 and the fit statistics (both infit and outfit) are representative of the squares of these residual values (since they are merely an average value).
Although there is some controversy as to whether the infit or the outfit MNSQ should be used to determine how well items fit the Rasch scale, we will use infit MNSQ for our purposes. According to Bond and Fox,10 the infit and outfit statistics adopt slightly different techniques for assessing an item's fit to the Rasch model. The infit statistic gives relatively more weight to the performances of persons closer to the item value. The argument is that a person whose ability is close to the item's difficulty should give a more sensitive insight into the item's performance. The outfit statistic is not weighted and therefore is more sensitive to the influence of outlying scores. It is for this reason that users of the Rasch model routinely pay more attention to infit scores than to outfit scores. Aberrant infit scores usually cause more concern than large outfit statistics.10 Furthermore, Linacre (personal communication, January 2003) developed a criterion for rejecting items with infit errors larger than 2.00. Therefore, it is possible that items with an infit score greater than 2.00 have characteristics that are sensitive to issues not reflective of the scale: they may not have fit because they are too extreme for the scale or because they lie on another dimension.
This pilot study was designed as a prelude to further studies and analyses of data related to the views of attorneys about mental health experts. One of the aims of this pilot was to sample whether the Rasch model would accurately measure attorneys' biases regarding mental health experts on a number of variables.
Methods
Participants
Because this was a survey, the Massachusetts Mental Health Center human studies committee approved its exempt status. Participants in this study were a random sample of 40 attorneys (20 men and 20 women, a coincidentally even distribution) who are members of the Pennsylvania Bar Institute. The mean age of the respondents was 48.58 (SD 9.27), with a mean number of years in practice of 19.55 (SD 9.37). An analysis of area of professional practice showed that 42.5 percent of respondents practiced private law, 25 percent practiced public law, and 12.5 percent were jurists. All respondents reported working with mental health experts over an average of 17.74 years (SD 8.76). All participants were mailed the instrument that appears in the Appendix and were asked to respond anonymously, stating only age, gender, and other demographic information.
Procedure
The instruments were mailed to the attorneys in hard copy form and returned in enclosed stamped envelopes to one of the authors (M.L.C.) for data analysis. The return rate was 100 percent. This atypical return rate may have occurred because the attorneys were willing to participate in the study due to its nature and content.
Instrument
The Appendix contains the relevant items from the instrument used. Note that the questionnaire did not use the word "bias" in its prologue. Respondents were asked to think of recent cases in which they had retained mental health expert witnesses as they answered the questions. As shown, the queries characterized experts in various ways that an attorney might be aware of. The final series of queries focused on attorney attitudes toward bias and biasing factors.
Results
There were two types of questions regarding respondents' views of experts on this instrument. One type asked about the quality of expert witnesses in general. The second type asked for views toward expert witnesses, given certain circumstances. For each of these items, a one-sample t test was conducted, assessing the mean rating against a fixed value of 3.5 (on a scale of 1 to 6, 3.5 is the mean or neutral value). Of 17 sample means tested, 15 differed significantly from that mean or neutral value. The level of significance against which each of the items was tested was corrected using the Bonferroni method, which allowed for multiplicity of statistical tests, as referred to in Rosenthal and Rosnow,14 and resulted in a more conservative criterion of significance. The new value was obtained by dividing .05 (the usual criterion of significance) by the number of items that were tested against 3.5.
The first three questions asked for respondents' view of an expert witness who works for both sides (namely, prosecution and defense, and plaintiff and defense), in terms of credibility, trustworthiness, and loyalty. For the question addressing credibility, respondents viewed these expert witnesses as significantly more credible than not (M = 5.13, SD = 1.005, t(38) = 10.120, p < .0005). It is important to note that, because of the number of comparisons to the value of 3.5, a criterion of significance was used that is more conservative than the typical .05. This was corrected to .0029, using the Bonferroni method, and each of the items that are reported as significant was compared with this value of .0029. Furthermore, the effect size,15 (d' = 1.62), for this item was large, therefore accounting for a large portion of the item's variability. (Effect size was calculated using: d' = m – c/, where m is the estimated population mean, c is the expected population mean, and is the estimated standard deviation of the population.) Participants also found such experts to be significantly more trustworthy (M = 5.08, SD = 1.010, t(38) = 9.750, p < .0005) and loyal (M = 4.50, SD = 1.285, t(33) = 4.537, p < .0005). The effect size for the item pertaining to trustworthiness was also large (d' = 1.56), therefore accounting for a large portion of the item's variability. The item pertaining to loyalty had a large effect size as well (d' = .778), accounting for a large amount of variability.
Consider that, as suggested by responses to the three previous questions, experts who routinely work for both the defense and the prosecution are seen as generally credible, trustworthy, and loyal. Then, would an expert who typically worked for one side and then changed sides, be viewed as more or less desirable It was found that respondents were relatively varied (or neutral) about whether or not this would affect their opinions of the expert. The mean rating (on a scale of 1 to 6, where 1 represents "does not affect my opinion " and 6 represents "seriously affects my opinion") did not differ significantly from 3.5 (M = 2.97, SD = 1.739, t(38) = –1.187, p = .067). Because the standard deviation was 1.739 and the mean value was not significantly different from 3.5, one could infer that some respondents would change their opinions and others would not, therefore causing the net effect to be close to neutral (3.5). In other words, the mean rating was close to the midpoint of the scale, therefore inferring either mixed feelings or a "slight" change of opinion (since the extremes of the scale are "no effect" and "serious effect"). However, there was a significant tendency for participants to say that they would rehire such an expert, despite this factor (M = 5.11, SD = 0.906, t(38) = 10.793, p < .0005). The effect size for this item was large (d' =1.78), therefore explaining most of the variance.
The second type of question addressed the general matter of the appropriateness of expert witnesses' (1) testifying to what they are told to say by retaining attorneys versus (2) testifying to what they believe to be true. For example, respondents believed that it was untrue that "expert witnesses who say whatever they are paid to say are doing their jobs" (M = 1.40, SD = 1.033, t(39) = –12.860, p < .0005) and that "those who say what they believe do not know how to work with attorneys" (M = 1.30, SD = 0.564, t(39) = –24.676, p < .0005). The effect sizes for these items were both very large (d' = –2.03 and d' = –3.90, respectively), therefore explaining much of the items' variance.
It was also found that respondents believed it to be true that "those who say whatever they are paid to say are prostituting themselves" (M = 5.48, SD = 1.154, t(39) = 10.820, p < .0005), and that "those who say what they believe to be true are objective and well-balanced" (M = 5.49, SD = 0.942, t(38) = 13.169, p < .0005). The effect sizes for these items were both very large (d' = 1.72 and d' = 2.11, respectively), therefore explaining much of the variance. It is important to note, however, that the item stating, "those who say whatever they are paid to say are prostituting themselves" had an infit MNSQ of 2.16, which indicates that the item may have had characteristics that were sensitive to issues not reflective of the scale. Yet, since this item and its answer scale were similar to that of other items, the reason for its anomalous infit value may have been extreme ratings, which could cause it to not fit on the scale.
A related set of questions addressed respondents' opinions of the degree to which expert witnesses offer objective opinions in given situations. For example, they believed that "those who are paid to do what they are told," in fact say what they are told to say (M = 2.23, SD = 1.693, t(38) = –4.628, p < .0005). The effect size for this item was large (d' = –75), which means that a large amount of this item's variance was explained. Likewise, respondents believed that "those who prostitute themselves," say whatever they are told to say (M = 1.10, SD = .304, t(39) = –49.960, p < .0005, d' = –7.89).
In contrast, respondents believed that "those expert witnesses who say what they believe and are objective and well-balanced" offer objective opinions (M = 5.56, SD = 0641, t(38) = 20.125, p < .0005, d' = 3.21). Respondents were essentially neutral in rating "experts who do not know how to work with attorneys" as neither offering objective opinions, nor in saying whatever they are told to say (M = 3.12, SD = 1.871, t(33) = –7.105, p = .242). It is important to note, however, that this item had a relatively large standard deviation (1.871) and therefore a wide range of responses. This large standard deviation might result in the mean's being closer to neutral, as we have seen here. Also, the large standard deviation reduces the chances that whatever difference (from the fixed value of 3.5) there was would be found significant.
The next set of questions inquired into whether experts who testified in various ways were in fact desirable to attorneys or convincing to jurors. For example, respondents believed that expert witnesses who testify in the direction that the retaining attorney desires were not necessarily convincing to jurors (M = 2.21, SD = 1.119, t(37) = –7.105, p < .0005, d' = –1.15), yet those who testify to what they believe to be true were, in fact, more convincing to jurors (M = 5.08, SD = 0.722, t(36) = 13.324, p < .0005, d' = 2.19).
Another interesting finding was that respondents had more positive regard for expert witnesses who were repeatedly court appointed, as opposed to those hired by the opposition (M = 4.65, SD = 0.949, t(39) = 7.667, p < .0005). The effect size was large (d' = 1.21), indicating that much of this item's variance was explained.
The Rasch analysis in this study linearly ordered how severe the perceived bias was for each item. On the right-hand side of the Rasch map (Fig. 1 ) are the item-scaled scores. Each item label represents a questionnaire item (see Table 1for label key). At the top of the map are the items that display the most perceived bias. At the bottom are the items with the least perceived bias. It is more difficult to be perceived as less biased, which is what makes the item negative. On the left-hand side of the Rasch map are the respondent ratings. Each X represents one respondent. These ratings were determined according to each respondent's perception of bias. Notice that the X's form a near normal distribution in the center of the map, indicating that most respondents rated the items in a similar manner and were able to recognize the items of moderate bias most of the time. Also, note the variables M, S, and T on the Rasch map. The variable M on the right side of the map indicates the mean rating for the items tested and gives a reference point against which to compare the items. The variable M on the left side of the map represents the mean rating for respondents. The variable S is located one standard deviation from the mean for both items and respondents, whereas the variable T is located two standard deviations from the mean. These variables allow one to determine the distribution of items, as well as to compare item ratings with the mean rating.
Not only does the Rasch analysis order the items according to severity of perceived bias, but it also allows one to view the actual linear magnitude by which each item differs from the other items in severity. This is the advantage provided by the Rasch analysis, as opposed to other analyses. Instead of simply ordering items, it gives the magnitude of differences between items so that we have a clear idea of how much of a difference those magnitudes of perceived bias make. This can be useful to both attorneys and experts, when either presenting an argument or cross-examining, because they will have an empirical scale from which to guide their examinations/testimonies, based on degree of severity for different types of perceived bias.
Furthermore, the ruler-like nature of the Rasch scale allows one to compare items with equally spaced perceived biases. In other words, it becomes clear, based on each item's scaled score, how much each item differs from other items. For example, the item that asked participants whether "those who prostitute themselves" say what they are told to say (1) or say what they believe to be true (6), had a scaled score of 2.46. A second item, which asked participants to assess the truth value of the statement: "those expert witnesses who say whatever they are paid to say are doing their jobs" (where 1 denotes untrue and 6 true), had a scaled score of 1.47, approximately one unit away from first item. A third item, which asked participants to rate the likelihood of effect on their opinion of an "expert witness who has worked primarily for one side or the other in the past now working for the opposite side," had a scaled score of 0.38, approximately one unit away from the second item. A fourth item, which asked whether they looked more positively or negatively on "an expert witness who is repeatedly appointed by the court, as opposed to one usually hired by the legal adversaries," had a scaled score of –0.42, which is approximately one unit away from the third item. Finally, a fifth item, which asked participants to assess the expert witnesses who "say what they believe are objective and well-balanced," had a scaled score of –1.44, which is approximately one unit away from the fourth item. This illustrates the ruler-like properties of the Rasch scale, in which items are equally spaced by a single unit.
Representative Comments
The statistical findings in this study may be supplemented by a review of the written comments provided by some respondents. It should be noted that while a space was provided for such comments, not all respondents offered commentary. Comments that were volunteered reveal attorney attitudes in more detail:
Respondent 002:
Generally, most experts understand that their job is to provide an opinion that is as positive as possible about the client. Believe most experts do not give opinions that go beyond supported data, though most experts would tend to minimize or overlook problematic aspects or overstate the positives.
Respondent 010:
...I don't care what an expert "believes,"—I want to know what they have concluded after competent objective evaluation.
Respondent 016:
Experts who do not stay true to their opinions (honest opinions) are of little help to either the prosecution or defense. I would still like to believe that the truth will usually prevail provided the expert is honest. I also would like to believe that a dishonest expert always goes over like a lead balloon.
Respondent 018:
...I think the best expert is the one who speaks from the heart [and] juries can usually detect that. If an expert does not honestly believe in the position of the hiring atty, do not take the case.
Respondent 026:
The proper use of experts has become the Rosetta Stone of criminal defense. To obtain any leeway at all in sentencing discretion, the defendant's pathology must be explained and a roadmap for rehabilitation shown. Also, to gain empathy of the jury, the compulsions and phobias driving criminal obsession need to be de-mythologized and humanized. Experts give a helluva framework for doing this.
Respondent 030:
Experts' credentials and courtroom demeanor are critical factors for consideration.
Discussion
The results of the study shed light on a topic that has been a point of discussion among attorneys and experts for many years. In an adversarial courtroom atmosphere, one might expect that suspicions of bias would exist strongly among experts. Consequently, experts who work for only one side may clearly be perceived to be evincing a bias, simply by nature of their one-sided role. However, attorneys may also harbor conflicting feelings about experts who do work for both sides and may believe that such experts prostitute themselves by working for whoever pays them without allegiance to one side or the other; such experts may be perceived as saying what they are paid to say.
This brief pilot study represents an assessment of just what these perceptions actually are. The study reveals the very telling finding that the majority of respondents in this study agreed that experts who work for both sides are perceived to be "more balanced" and "more credible" than those who consistently work for only one side. Those who work for both sides of cases were also found to be generally more trustworthy and loyal. Hence, trustworthiness was an important aspect of an expert's integrity, next to loyalty. Experts who were found to testify always for the same side were considered less desirable. Even more telling is the result that, if an expert were subsequently to shift to another side, the move would not negatively affect the respondents' view of the expert; in fact, the respondents would be inclined to rehire that expert for future cases because the expert was more objective and was perceived so by the court.
The results also indicated that respondents did not believe that experts who say what they are told to or are paid to say are actually doing their job. In fact, respondents tended to believe that such experts are perceived as being biased. Respondents did not indicate that such experts were merely naive and did not know how to work with attorneys. It is not surprising that respondents also thought that experts who say what they are paid to say are considered to be prostituting themselves. It may be argued that the item addressing this question could be characterized as a "truism," and respondents had no choice but to agree with it (due to the working definition of the term "prostitute" in the survey item). However, this item served as a type of control, against which to compare the mean responses to items such as: "those who say what they believe to be true and objective are considered to be better balanced." We expected that mean responses to these two items would be on opposite ends of the spectrum—and indeed, they were just that.
For the most part, the results suggest that respondents were quite traditional in their attitudes toward expert witnesses. For example, they believed that experts who say what they are paid to say are not necessarily convincing to jurors. Respondents tended to perceive experts as being more convincing if they say what they truly believe. What makes the difference in terms of credibility of the expert was not specifically clear and perhaps might be an area for additional study. It was also not surprising that court-appointed experts were viewed by all respondents as being more highly regarded. This view likely derives from the perception that court-appointed experts are not under any pressure to sway their testimony one way or another. They are probably the epitome of the neutral expert and, in a sense, serve as experts to the court itself. It is important to note that a larger sample would increase the reliability of the item estimates to some degree. The items on the Rasch scale (Fig. 1) could be slightly out of order by up to one logit; however, the overall story should stay the same with an increased sample size because of the relatively small infit errors.
Respondents were provided a space for additional written comments that they wanted to offer. In these comments, respondents indicated that they believed that most experts give opinions that do not go beyond supported data; but they cautioned that some experts may tend to minimize or overlook problematic aspects of their opinions or to overstate positives to bolster the side for which they are testifying. Respondents apparently hope that the truth will usually prevail, provided that the expert is honest. Respondents also stated that they would like to believe that the dishonest experts always "go over like a lead balloon" and are seen by juries as being phony. Respondents also believed that juries could detect when an expert is not being honest and that— if experts don't believe that they can be helpful to one side of the case, they won't accept the assignment—an optimistic view, indeed.
The desire of subjects to be politically correct may have had a contaminating influence on the results of the present study. Since respondents were asked to report about themselves (rather than, as in other studies, what others in their profession might do), they may have felt pressure to give more desirable responses. We hope that the fact that these respondents answered anonymously allayed any anxiety that their identities would be revealed and thus allowed them to respond in a genuine and free fashion. Overall, the study did indicate that attorneys lack sensitivity to bias, a finding that could not be concealed by politically correct answers. Also, it should be mentioned that there are other factors that may influence attorneys' perception of expert bias that were not addressed in this questionnaire. However, we were able to come upon important and highly significant findings with the items that we chose to include. Therefore, with the few exceptions noted, our pilot study supports traditional views of attorneys' perceptions of expert witnesses and their role in litigation. Their ratings say they want unbiased experts. Their behavior shows that they want biased experts.
Appendix 1: Questionnaire on Mental Health Expert Witnesses
We would greatly appreciate your completing this questionnaire and returning it in the enclosed stamped envelope. Help us keep this completely anonymous by avoiding any self-identifying data or material. All material will be presented in the aggregate without individual data. Please accept our personal thanks for your help.
This study will ask you questions about various aspects of your hiring of forensic experts. The purpose of this study is to further our understanding of our forensic practices and to facilitate the discussion among members of such professional practice by exploring its empirical basis.
Each X on the left-hand side of the scale represents one participant rating, and the labels on the right-hand side of the scale represent each item's rating (see Table 1 for item label key).
Acknowledgments
The authors acknowledge their indebtedness to members of both Programs and members of the Dare Institute for help with data analysis and critical review and comments. Much appreciation is also owed to Eric Frey of Lehigh University for his assistance with the literature search. Thanks are extended to members of the Pennsylvania Bar Association for their participation in the study, as well as to Michael Linacre who provided consent for the use of his personal communication.
References
Gutheil TG: The presentation of forensic psychiatric evidence in court. Isr J Psychiatry Relat Sci 37:137–44, 2000
Resnick PJ, Knoll JL: The psychiatric expert witness. Isr J Psychiatry Relat Sci 37:145–53, 2000
Kennedy WA: The psychologist as expert witness, in Forensic Psychology and Psychiatry: Perspective and Standards for Interdisciplinary Practice. Edited by Curran WJ, McGarry AL, Shah SA. Philadelphia: FA Davis, 1986
Gutheil TG, Simon RI: Mastering Forensic Psychiatric Practice: Advanced Strategies for the Expert Witness. Washington, DC: American Psychiatric Press, Inc., 2002
American Academy of Psychiatry and the Law: Ethical Guidelines for the Practice of Forensic Psychiatry. Bloomfield, CT: AAPA, 1991
Commons ML, Miller PM, Gutheil TG: Expert witness perceptions of bias in experts. J Am Acad Psychiatry Law 32:70–5, 2004
Rasch G: Probabilistic Models for Some Intelligence and Attainment Tests. Chicago: University of Chicago Press, 1960
Andrich D: Rasch Models for Measurement. Thousand Oaks, CA: Sage Publications, 1988
Wright BD, Stone MH: Best Test Design. Chicago: Mesa Press, 1979
Bond TG, Fox CM: Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Mahwah, NJ: Lawrence Erlbaum, 2001
Commons ML, Miller PM, Gutheil TG: A pilot Rasch scaling of forensic experts' perceptions of expert bias. J Am Acad Psychiatry Law, in press
Linacre JM: Winsteps: Multiple-Choice, Rating Scale and Partial Credit Rasch Analysis. Chicago: Mesa Press, 2003
Wilson M, ed: Objective Measurement: Theory into Practice. Norwood, NJ: Ablex Publishing Corp., 1992
Rosenthal R, Rosnow RL: Contrast Analysis: Focused Comparisons in the Analysis of Variance. New York: Cambridge University Press, 1985
Cohen J: Statistical Power Analysis for the Behavioral Sciences (ed 2). Hillsdale, NJ: Erlbaum, 1988(Frank M. Dattilio, PhD, Michael Lamport )