COMPARING THE SOGS AND THE DSM-IV

A variety of methodological questions have been raised in recent years about research on gambling and problem gambling in the general population (Dickerson 1993; Lesieur 1994; Walker 1992). One serious concern has to do with changes in the criteria for identifying pathological gamblers that have been adopted by the American Psychiatric Association. The South Oaks Gambling Screen was based on the original DSM-III criteria published in 1980 and was tested in clinical trials against the DSM-III-R criteria published in 1987. In the DSM-III, a diagnosis of pathological gambling required an individual to meet four of seven criteria with an exclusion of Anti-Social Personality Disorder. In the DSM-III-R, the same diagnosis required an individual to meet four of nine criteria and the exclusion of Anti-Social Personality Disorder was dropped. In the DSM-IV, a diagnosis of pathological gambling requires an individual to meet five of ten criteria with an exclusion of Manic Personality Disorder.

Since so many surveys have been carried out using the South Oaks Gambling Screen,³ use of this instrument allows comparisons of gambling problems across jurisdictions as well as over time (Walker & Dickerson 1996). With the recent changes in the psychiatric criteria for pathological gambling, however, researchers have become concerned about whether the South Oaks Gambling Screen is the best tool for measuring the prevalence of pathological gambling in the community. Recent work in Minnesota suggests that while the South Oaks Gambling Screen is well-suited for identifying individuals at risk for developing a gambling pathology, the DSM-IV may be more useful if the goal of a study is to estimate the prevalence of pathological gambling in the general population (Stinchfield 1997).

In moving forward, it is essential that the performance of any new instrument, such as the DSM-IV, be compared to the South Oaks Gambling Screen as well as to clinical assessments so that findings based on these new measurements can be matched to findings based on the South Oaks Gambling Screen. In this way, the field of gambling research can move forward in an evolutionary, rather than revolutionary, manner.

The Oregon Survey

In the Oregon survey, the DSM-IV Screen was used in addition to the South Oaks Gambling Screen. The South Oaks Gambling Screen was used in order to obtain prevalence data comparable to data from many other North American jurisdictions. The DSM-IV Screen was used in order to assess pathological gambling using the most current criteria and to contribute to the development of problem gambling research. While this and similar studies do not answer questions about the validity and reliability of the DSM-IV Screen in relation to clinical assessments, use of the DSM-IV Screen does provide an important opportunity to understand how the two most widely-used methods to identify problem and pathological gamblers operate in relation to one another.

In administering the questionnaire for the Oregon survey, the two problem gambling screens were rotated so that half of the sample answered the items from the South Oaks Gambling Screen first and the other half of the sample answered the items from the DSM-IV Screen first. There were no statistically significant differences between the two halves of the sample in terms of demographics, gambling involvement or scores on either of the problem gambling screens.

Since there were no statistically significant differences between the two halves of the sample, we elected to analyze the results as a single sample. Further, because both screens were administered only to respondents who had ever gambled, all of the information reported in this section is based on the sample of gamblers (N=1,305) rather than on the total Oregon sample.

The DSM-IV Screen

The South Oaks Gambling Screen is a 20-item scale based on the diagnostic criteria for pathological gambling (American Psychiatric Association 1980). Weighted items on the South Oaks Gambling Screen include hiding evidence of gambling, spending more time or money gambling than intended, arguing with family members over gambling and borrowing money to gamble or to pay gambling debts. In developing the South Oaks Gambling Screen, specific items as well as the entire screen were tested for reliability and validity with a variety of groups, including hospital workers, university students, prison inmates and inpatients in alcohol and substance abuse treatment programs (Lesieur & Blume 1987; Lesieur, Blume & Zoppa 1986; Lesieur & Klein 1985).

The DSM-IV Screen is a 10-item scale based on the most recent diagnostic criteria for pathological gambling (American Psychiatric Association 1994). In developing the DSM-IV criteria, 222 self-identified pathological gamblers and 104 substance abusers who gambled socially tested the individual items (Lesieur & Rosenthal 1991). Discriminant analysis was used to identify the items that best differentiated between pathological and non-pathological gamblers. While the results from this sample indicated that a cutoff of 4 points was appropriate, the American Psychiatric Association (1994) subsequently established a diagnostic cutoff of 5 points. The individual DSM-IV criteria include the following behaviors:

The DSM-IV criteria were adapted slightly for use in a survey of British casino patrons (Fisher 1996) and it is this DSM-IV Screen that was used in the surveys in Colorado, New York and Oregon (Volberg 1996 NY, 1997 CO). In developing the DSM-IV Screen, Fisher made some minor adjustments to the wording of the DSM-IV criteria and increased the number of response categories from "Yes/No" to "Never," "Once or Twice," "Sometimes" and "Often." In the surveys in Colorado, New York and Oregon, respondents received a score of one for any of the DSM-IV Screen items to which they gave a positive response ("Once or Twice," "Sometimes" or "Often").⁴ Total scores were obtained by adding the positive items for each respondent.

In her analysis of problem gambling among British casino patrons, Fisher (1996) identified respondents who scored 3 or 4 points on the DSM-IV Screen as "problem gamblers" and respondents who scored 5 or more points as "severe problem gamblers." In our analysis of the DSM-IV Screen, we have followed Fisher's lead and used the terms "problem gambler" to identify respondents who score 3 or 4 points on the DSM-IV Screen and "severe problem gambler" to identify respondents who score 5 or more on the DSM-IV Screen.

Statistical Characteristics of the DSM-IV Screen

In this section, we examine the psychometric properties of the DSM-IV Screen among the Oregon respondents who have ever gambled. These psychometric properties are important in assessing the accuracy of the two different methods used to identify problem and pathological gamblers in the general population. There are different kinds of error inherent in any set of data. While random error is addressed by using statistical techniques to reject the "null hypothesis" and to calculate the probability that a particular result is not due to random error, measurement error is more difficult to assess.

The accuracy of any instrument is measured by looking at the reliability and validity of the instrument (Litwin 1995). The reliability of an instrument refers to the ability to reproduce the results of the application of the test. The validity of an instrument refers to the ability of the instrument to measure what it is intended to measure. In examining the psychometric properties of the DSM-IV Screen, we assess its reliability by examining the internal consistency of the screen and then analyze the individual items to determine the ability of the screen to discriminate effectively between non-problem and problem gamblers. We then examine several forms of validity for the DSM-IV Screen.

Reliability

The most widely accepted test of reliability is a measure if the internal consistency of an instrument. The reliability of the DSM-IV Screen in the Oregon sample of gamblers is excellent with Cronbach's alpha at .80, substantially higher than the .70 that is generally accepted as representing good reliability.

In addition to testing the internal consistency of the DSM-IV Screen, we carried out a factor analysis of the screen to assess how the individual items cluster together. Factor analysis shows that 45% of the variance for the DSM-IV Screen was accounted for by one factor in Oregon, Preoccupation. The only other factor with an eigenvalue over 1.0 was Tolerance which accounted for an additional 13% of the variance. These findings suggest that the scale is homogeneous and measures the desired behavior.

Item Analysis

Endorsement of DSM-IV Screen items among Oregon gamblers ranged from a high of 14.3% (Preoccupation) to a low of 0.6% (Beyond the Legal). It is instructive to compare positive responses to specific items by problem gamblers and non-problem gamblers to see how well the different items discriminate between these groups. For this analysis, we have used the SOGS classification of non-problem and problem gamblers in order to prevent confusion between the method of classifying respondents and the items by which they were classified. Since all of the DSM-IV Screen items are framed in the past year, the current problem and probable pathological gamblers in Oregon were used in this analysis.

Table 12: Comparing Non-Problem and Problem Gamblers on the DSM-IV Items

* Significant

** Highly significant

Table 12 shows that all of the DSM-IV items discriminate effectively between SOGS-defined problem and non-problem gamblers in Oregon. The most effective discriminator among the DSM-IV items was Chasing with 69.6% of the current problem and probable pathological gamblers scoring a positive response in contrast to only 6.6% of the non-problem gamblers. The next best discriminator was Preoccupation, with 59.0% of the problem and probable pathological gamblers scoring a positive response compared to 12.6% of the non-problem gamblers. Table 12 also shows that there is a significant difference in the mean DSM-IV scores for non-problem and problem gamblers, supporting the notion that the DSM-IV Screen measures something similar to the SOGS.

Validity

There are several different types of validity that can be measured to assess the performance of an instrument. These include content, criterion, congruent and construct validity. Content validity is a subjective measure of how appropriate the items seem to a set of reviewers who have some knowledge of the subject matter. The DSM-IV Screen has already been found to have good content validity by a variety of appropriate audiences including self-identified pathological gamblers as well as treatment professionals and survey researchers (Fisher 1996; Lesieur & Rosenthal 1991).

Criterion Validity

Criterion validity requires that the instrument be judged against some other method that is acknowledged as a "gold standard" for assessing the same variable. In the case of the DSM-IV Screen, we must use the SOGS as the "gold standard" since this is the primary method that has been used to identify problem and pathological gamblers since the late 1980s (Volberg & Banks 1990). As a first step, we calculated the correlation coefficient between the DSM-IV Screen and the current South Oaks Gambling Screen. The result of this analysis was statistically significant at 71% (correlation coefficient = .706, p = .000).

To better understand how the SOGS and the DSM-IV Screen operate in relation to one another, it is useful to examine how respondents scored on each of these instruments in more detail. Overall, the prevalence of the less severe DSM-IV category (3 or 4 points) is 2.37% while the prevalence of the more severe DSM-IV category (5 or more points) is 1.53% among respondents in Oregon who gambled. These figures compare to 2.22% and 1.61 for the current SOGS scores among respondents who gambled. Table 13 shows the number of respondents who scored at different levels on the SOGS and the DSM-IV.

Table 13: Comparing Scores on the SOGS and the DSM-IV

Table 13 shows that the DSM-IV Screen operates quite well in relation to the SOGS. On the one hand, respondents who score low on the DSM-IV Screen also tend to score low on the SOGS. On the other hand, 70% of respondents who score high on the DSM-IV Screen (5 or more) score 3 or more points on the SOGS. In contrast to the Colorado and New York surveys, the SOGS also performs well in relation to the DSM-IV Screen. The majority of respondents who score as current probable pathological gamblers on the SOGS (86%) score 3 or more points on the DSM-IV Screen and 52% of these respondents score at the highest level on the DSM-IV Screen. This analysis shows that the DSM-IV Screen and the SOGS have a strong relationship to one another although it is still unclear whether the strictest DSM-IV criteria represent the best cutoff for identifying pathological gamblers in the general population.

Congruent Validity

Since several of the items on the SOGS and DSM-IV Screen are similar, it is possible to check whether respondents answered similar questions differently in different places in the interview. Table 14 on the following page shows how respondents who gambled answered several similar questions from the current SOGS and the DSM-IV Screen.

Table 14: Comparing Scores on Similar SOGS and DSM-IV Items

Table 14 shows that respondents are less likely to give a positive answer to the DSM-IV questions than to the current SOGS items assessing Tolerance. Respondents are more likely to give a positive answer to the DSM-IV questions than to the current SOGS items assessing Chasing and Loss of Control. In the New York survey, we speculated that some of these differences might be the result of an ordering effect. However, the same differences were noted in Colorado where the screens were also rotated and a more likely explanation for these differences may be that they are the result of the specific wording of the items.

Construct Validity

In assessing the performance of a new instrument, it is helpful to examine differences between classified groups with respect to behaviors that are associated with problem gambling but are not included in the measurement scale. In gambling surveys, we can examine differences between DSM-IV-defined non-problem and pathological gamblers in their mean DSM-IV Screen scores as well as other measures related to gambling difficulties, including weekly gambling, time spent gambling per session, largest amount lost in a single day, total expenditures on gambling, parental gambling problems and age when gambling started.

There are significant differences in the mean scores of problem and non-problem gamblers, as defined by the DSM-IV Screen. The mean score of non-problem gamblers on the DSM-IV Screen is 0.2 compared with 3.2 for problem gamblers and 6.7 for severe problem gamblers.

There are numerous other behaviors that provide support for the construct validity of the DSM-IV Screen. For example, problem and severe problem gamblers, as defined by the DSM-IV Screen, are significantly more likely than non-problem gamblers to gamble weekly or more often, to gamble for 3 or more hours at a time, to have lost $1,000 or more in a single day, to have felt nervous about their gambling, to believe that a parent had a gambling problem and to have desired help for a gambling problem. Problem and severe problem gamblers, as defined by the DSM-IV Screen, acknowledge starting to gamble at a significantly younger age than non-problem gamblers. Problem gamblers also estimate that they spend significantly more on gambling in a typical month than non-problem gamblers. Finally, problem and severe problem gamblers as defined by the DSM-IV Screen are significantly more likely than non-problem gamblers to identify video poker as their preferred type of gambling.

Comparing the SOGS and DSM-IV Problem Gamblers

The prevalence of problem and severe problem gambling, measured by the DSM-IV Screen, is nearly identical to the prevalence rates identified with the South Oaks Gambling Screen. While 2.0% of the total sample (N=1,502) scored 3 or 4 points on the DSM-IV Screen, 1.9% of the total sample scored 3 or 4 points on the current South Oaks Gambling Screen. While 1.3% of the total sample scored 5 or more points on the DSM-IV Screen, 1.4% of the total sample scored 5 or more points on the current South Oaks Gambling Screen.

Table 15 compares the demographic characteristics of problem and severe problem gamblers as defined by the DSM-IV Screen with problem and pathological gamblers as defined by the SOGS. Since both the SOGS and the DSM-IV groups are small, and since the majority of the DSM-IV group is part of the SOGS problem group as well, we made no effort to test the differences for statistical significance. Table 15 does show that problem gamblers, as defined by the DSM-IV, are more likely than problem gamblers as defined by the SOGS, to be male, under the age of 30, divorced or separated, working or unemployed and with annual household incomes between $25,000 and $50,000.

Table 15: Comparing Demographics of SOGS and DSM-IV Problem Gamblers

Summary

Comparison of the South Oaks Gambling Screen and the DSM-IV Screen in the Oregon survey shows that the two screens are highly consistent and appear to be measuring the same phenomenon. The DSM-IV Screen is slightly more strict than the South Oaks Gambling Screen in classifying individuals as problem or pathological gamblers. As in New York and Colorado, psychometric analysis of the results of the Oregon survey suggests that the cutoff point for the DSM-IV Screen (5+ = pathological) may be too severe. Separate identification of the group of individuals who score three or four points on the DSM-IV Screen, as recommended by Lesieur and Rosenthal (1991), would allow the screen to capture individuals whose pathology is well-developed but perhaps not yet extreme.

Use of the DSM-IV Screen in the Oregon survey provided a valuable opportunity to improve our understanding of the DSM-IV Screen in relation to the South Oaks Gambling Screen. In addition, use of this screen provides a basis for comparison in future surveys of gambling and problem gambling in Oregon if the DSM-IV Screen, or any other instrument based on the DSM-IV criteria, becomes the instrument of choice for identifying problem and pathological gamblers in the general population.

In the future, it will be important to compare the SOGS and the DSM-IV in problem gambling treatment programs where clinical assessments can be used to triangulate the results of these measurement tools and to determine the best cutoff points for classifying individuals as problem and pathological gamblers.

3 Baseline studies based on the South Oaks Gambling Screen have been carried out in 29 United States and Canadian jurisdictions, including Oregon, as well as in Australia, New Zealand and Spain. Replication surveys based on the South Oaks Gambling Screen have been carried out in nine jurisdictions.

4 The scoring method used with the Oregon sample is somewhat different from the scoring method used by Fisher (1996). In Fisher's approach, the first seven items were scored only if the response was "Often" while the last three items were scored for any positive response. The different scoring method was adopted because of the low response rate to the DSM-IV Screen items in these surveys compared to the sample of casino patrons used by Fisher.