Press Release - February 17, 2004
Press coverage - This study received national and international attention from the mass media, including coverage on MSNBC Today Show, CBS News, hundreds of newspapers, and some video game related media. Listen to Kevin Haninger talk about video game ratings on NPR's Radio Times (August 12, 2005), and read articles by Katie Hafner in The New York Times (Circuits, December 16, 2004) and Peter Slevin in The Washington Post (December 16, 2004) on the hot topic of video games

Correction to printed article: The first entry in Table 4 lists an abbreviation for Sega Dreamcast in the Console column as "Dc" instead of "DC."

Answers to frequently asked questions -

What are the study’s main findings?
Why did the results sound worse in some of the popular media?
What are the study’s main recommendations?
How did you select the video games? How many video games did you play and review?
What information did you collect about each video game?
How did you define and measure each type of content?
What about any content that you might have missed by only playing the games for 1 hour?
Background on video game ratings

What are the study’s main findings?

• The Entertainment Software Rating Board (ESRB) content descriptors provide a good indication of game content, but the absence of a content descriptor does not mean the absence of content.

• In the random sample of 81 games we played, we observed 79 games (98%) that contained violence for an average of 36% of game play, with 77 games (95%) receiving content descriptors for violence. We found 73 games (90%) that rewarded or required the player to injure characters and 56 games (69%) that rewarded or required the player to kill.

• We observed blood in 34 games (42%), with 22 games (27%) receiving content descriptors for blood.

• We observed sexual themes in 22 games (27%), with 16 games (20%) receiving content descriptors for sexual themes. Video games were significantly more likely to depict females partially nude or engaged in sexual behaviors than males.

• We observed the use of profanity in 22 games (27%) for an average of 4.3 uses per hour, with 14 games (17%) receiving content descriptors for profanity.

• We observed the depiction or use of substances in 12 games (15%) for an average of 3.4% of game play, with only one game (1%) receiving a content descriptor for substances.

• We observed the player character gambling for money in one game (1%), representing 1% of game play, with no games receiving a content descriptor for gambling.

• Overall, we observed content that could warrant an ESRB content descriptor in 39 games (48%) in which the ESRB had not assigned a content descriptor. The ESRB assigned seven content descriptors for seven video games (9%) in which we did not observe the content indicated within one hour of game play. These results suggest that T-rated (for "Teen") video games may be source of exposure to a wide range of unexpected content.
  Back to questions

Why did the results sound worse in some of the popular media?

Unfortunately, some of the press coverage included the following sentence: "Among the sample the researchers viewed, 20 percent of games with sexual content including partial nudity listed that content on the label; 17 percent of games with profanity listed it on the label, and just 1 percent of games with depictions of tobacco or alcohol listed that on the label." This statement should have read: "Among the sample the researchers viewed, 27 percent of games depicted sexual content including partial nudity; 27 percent of the games included the use of profanity; and 15 percent of the games included the depiction of alcohol or tobacco. In contrast, only 20 percent of the games in the sample listed a content descriptor for sexual content, 17 percent listed profanity, and just 1 percent of games listed depictions of tobacco or alcohol on the label." We did not make the conditional statement in the form of what was reported, in part because of the complication that 3 of the games that received content descriptors for sexual themes (The King of Fighters '99, The Simpsons Wrestling, and Final Fantasy VIII) and 3 of the games that received content descriptors for profanity (C: The Contra Adventure, Tiny Tank: Up Your Arsenal, and Lunar 2: Eternal Blue) were ones in which we did not observe such content. For this reason, making a comparison is complicated. However, the numbers imply that 13 of the 22 games (59 percent) in which we observed sexual themes and 11 of the 22 games (50 percent) in which we observed profanity did receive an ESRB content descriptors for this content. Similarly, 1 of the 12 games (8 percent) containing substances received an ESRB content descriptor for this content. The Associated Press issued a correction.
Back to questions

What are the study’s main recommendations?

• Our findings suggest the need for greater clarity and transparency in the use of ESRB content descriptors and in the overall rating process. We believe that ESRB content descriptors should provide complete and accurate information about game content so that parents can make informed decisions. We further believe that the ESRB should play the video games as part of its rating process to help ensure the absence of content other than that indicated by the materials submitted to the ESRB by the game manufacturers.

• We encourage physicians, particularly pediatricians and specialists in adolescent medicine, to ask patients and their parents about their experiences with video games. Finally, we emphasize that parents need to judge for themselves the appropriateness of the game content, both by using the ESRB rating information and by being a part of their child's experience.
  Back to questions

How did you select the video games? How many video games did you play and review?

We developed a database of all 396 T-rated video game titles released on the major video game consoles in the United States by April 1, 2001. The database contained each game title's genre, console, release year, and ESRB-assigned content descriptors. The game titles can be viewed alphabetically or by genre. After verifying the database contents, we stratified the 396 video game titles by genre and randomly selected 20% to play. One selected game title contained two separate games, so overall we played 81 T-rated video games in the random sample. For consistency, an undergraduate student (Seamus Ryan) with considerable video gaming experience played all of the video games. The player first familiarized himself with each video game, then restarted the video game and recorded at least one hour of game play on videocassette for coding.
Back to questions

What information did you collect about each video game?

One author (Kevin Haninger) reviewed and coded all of the recorded game play. We collected information about the following types of content in each game: violence, blood, sexual themes, profanity, substances, and gambling. The JAMA article contains tables that list each video game we played, as well as the genre, console, release year, ESRB-assigned content descriptors, and our observations of game content.
Back to questions

How did you define and measure each type of content?

We established consistent definitions and measures to apply through the study.

• Violence
- We defined violence as intentional acts in which the aggressor causes or attempts to cause physical injury or death to another character. We did not include actions that led to unintentional physical harm, the effects of natural disasters, or the presence of dangerous obstacles not attributed to another character. We also did not include intentional acts of physical force that represent normal play in a sports game (e.g., tackling in football), but we counted all punches and kicks in boxing and wrestling games are violence because the intention in these sports is to cause injury. We noted whether each game contained violence and whether the game rewarded or required the player to injure or kill characters. To quantify the amount of violence in each game, we coded whether each second of recorded game play contained acts of violence and then calculated the percentage of violent game play.

• Blood
- We defined blood as a red fluid originating from an injured human or any colored fluid from an injured creature. We noted whether each game contained depictions of blood and the color of the blood.

• Sexual Themes
- We defined sexual themes as behaviors or dialog related to sex, as well as depictions of exposed breasts, buttocks, or genitals. Given the ESRB definition of Suggestive Themes ("Mild provocative references or materials"), we did not otherwise count pronounced cleavage, large breasts, or provocative clothing as sexual themes, although we separately noted the presence of this content. We noted whether each game contained sexual themes, the type of sexual themes observed, and the gender of characters involved.

• Profanity
- We defined profanity as the use of abusive and vulgar language, anatomical references without the use of such words, and obscene gestures involving the middle finger or its full-arm equivalent. We noted whether each game contained profanity. To quantify the amount of profanity in each game, we counted each occurrence of profanity, noted whether it occurred as dialog, written text, song lyrics, or gestures, and then calculated the number of uses of profanity per hour of game play.

• Gambling
- We defined gambling as scenes where characters bet money for prizes. We noted whether each game contained gambling. To quantify the amount of gambling in each game, we coded whether each second of recorded game play contained gambling and then calculated the percentage of game play that depicted gambling.

• Substances
- We defined substances as scenes in which characters use or discuss use of alcohol, tobacco, or other drugs, or when the player views images of at least one substance. We did not count medicinal herbs, tonics, or ambiguous brews as substances. We noted whether each game contained substances and whether the player or other characters bought or used them. To quantify the amount of substances in each game, we coded whether each second of recorded game play contained substances, noted the type of substance depicted, and then calculated the percentage of game play that depicted substances.
Back to questions

What about any content that you might have missed by only playing the games for 1 hour?

We emphasized in the paper that our method may miss some game content since we do not have access to the information submitted to the ESRB from game manufacturers and since we only played each game for one hour. We recognize that the choice of how long to play the game clearly affects the amount of content observed, and we model this relationship in this Figure. To construct the Figure, we first noted the time (i.e., the seconds from the beginning of the game) when we initially observed each type of content. Next, for each game we identified the latest time where we first observed a new type of content, since the latest time corresponds to the length of time required to observe all new types of content in the game. We know from experience that content in some games occurs after 1 hour of game play, and our method misses that content. For example, as we noted in the paper, the first episode of violence in Overblood occurs well into the game. Consequently, for those 7 games where the ESRB assigned content descriptors and we did not observe such content within our hour of game play, we assigned times of greater than 70 minutes. Finally, we characterized the cumulative percentage of games for which we observed all content indicated by the content descriptors as a function of game duration. As described in the paper, this analysis suggests that by playing the games for approximately 1 hour we probably observed approximately 90% of the content indicated by the content descriptors and that playing the games for only 10 minutes would lead to missing 1 or more types of game content over 40% of the time.
Back to questions

The Entertainment Software Rating Board (ESRB) is a self-regulatory body created in 1994 by the computer and video game industry. The ESRB is responsible for applying and enforcing the rating and advertising standards adopted by the industry. The ESRB video game ratings include age-based rating symbols and content descriptors. Age-based rating symbols include EC (for "Early Childhood"), E (for "Everyone"), E10 (for "Everyone 10+"),T (for "Teen"), M (for "Mature 17+"), and AO (for "Adults Only 18+") that game manufacturers display on the front of the game box. Content descriptors are short phrases that indicate game content (e.g., "Violence" or "Mild Language") that game manufacturers display on the back of the game box. To receive a rating, game manufacturers provide the ESRB with videotaped game footage and other information about game content. Three trained ESRB raters independently review the materials submitted by manufacturers and assign the rating and content descriptors they believe are appropriate, but they do not play the final games that consumers ultimately purchase as part of the process of assigning the rating.
Back to questions

http://www.kidsrisk.harvard.edu/faqs4.html