JRSA Hosts Track of Sessions at National Forum
The Justice Research and Statistics Association partnered with the National Criminal Justice Association and IJIS Institute to put together sessions and workshops as part of the 2013 National Forum on Criminal Justice held in Chicago on August 4-6. More than 350 people attended the Forum, including directors and staff from more than 30 Statistical Analysis Centers (SACs). The theme of this year's meeting was "Integrating Research, Policy, and Technology to Improve Public Safety."
The meeting officially opened on Sunday evening with remarks from leaders of the three partner organizations as well as Denise O'Donnell, Director of the Bur-eau of Justice Assistance, which provided support for the Forum. The remainder of the session highlighted Chicago's Cure Violence program (formerly Ceasefire), which applies a public health approach to stopping shooting and killings in neigh-borhoods with high levels of violence. A clip from the PBS documentary The Interrupters showed how the program's "peacemakers" work with the community, and a panel of people involved with the initiative discussed the program model from a variety of perspectives.
Monday morning kicked off with "Inside the Beltway." Cabell Cropper, Executive Director of NCJA and Elizabeth Pyke, NCJA Director of Government Affairs, offered a look at activities, appropriations, and upcoming legislation in Washington that affect criminal justice policy makers and practitioners. The opening plenary followed, with Michael Nila, Founder and President of Shibumi, speaking on the nobility of public service, with a focus on those who serve in the justice system. The morning was rounded off by "Voices from the Field" - regional caucuses in which participants were encouraged to meet with others from their area of the country to discuss issues, priorities, and challenges of concern. The Luncheon Keynote featured Chicago Police Superintendent Garry McCarthy, who spoke about his department's comprehensive policing strategy.
Monday Workshop Sessions
Four tracks of workshop sessions began Monday afternoon and ran though Tuesday afternoon. JRSA sponsored a practitioner track, with four workshops held over the course of the two days.
Three SAC director - Terry Salo of New York, Max Schlueter of Vermont, and Paul Stageberg of Iow - discussed some of the challenges and opportunities in using administrative records for research and decision making rather than for operational purposes. Devon Adams from the Bureau of Justice Statistics served as moderator.
Dr. Schlueter began by defining administrative records as "the data routinely gathered for operational or business purposes by public or private agencies." He provided some examples of administrative records being used by the three states (IA, NY and VT), such as police computer-aided dispatch (CAD) and record management system (RMS) data, Uniform Crime Reporting/National Incident-Based Recording System data, criminal arrest and disposition data, juvenile justice court filings, prisoner custody records, correctional treatment data, and education records.
Dr. Stageberg noted that in Iowa they were interested in the "school to court pipeline." The SAC started working with the schools to get information on disciplinary actions with the goal of helping to develop consistencies in disciplinary policies across schools and disciplines. He also discussed the importance of technical credibility in obtaining administrative records. This includes: using an Institutional Review Board; identifying groups that you've worked with that have also worked with the group you are now trying to work with; and emphasizing the importance of the research by ensuring that contributors know how the research will benefit them. To maintain credibility once established, it is important to institute data sharing agreements, spend time in face-to-face discussions, and stay in regular communication.
Ms. Salo discussed issues related to the use of legal agreements, including when to use legal agreements and what to include in such agreements. She noted that there are two kinds of legal agreements: non-disclosure agreements, for one-time research projects and one-way data sharing; and Memoranda of Understanding, which are long-term arrangements for sharing between state agencies.
Finally, the panelists discussed challenges in matching criminal justice records, how to handle inconsistent data, and issues related to privacy and security.
The Bureau of Justice Statistics (BJS) has been working on the redesign of the National Crime Victimization Survey (NCVS) for the past several years. A major drawback of NCVS to date has been the inability to disaggregate the data to the state level. Bill Sabol, Acting Director of BJS, gave an overview of the redesign and their efforts to provide subnational estimates based on samples drawn from the largest states. Two SAC staff members, Phillip Stevenson, Director of the Arizona SAC, and Marjorie Stanek, Research Coordinator at the Kentucky SAC, discussed ways in which subnational estimates and complementary efforts can be useful to the states for research and planning, as well as an example of the data that a state- level victimization survey can provide. Janeena Wing, director of the Idaho SAC, served as moderator.
Dr. Sabol began the session by presenting information on the NCVS Subnational Estimation Program. After presenting background on the NCVS, he noted that the survey is limited because it is designed to be representative at the national level, not necessarily for states, cities, or other smaller geographic areas. It would be valuable to have information at the state and local levels to allocate resources and evaluate local initiatives.
There are two components to the estimation program: direct estimation, using collected survey data; and indirect model-based estimation, which involves producing estimates using statistical methods and data from other sources. There is a range of ways to obtain both direct and indirect estimates from NCVS. BJS is looking at making direct estimates for the larger states, or using the current data to make estimates for generic areas or MSAs. They are also looking at boosting the sample size in select states - they are currently conducting a pilot test in 11 states to produce three-year rolling averages. This feasibility study began in July and will continue for two years. They are also looking at indirect model-based estimates by correlating current data with Uniform Crime Reporting rates and household information. They have state level estimates for 1997 - 2011.
Dr. Stevenson discussed using data to improve services for crime victims. He noted that the Arizona SAC partnered with Arizona State University to conduct a random digit dial survey that produced victimization estimates for each of the two largest counties in the state, and a third for the rest of the state. Some of the key findings of the Arizona survey included: half of victims report their victimization to the police; few victims receive victim services; there are differences by county in the percentage of victims that receive services; and Hispanics are less likely to be victims of identity theft, but more likely to report when they are. Dr. Stevenson noted that there are many positives to conducting state victimization surveys, including obtaining sub-state measures of crime, identifying the proportion of victims who receive services and are aware of victims' compensation programs, and measuring satisfaction with local criminal justice systems.
Ms. Stanek began her presentation by noting that Kentucky has some unique issues: It is small, it has lots of counties, and it is largely rural and white. Because of this there is a sense in the state that national issues and results do not adequately reflect what is going on in the state. As a result, since 1985 they've commissioned several statewide victimization surveys, the most recent of which was in 2008. They are now planning the 2014 survey. Because of past issues with incorrect information on potential respondents, they have decided to use voter registration records in an effort to obtain more accurate information. The survey will be a mail survey and the sample size will be limited due to limited resources. Marjorie noted that the results will be limited due to the sample size and surveying of registered voters only.
Tuesday Table Sessions
"Breakfast with the Experts" kicked off Tuesday's meeting activities. Participants could choose from table sessions in 12 topic areas and participate in a discussion led by an expert facilitator. JRSA coordinated topics for three of the tables: 1) BJS' Interest in Administrative Records and Recidivism Research, with Howard Snyder; 2) Criminal Justice Cost-Benefit Analysis with Mike Wilson and John Roman; and 3) NCJA-JRSA Opportunities to Enhance State Research and Data Analysis, with Phil Stevenson and Bob Boehmer.
Tuesday Workshop Sessions
Concerns about recidivism drive many decisions about sentencing, corrections, and diversionary programs. Sue Burton, Director, of the Florida Statistical Analysis Center, moderated this session that featured Howard Snyder, Deputy Director of the Bureau of Justice Statistics, and Mark Powers, Research Analyst at the Illinois Criminal Justice Information Authority (ICJIA). Bonnie Locke, Nlets Director, also participated in the panel. Dr. Snyder discussed BJS' work with Nlets and NORC to develop a system to use standardized state criminal history records to study recidivism, the first national study of recidivism since 1994. He described the project and explained that this will be the first time automated data from the states will be made available to them in a way that allows their use in critical decision making. Ms. Locke described Nlets, the Inter-national Justice and Public Safety Network, as the interstate network for the exchange of law enforcement-, criminal justice-, and public safety-related information. For more than 45 years Nlets has provided its users with Criminal History Records Information data from state and federal repositories via a standardized, secure and efficient methodology. This capability provides a way for the states and the FBI to respond automatically to requests from other users over the Nlets network. Mr. Powers discussed the new Illinois data tool developed to enable exploratory analyses of Illinois adult offender populations' criminal history and recidivism. The tool tracks criminal justice events such as arrests, convictions, probation sentences, and IDOC admissions and exits. Additional updates to the tool are planned for this fall.
Many states are using part of their Byrne Justice Assistance Grant (Byrne JAG) money to fund MJTFs, which are cooperative law enforcement efforts involving two or more agencies from different jurisdictions working together to address the problem of drug crime. This session provided information on efforts by the Georgia and Illinois SACs to assess the performance of their MJTFs, and the outcomes of those efforts. Stan Orchowsky, JRSA Research Director, served as moderator.
Stefanie Lopez-Howard, Georgia SAC Director, spoke about working with the consulting group Applied Research Services on an impact evaluation of Georgia's MJTF. The study surveyed all persons signing MOUs to participate in the task force, did a document analysis related to boundaries and funding over time, and looked at data such as criminal histories, in the process of analyzing collected data, and Ms. Lopez-Howard mentioned some challenges regarding data quality and defining effectiveness.
Mark Myrent, Illinois SAC Director, said that the Illinois MJTF covered 62 of the state's 102 counties. He discussed the logic model template used to collect data and talked about the process of customizing the template to local conditions. He also pinpointed some difficulties in evaluating the effectiveness of the MJTF; it can take substantial time, for example, to complete a high-quality arrest compared to a typical drug arrest, and it can also be hard to attribute reduction in drug availability and crime specifically to MJTFs. Some analyses planned for the future relate to deployments over time, individual versus task force officers, and percentage of investigations dropped versus successful prosecution.
The emerging field of "implementation science" has much to teach us about how to successfully take evidence-based practices to scale. This session highlighted the work of Colorado's Evidence-Based Practices Implementation for Capacity (EPIC) initiative. Two researchers from the Colorado Department of Public Safety - Kim English, Director of Research, and Diane Pasini-Hill, Manager of the EPIC Project - presented. Roger Przybylski, Founder and Consultant, RKC Group, was the discussant, and Stephen Haas, Director of the West Virginia Office of Research and Strategic Planning, served as moderator.
The EPIC initiative began with the State Criminal Justice Commission's interest in evidence-based practices (EBPs) and its desire to coordinate training across agencies. The effort focuses on skill-building and is grounded in research showing that training alone is not sufficient for retention and implementation of information - "hands-on" practice with one-on-one coaching is the most effective method. The research also shows that there are "implementation drivers" that determine whether an initiative will be successfully implemented.
The EPIC initiative is implementing an EBP, Motivational Interviewing, by employing broad-based coalitions of different Colorado agencies. They created implementation teams around the state to promote and plan the implementation of the EBP and maintain fidelity to the EBP.
Research shows that obtaining administrative support is an important implementation driver. The presenters noted that this has been an ongoing challenge for the initiative. Mr. Przybylski noted that there has been a general lack of attention to implementation in all of the recent discussion on EBPs. He said that high quality implementation is very difficult to do, but the EPIC initiative demonstrates that the implementation principles identified in the research are not abstract but actually apply to real world settings, particularly in criminal justice. The EPIC initiative also demonstrates the importance of being data-driven and measuring quantitatively how implementation is being done. Finally, he noted that as a field we need to get better at measuring fidelity - how do we measure whether we have implemented something "fully?"
Closing Plenary Sessions
Building on the priority issues identified in the Regional Caucuses on Monday, Forum participants met with the leadership of NCJA, JRSA, IJIS and BJA to further define the issues and develop next steps for action on these priorities in the coming year.
JRSA Postconference Seminars
Cost-Benefit Analysis provided a step-by-step description of the methodology used to create a cost-benefit model for programs designed to reduce crime. Michael Wilson, former Oregon SAC Director who has worked extensively on cost-benefit models and taught numerous seminars and webinars on the topic, served as instructor. He discussed how to estimate tax payer and victimization costs of crime, and also how to use the effect size of a program to determine the estimated number of crimes avoided and the estimated benefit of avoiding them. The seminar provided analysts with resources and practical advice to enable them to develop a cost-benefit model in their state.
Introduction to Propensity Score Matching and Interrupted Time Series Analysis Using SPSS focused on the practical application of these two techniques using SPSS software. Criminal justice research and evaluation are often challenged by the practical inability to use experimental designs with random assignment. Practical and ethical concerns arise when thinking about randomly assigning criminal justice-involved individuals to different treatments. Similarly, when evaluating the effectiveness of new programs or policies, it is extremely difficult to fully control real-world conditions surrounding the program's implementation. Instructor Dr. Gipsy Escobar provided an introduction to Propensity Score Matching and Interrupted Time Series Analysis, two powerful quasi-experimental tools designed to overcome these obstacles. Dr. Escobar, a criminologist and faculty member at Loyola University Chicago, specializes in quantitative methods and has conducted evaluation research in different areas of criminal justice.