IEA INTERNATIONAL RESEARCH CONFERENCE PRE-CONFERENCE WORKSHOPS
Analyzing IEA International Databases
29-30 June 2010
The IEA Data Processing and Research Center (IEA DPC) will be conducting a series of pre-IRC workshops. These workshops will be given in parallel on 29-30 June 2010, as part of the pre-conference activities. Each workshop starts at 09:00 and runs till 17:00 in four sessions, with two 15 minute coffee breaks and a lunch break (12:15-13:45).
Workshop #1: Introduction to IEA Databases and IDB Analyzer
Presenters: Plamen Mirazchiyski & Oliver Neuschmidt
This workshop will provide an overview of the IEA databases currently available, with particular emphasis on the most recently released databases (TIMSS 2007, PIRLS 2006, and SITES 2006). As part of the workshop, participants will also be instructed on how to use the IDB analyzer. The IDB Analyzer is an application developed by IEA DPC to facilitate access to and analysis of the large-scale assessment databases available from IEA. The IDB Analyzer creates SPSS code that can be used with SPSS to conduct statistical analyses, taking into account the complex sample structure of the databases.
The following topics will be covered during the workshop:
Overview of IEA databases
- Structure of the databases
- Accessing the data
- Reviewing the documentation
Sampling and test design: Implication for analysis
- General sample and test design
- Computing sampling and measurement variance
- Estimating statistical significance of results
Using the IDB Analyzer to combine datasets
- Merging data from students, schools, teachers, and parents
- Combining data from different countries
- Modifying and recoding data for analysis
Using the IDB Analyzer to conduct analysis
- Calculating means, standard deviations, and percentages
- Calculating correlations and regression coefficients
- Calculating percentages of students meeting achievement benchmarks
- Calculating differences between groups
This workshop is aimed at individuals who have only limited or superficial familiarity with IEA databases and analysis procedures, and who want to become familiar with them. Participants are expected to bring their own laptops with SPSS installed (SPSS is required to use the IDB Analyzer). Free copies of the IDB Analyzer will be distributed at the workshop.
This workshop is open to the first 30 applicants.
List of Participants
| Name | Country | Affiliation |
|---|---|---|
| Ewa Andersson | Sweden | Umeå Universitet |
| Luisa Araujo | Italy | EC Joint Research Centre |
| Daniel Arnesson | Sweden | Umeå Universitet |
| Altangerel Ayush | Mongolia | University of Science and Technology |
| Carolyn Barber | United States | University of Missouri Kansas City |
| Ã…sa Berndtsson | Sweden | IPD |
| Maria Canadas | Spain | Universidad de Granada |
| Jaguthsing Dindyal | Singapore | National Institute of Education Singapore |
| Sabrina Greco | Italy | INVALSI |
| Ã…se Hansson | Sweden | University of Gothenburg |
| Tae Jun Kim | Korea | Korean Educational Development Institute |
| Nader Leis | Iran | Tarbiat Modares University |
| Bo Nielsen | Sweden | IPD |
| Trude Nilsen | Norway | University of Oslo |
| Ida Friestad Pedersen | Norway | University of Oslo |
| Natalia Sheveleva | Russia | Moscow City Pedagogical University |
| Anna Toropova | Sweden | Stockholm University |
| Batmunkh Tsend-ayush | Mongolia | Ministry of Education |
Workshop #2: Using HLM with Large-Scale Assessment Data
Presenters: Daniel Caro & Leslie Rutkowski
The workshop will train participants on how to use the HLM software through analysis of socioeconomic gradients in PIRLS 2006. First, the rationale for HLM analysis will be introduced and participants will learn how to prepare datasets and import them to HLM. Second, relevant hypotheses for policy research regarding socioeconomic gradients will be evaluated, stressing theory, model specification, and interpretation of results.
An example of the type of analysis that will be presented is testing for the presence of a socioeconomic gradient. This hypothesis test evaluates whether there is a significant relationship between family socioeconomic status (SES) and academic performance within a two-level model (students in schools in a specific country). The theoretical background will be presented, various model specifications will be considered (linear and curvilinear effects), key statistics will be interpreted (i.e., slope, R-squared, curvilinear SES term), and the policy implications of each hypothesis test and resultant statistics will be discussed.
Next, increasingly complex hypotheses will be evaluated. For example, participants will test whether socioeconomic gradients vary between schools, whether the school SES has an effect above and beyond the family SES effect, whether specific variables mediate and/or moderate family SES effects and, finally, within a three-level model framework, whether socioeconomic gradients vary between countries, and if so, how.
The following topics will be covered:
- Theoretical background of multilevel models
- Data preparation and importing data to HLM
- The socioeconomic gradients framework: Theory and hypotheses set out
- Hypothesis testing: Model specification and interpretation of results
- Two- and three-level HLM analysis
Participants will learn to specify, estimate, and interpret results of two- and three-level models within the HLM software environment, as well as to formulate and test hypotheses with implications for research and policy. This workshop is aimed at individuals with a working knowledge of IEA databases and a solid knowledge of intermediate statistics. Participants will be required to bring their own laptops with SPSS installed. The HLM free-trial version and PIRLS 2006 data will be made available and used during the workshop. The workshop will comprise lectures mixed with hands-on training.
25 applicants will be accepted.
List of Participants
| Name | Country | Affiliation |
|---|---|---|
| Francesco Agrusti | Italy | Roma Tre University |
| Anders Auer | Sweden | SKOLVERKET |
| Sigrid Blömeke | Germany | Humboldt University of Berlin |
| Kok Leong Boey | Singapore | National Institute of Education |
| Ireta Cekse | Latvia | University of Latvia |
| Noor Azina Ismail | Malaysia | University of Malaya |
| Ana Kozina | Slovenia | Educational Research Institute |
| Moo Sung Lee | Hong Kong SAR | Hong Kong Institute of Education |
| Petra Lietz | Australia | ACER |
| Ingrid Munck | Sweden | University of Gothenburg |
| Monica Rosén | Sweden | University of Gothenburg |
| Mojca Rozman | Slovenia | Educational Research Institute |
| Surette van Staden | South Africa | University of Pretoria |
| Eva Wiren | Italy | EC Joint Research Centre |
| Kajsa Yang Hansen | Sweden | University of Gothenburg |
| Suleiman Khoja | Libya | Curricula & Educational Research Center |
Workshop #3: Assessment Designs, Item Response Theory, and Proficiency Estimates
Presenter: Eugenio Gonzalez
This workshop will provide an overview of the principles surrounding the design of large-scale assessments, the item response theory models used to calibrate items, and the methodology used to assign proficiency estimates, also known as plausible values. Presentations will be mostly theoretical, with ample time given for discussion.
The following topics will be covered during the workshop:
Overview of assessment design principles
- Advantages and disadvantages of current designs of large-scale assessments
- Consequences for analysis
Overview of principles of item response theory
- Advantages and disadvantages of different models
- Limitations of IRT models
Overview of principles of population modeling and proficiency estimation
- Review of procedures and techniques for carrying out conditioning
- Multiple imputations in large-scale assessments
- Plausible values and why they are useful
The general daily schedule will consist of presentations and limited hands-on practical assignments. Participants are expected to bring their own laptop PC with statistical software installed.
The workshop will be open to 25 participants.
List of Participants
| Name | Country | Affiliation |
|---|---|---|
| Fabio Alivernini | Italy | INVALSI |
| Marjeta Doupona Horvat | Slovenia | Educational Research Institute |
| Jan-Eric Gustafsson | Sweden | University of Gothenburg |
| Tsung-Hau Jen | Chinese Taipei | National Taiwan Normal University |
| Stefan Johansson | Sweden | University of Gothenburg |
| Hege Kaarstein | Norway | University of Oslo |
| Masoud Kabiri | Iran | Research Institute for Education |
| Sara Manganelli | Italy | INVALSI |
| Torgeir Onstad | Norway | University of Oslo |
| Laura Palmerio | Italy | INVALSI |
| Artur Pokropek | Poland | Polish Academy of Science |
| Rolf Strietholt | Germany | Dortmund University of Technology |
| Christian Tallberg | Sweden | SKOLVERKET |
Download the Workshop brochure.