An analysis of student learning with the LISP tutor indicates that while is complex, it simple. The key to factoring out complexity monitor 500 productions in which describe programming skill. these follows power‐law curve typical skill acquisition. There transfer from other experience extent this involves same productions. Subjects appear differ only on general dimensions how well they acquire and retain Instructional manipulations such as remediation, content feedback, timing feedback are...

Web surveys can be programmed to capture a variety of paradata regarding how respondents answer questions. These provide great opportunities for researchers assess response quality, specifically whether engage in satisficing – not spending enough effort accurate responses. In particular, speeding (i.e., giving answers very quickly) has increasingly been used as an indicator and low quality. However, few studies have examined actually leads compromised To address this gap the literature,...

Demonstrations that analyses of social media content can align with measurement from sample surveys have raised the question whether survey research be supplemented or even replaced less costly and burdensome data mining already-existing "found" content. But just how trustworthy such be—say, to replace official statistics—is unknown. Survey researchers scientists approach key questions starting assumptions analytic traditions differ on, for example, need representative samples drawn frames...

Standardized survey interviewing is widely advocated in order to reduce interviewer-related error, for example by Fowler and Mangione.But Suchman Jordan argue that standardized wording may decrease response accuracy because it prevents the conversational flexibility respondents need understand questions as designers intended.We propose arguments these competing positions-standardized versus flexible approaches-may be correct under different circumstances.In particular, both should produce...

The use of visual analog scales (VAS) in survey research has been relatively rare, part because operational difficulties. However web surveys permit the continuous input devices such as slider bars, making VAS more feasible. authors conducted an experiment to explore utility a survey, comparing it radio button and numeric entry text box on series bipolar questions eliciting views genetic versus environmental causes various behaviors. included variety additional comparisons including presence...

Survey researchers since Cannell have worried that respondents may take various shortcuts to reduce the effort needed complete a survey. The evidence for such is often indirect. For instance, preferences earlier versus later response options been interpreted as do not read beyond first few options. This really only hypothesis, however, supported by direct regarding allocation of respondent attention. In current study, we used new method more directly observe what and look at recording their...

We carried out two experiments to investigate how the shading of options in a response scale affected answers survey questions. The were embedded web surveys, and they varied whether ends represented by shades same or different hues. also numerical labels for points examined responses both unipolar scales (assessing frequency) bipolar favorability). predicted that use hues would affect respondents viewed low end scale, making seem more extreme than when hue. This hypothesis was based on...

Grid or matrix questions are associated with a number of problems in Web surveys. In this paper, we present results from two experiments testing the design grid to reduce breakoffs, missing data, and satisficing. The first examines dynamic elements help guide respondent through grid, on splitting larger into component pieces. second manipulates visual complexity simplifying grid. We find that using feedback respondents multi-question helps data. Splitting grids further reduces data motivated...

Every cognitive interview pretest designer must decide how many interviews need to be conducted. With little theory or empirical research guide the choice of sample size, practitioners generally rely on examples other studies and their own experience preferences. We investigated size both theoretically empirically. Using a model relationship question problem prevalence, detection power technique, probability observing problem, we computed necessary, under varying conditions, detect problems....

The rise of social media websites (e.g., Facebook) and online services such as Google AdWords Amazon Mechanical Turk (MTurk) offers new opportunities for researchers to recruit study participants. Although have started use these emerging methods, little is known about how they perform in terms cost efficiency and, more importantly, the types people that ultimately recruit. Here, we report findings performance four sources recruiting iPhone users participate a web survey. reveal very...

Survey participants are increasingly responding to Web surveys on their smartphones as opposed personal computers (PCs), and this change brings with it some potential data-quality issues. This study reports a randomized crossover experiment compare the effect of two different devices, PCs, response quality in survey conducted probability-based panel. Participants (n = 1,390) were invited complete an online questionnaire both smartphone (mobile Web) PC (PC sequence. We hypothesized that use...

Interviewer-administered surveys are an important method of collecting population-level epidemiological data, but suffer from declining response rates and increasing costs. Web offer more rapid data collection lower There concerns, however, about quality web surveys. Previous research has largely focused on selection biases, few have explored measurement differences. This paper aims to assess the extent which mode affects responses given by same respondents at two points in time, providing...

As people increasingly communicate via asynchronous non-spoken modes on mobile devices, particularly text messaging (e.g., SMS), longstanding assumptions and practices of social measurement telephone survey interviewing are being challenged. In the study reported here, 634 who had agreed to participate in an interview their iPhone were randomly assigned answer 32 questions from US surveys or speech, administered either by a human interviewer automated system. 10 interviewers University...

This study contrasts two interviewing techniques that reflect different tacit assumptions about communication. In one, strictly standardized interviewing, interviewers leave the interpretation of questions up to respondents. other, conversational say whatever it takes make sure are interpreted uniformly and as intended. Respondents from a national sample were interviewed twice. Each time they asked same factual ongoing government surveys, five housing recent purchases. The first interview...

Several alternative response formats are available to the web survey designer, but choice of format is often made with little consideration measurement error. The authors experimentally explore three common used in surveys: a series radio buttons, drop box none options initially displayed until respondent clicks on box, and scrollable some visible, requiring scroll see remainder options. reversed order for half sample. find evidence effects stronger that visible endorsed more frequently,...

Journal Article Visual Context Effects in Web Surveys Get access Mick P. Couper, Couper Address correspondence to Couper; (e-mail: mcouper@umich.edu); Frederick G. Conrad; fconrad@umich.edu); or Roger Tourangeau RTourang@survey.umd.edu) Search for other works by this author on: Oxford Academic Google Scholar Conrad, Conrad Public Opinion Quarterly, Volume 71, Issue 4, Winter 2007, Pages 623–634, https://doi.org/10.1093/poq/nfm044 Published: 01 January 2007

Memories of war, terrorism, and natural disaster play a critical role in the construction group identity persistence conflict. Here, we argue that personal memory knowledge collective past become entwined only when public events have direct, forceful, prolonged impact on population. Support for this position comes from cross-national study which participants thought aloud as they dated mundane autobiographical events. We found Bosnians often mentioned their civil war Izmit Turks made...

article Free Access Share on A heuristic evaluation of a World Wide Web prototype Authors: Michael D. Levi Bureau Labor Statistics StatisticsView Profile , Frederick G. Conrad Authors Info & Claims InteractionsVolume 3Issue 4July/Aug. 1996pp 50–61https://doi.org/10.1145/234813.234819Published:01 July 1996Publication History 44citation2,872DownloadsMetricsTotal Citations44Total Downloads2,872Last 12 Months111Last 6 weeks16 Get Citation AlertsNew Alert added!This alert has been successfully...

Abstract Leaving the interpretation of words up to participants in standardized survey interviews, aptitude tests, and experiment instructions can lead unintended interpretation; more collaborative interviewing methods promote uniform understanding. In two laboratory studies (a factorial a naturalistic investigation), respondents interpreted ordinary concepts like ‘household furniture’ ‘living house’ quite differently than intended strictly when was left entirely them. Comprehension accurate...

Journal Article Sources of Error in Cognitive Interviews Get access Frederick G. Conrad, Conrad Address correspondence to Conrad; e-mail: fconrad@isr.umich.edu. Search for other works by this author on: Oxford Academic Google Scholar Johnny Blair Public Opinion Quarterly, Volume 73, Issue 1, Spring 2009, Pages 32–55, https://doi.org/10.1093/poq/nfp013 Published: 02 April 2009