What do researchers use to reduce bias and error in measurement of data?

Answer 1

RESEARCH BIAS

bias = unknown or unacknowledged error created during the design, measurement, sampling, procedure, or choice of problem studied

• bias is so pervasive because we want to confirm our beliefs
• science is organized around proving itself wrong not right
• key difference between qualitative and quantitative research is
• • attempts to eliminate bias by quantitative researcher
  • explicit acknowledgement of bias by qualitative researchers

(1) design bias

• research design bias is introduced NOT when the study fails to control for threats to internal and external validity BUT RATHER
• • when the study fails to identify the validity problems OR
  • when publicity about the research fails to incorporate the researchers cautions
  • e.g. Judith Wallerstein's longitudinal study of divorced children is based on a very small sample of white, upper middle class, California families and no control group BUT it has confirmed the basic belief that divorce is bad for kids, so it is difficult to argue that divorce can be either harmless or useful.
  • • confirms the bias that children of divorced parents are damaged goods
    • evidence to the contrary does not fit the public bias against divorce and gets little exposure
• 2nd eg: selecting the most or least of anything creates a regression effect
• • selecting the poorest peple to study the effects of an anti-poverty program
  • selecting the lowest functioning mentally ill people to study the effects of a therapy program
  • selecting chronic homeless to study the effects of a housing program
  • Molidar selected incarcerated women in his qualitative study of female gang members
  • SUMMARY: unless your report addresses the problem of regression, it will be a biased report (Molidar does not address this)
• 3rd eg: study dropouts (attrition effect)
• • people who drop out may be the ones who needed it most
  • alternately, people also drop out when the program works
  • SUMMARY: unless your report addresses the problem of attrition or experimental mortality, it will be a biased report

(2) measurement bias

• measurement bias exists when reseacher fails to contol for the effects of data collection and measurement
• e.g. tendency of people to give socially desirable answers
• • big problems when asking about violence, sex, money, criminal behavior, or when the person perceives there is something to loose by their answer
  • using self report is often biased by social desirability
  • in self report, we often use a "lie scale" or a social desirability index to control for "impression management"
  • • e.g. plant a series of questions like:
    • • I have never told a lie on purpose
      • I always know the difference between right and wrong
    • the score on the social desiability or lie scale is then used to statistically control for self-reporting bias
  • most clinical research is highly vulnerable to measurement bias
  • • e.g. using an invalid measure
  • the "self esteem" problem: tendency to think self esteem covers everything

(3) sampling bias

• sampling bias exists (beyond regression) when the sampling procedure introduces bias
• Key Sampling Problem #1: omission of women, Hispanics or other minorities from samples OR studying only minorities
• • most medical studies have been done on white or black males
  • • we know little about women and Heart disease

• • ALL child abuse studies have been done on women abusers
  • • we know virtually nothing about child abuse by men
  • almost ALL partner abuse studies have been done on heterosexual couples
  • • we know next to nothing about gay and lesbian partner violence
• Key Sampling Problem #2: targeting the most desirable or most accessible sample
• • e.g. in research on the effectiveness of batterers treatment programs, some researchers use conflictual couples seeking marriage counseling, and exclude court referred batterers, batterers with co-existing mental disorders, batterers who are severely violent, and batterers who are substance abusers . . . and then conduct the research in suburban university settings
  • the problem is not the sample, it's the failure to acknowledge the bias the sample brings

(4) procedural bias

• procedural bias exists most often when we administer the research interview or questionnaire under adverse conditions
• • using students
  • • e.g. using psych students for course credit
  • paying subjects
  • • e.g. my own study of addiction and domestic violence--Ss were paid $25 each, or $50 a couple
    • • for a couple of crack addicts, $50 is a powerful incentive to date for a few hours
  • e.g. administering questionnaires in a brief interval
  • • e.g. our own study of domestic violence in a Tennessee garment factory
    • • In addition to social desirability and anti-feminist measurement bias, we also have the questionnaires group administered in a small room during a meeting, and the subjects are paid by the piece so the longer it takes to fill out the questionnaire, the more money it costs them.

(5) "type III " error or problem bias

• type 1 error or false positive
• • independent variable had no effect, but you erroneously think it did
  • bias usually results from not acknowledging other factors which could account for the same result
• type 2 error or detection failure
• • independent variable had an effect, but you didn't notice
  • bias usually results from not acknowledging either
  • • your sample was too small
    • your measurement was too gross, or
    • you didn't do a good statistical analysis

• type 3 error: solving the wrong problem
• • bias results from
  • • failing to acknowledge we asked the wrong questions to the wrong people while trying to solve the wrong problem, or
    • we fail to adequately identify more problematic background assumptions

EXAMPLE #1: RQ: WHY DO BLACK STUDENTS SCORE LOWER ON IQ TESTS THAN WHITE STUDENTS?

• asks wrong question; some black students do score lower on IQ tests & some do not
• • unstated and untested assumption: black students are dumber than white students
• better question #1: WHICH BLACK STUDENTS SCORE LOWER ON IQ TESTS THAN WHICH WHITE STUDENTS?

• better question #2: WHAT FACTORS PREDICT BLACK STUDENTS SCORING HIGHER ON IQ TESTS THAN WHITE STUDENTS?

EXAMPLE #2: RQ: "WHY DO SOME RAPE VICTIMS HAVE A HIGHER INCIDENCE OF CHILDHOOD INCEST THAN NON RAPE VICTIMS?"

• this question is better than the previous question because it admits the possibility that some rape victims do not have a history of childhood incest, and implicitly wonders what factors predict which victims do and which victims dont.
• 3 problem w/ this question are:
• • ignores the rapist and focuses on the victim
  • implicitly assumes that one of the causes of rape is prior victimization
  • casts rape as a mental health problem rather than a criminal justice problem
• BETTER QUESTION #1: WHAT ARE THE CHARACTERISTICS OF MEN WHO RAPE?
• BETTER QUESTION #2: WHAT FACTORS ARE ASSOCIATED WITH SUCCESSFUL RAPE PROSECUTION

• This question part of a whole class of research which is victim-blaming
• • will talk more about doing victim-blaming research in a minute

EXAMPLE #3: WHAT EFFECT DO COMPANY PROVISION OF DAYCARE SERVICES HAVE ON THE JOB SATISFACTION OF WOMEN?

• • Implicitly assumes:
  • • women are (and should be) the caretakers of children
    • women should have children and be parents
    • women need to have children to be satisfied with their job
  • BETTER QUESTION: WHAT EFFECT DO COMPANY PROVISION OF DAYCARE SERVICES HAVE ON THE JOB SATISFACTION OF EMPLOYEES WHO HAVE CHILDREN and EMPLOYEES WHO DO NOT HAVE CHILDREN?
• all above questions
• • select vulnerable or minority populations to study
  • ask questions which hinge on an unstated assumption
• asking the wrong question and solving the wrong problem (Type 3 error) tends to support the phenomena of "blaming the victim" or what sociologists call thejust world hypothesis

Just World Hypothesis = in a fair and just world, good things happen to good people and bad things happen to bad people

• ever since adam & eve (in the levantine religions such as Judaism, Christianity, and Islam) anything connected to or close to nature is seen as implicitly BAD (bad=ill, mentally ill, the cause of problems, or the reason problems get worse)
• women are ascribed with mysterious natural powers (she casts a spell on men)
• non-white people, native or indigenous people are ascribed with dark powers (especially physical and sexual powers)
• animals or raw nature or "drive" are ascribed with uncontrollable impulses, which are seen as bad, "Libido"

How to blame victims in research:

1. Assume homogeneity. Assume a minority or at-risk population is homogenous

• "all MOYCHes are alike" (MOYCH=Minority of Your Choice)
• e.g. all women on welfare are similar

2. Wonder about variation. Ask the basic question of research: "why dont they score the average?"

• why arent they like the majority, not-at-risk group?
• why are black women on welfare?
• why are latinos so uneducated?

3. Speculate on a predictor of variation. Find some characteristic of the subgroup which may explain the deviance

• most black women on welfare are unmarried
• many latinos speak spanish in their home

4. Assume causality. If the characteristic is found in the minority subgroup, assume it explains the deviance from the norm

• the reason women are on welfare is that black men have abdicated their role as provider and black women have assumed leadership in the family
• • we now have a African American matriarchy of dominant, welfare dependent single mothers in the inner city (Plausible, except it ignores facts that: most women on welfare are white and child support non-payment, not race, is the major cause of welfare dependency)
• assume the reason latinos fail in school is speaking spanish; latino families do not care to be integrated into the mainstream by speaking english
• • we now see latino families as non-compliant, resistent, and mono-cultural (Plausible, but ignores the facts that most latino dropouts speak english; many latinos are bicultural, while most mainstream white people are mono-cultural;
  • • (Also ignores an alternative conclusion: the reason some latinos fail in school is not because they dont speak english, but because the schools dont speak spanish)

5. Attribution. Begin to view the characteristic as an attribute of the group

• • African americans are on welfare--thats how they are
  • • conclude: black women are welfare queens
  • Latino families are not interested in the "melting pot"--they want to do things the way they did in mexico or cuba or puerto rico
  • • conclude; latinos are lazy
• Another example: why do rape victims have a higher incidence of incest?
• • rape victims are over sexualized (victims of early abuse, so they are tuned in to sex)
  • study the sexual attitudes,belief & experiences of rape victims
  • if the results are different than non-rape victims (they will be), assume this supports the theory that rape victims are more sexualized than non-rape victims
  • therefore, sexualization is a predictor (cause) of rape
  • • i.e. the victim's sexuality causes the rape
• research with minorities require extra efforts to keep from bias
• • minority trait, not state
  • • key factor is power, not number
    • • Black africans in south africa used to be a minority, but in 1994 became a true majority, even though their numbers remained the same
    • "colored" and whites are now minorities in south africa
    • • colored=anyone who is not black or white
  • minority is anyone who is not (in USA):
  • • white
    • male
    • Christian
    • heterosexual
    • abled (physically & mentally)
    • age 18-65
• H2 do research with minorities:
• • be accountable to your Ss
  • carry out research with full involvement of Ss, from conceptualization to interpretation of results
  • use combination of qualitative & quantitative designs
  • • participant observation
    • semi-structured interviews
    • "back validation"
    • • panel/pretest/pilot
    • in depth interviews
    • language validation
    • narrative data (eg self anchored test)
  • avoid standardized instruments
  • avoid post-hoc studies; front load minority input and longitate
  • avoid assumption of homogeneity
  • select problems that are the felt difficulty of the community rather than the researcher
• Liane Davis' critique of "masculine research"
• • feminine research: connection, collaboration between Ss and rschr, studied in natural context, behaviors not studied in isolation
  • masculine rsch: hierarchal, researcher defines parameters in advance, Ss provides the information, phenomena & Ss are studied apart from their world, out of context
  • main result was the complexity revealed in the feminine study. Argues that social work problems are, in fact, complex, and therefore require a finine approach to research
• Martha Heineman Pieper's label of social work research as a "psuedoscience", and contasts to "huerism"
• • PSUEDOSCIENCE
  • • logical
    • positivistic: all information must come through the senses; if it isnt "sensible" it is not appropriate data; ultimately, if you cant measure it, it isnt real
    • empirical: observations representing reality
  • HUERISM (eg ethnography, phenomenology, ecology, structuralism)
  • Hueristic challenges:
  • • controlled experiments create a researcher-practitioner conflict
    • researcher is always biased, non-neutral observer
    • clinical problems are complicated, cant be reduced to testable statements
    • situational knowledge is more important than universal laws
    • statistical significance NE importance
    • • clinical significance
    • recognition of bias is more important tahn attempts to eliminate it
  • "solving the wrong problem" is Pieper's overall critiqie: she say that "it is better to find an approximate answer to the right question than an exact answer to the wrong question"