Samenvatting
Summary methods: experiments and surveys
This is a summary of the course methods: experiments and surveys. I incorporated my notes in this survey to make the slides more clear.
[Meer zien]Voorbeeld 3 van de 22 pagina's
Voorbeeld 3 van de 22 pagina's
In winkelwagengesponsord bericht van onze partner
Voorbeeld van de inhoud
Methods: Experiments and surveysLecture 1: introduction
Models are used by the central bank, etc., to get to policy advice. Within these models, we
need input like consumer and investor preferences and expectations. We can gather this kind
of data by doing experiments and surveys. This course is about data collection.
Pools of participants
- Sample of university students: a lot of internal validity
- Amazon Mechanical Turk: platform to recruit participants. Loss of internal validity.
Bots may respond, etc.
- Representative survey panel (LISS panel of Tilburg university): a lot of external
validity but internal validity is a bit lost (we do not have control over what
participants do in the experiment; they might communicate with others).
- Market research companies: more complexity possible, but often expensive
- Professionals: bring special expertise, not so much external validity
- Customers and clients: a lot of external validity
- Special groups: children/ elderly
All these groups have different costs and benefits and provide different implications for
internal and external validity (a lot of internal validity in laboratory experiments, but a lot of
external validity in field experiments).
Tools
- Pen and paper questionnaire
o Various degrees of complexity
o Interactive decisions are difficult
- Computerized experiment or survey (for example, veconlab)
o Programming toolboxes z-Tree, o-Tree.
Z-Tree = Zurich Toolbox enables unexperienced programmers to
quickly create experiments.
O-Tree = more complicated and requires some knowledge of
webserver infrastructure for the initial setup.
o A lot of complexity is feasible
- Simple online survey tools (for example, Qualtrics, SurveyMonkey, etc.).
o Simple tasks or questions
o No interaction possible
Lecture 2: survey structure, response modes and layout
Survey structure
Before conducting a survey, you need to think about how many questions you want to ask,
which of them are most important (put these first as participants lose concentration during the
task), what kind of topics you want to discuss, if there would be order effects and if this is the
case, you need to randomize (block or page randomization).
,Randomization = switch agree and disagree button sometimes. This makes the survey a bit
more difficult for people, but it rules out the left-right response bias.
We also need to think about how many questions in total we will ask and how many questions
we put on a page. If you have a coherent structure, you can improve the comprehension by
putting together questions with a similar topic.
Cross infection and suggestive questions can cause problems if they are on the same page.
In general, we need to minimize the number of questions that we ask, and we need to make
use of conditioning (that is, only ask question 5 if they answered Yes in question 4).
We also need to consider the possibility of revising answers. Of course, we want to know
people’s true views, so we should give them any possibility to reveal it. However, there might
be learning by answering later questions, in case revision leads to less ‘raw’ answers.
If we would have complex programming with conditioning, revisions might become
problematic. Also, if we have chained questions, later paid tasks depend on earlier answers
and the possibility of revision may lead to strategically wrong answers. I will explain the last
point by use of an example:
This person values the iPhone at 900$. The person knows that one out of the many steps is
actually paid. Therefore, he is incentive compatible: he chooses the option that is worth more
to him truthfully.
Using titration/bisection, we can zoom into his valuation. We just add more steps to get to
know his true valuation better.
The problem with titration is that people can answer untruthfully to get better/higher offers
afterwards. This becomes especially problematic when participants can go back and forth in
the survey and find out that this is indeed the case. The expected payoff over the 3 rounds is
higher if you do as if you value the iPhone at 1100$ instead of 900$. Therefore, the person
might want to misreport.
However, chained questions with real payments are still highly efficient in experimental
research. It is risky to give participants the possibility to revise their answer. There is also a
high risk that participants learn the mechanism if this design is not applied in a large
anonymous study. People can communicate with each other about the experiment in this case
, and share the procedure and payments, and therefore, get to know strategies that might give
them better payments. It is therefore important to disguise the method (people should not
know what is going on).
Introduction to a survey
- Don’t make it too long.
- There is a trade-off between giving a lot of details and respondents limited attentional
resources.
- Detailed instructions are only good if you can ensure that people will read and
understand them (like in an experiment, or one-on-one interview).
- We can introduce control questions that need to be answered correctly before
participants can proceed (can be annoying) or we can use thin slicing with
instructions. This means that we give general instructions and them later we make
group-of-items specific instructions.
Response modes
- Slider bars
o Intuitive visual response format
o Sometimes it is better if you do not indicate a starting point as this could
influence the decision (minimize framing bias)
o Often without step sizes indicated
- Open entry boxes
o They do not suggest potential answers and illicit the research question with
that
o They do not constrain answer by respondents
o However, we lose observations when people put unclear or confusing answers
o It may be time consuming, and the coding may become arbitrary (some people
might fill in male, other men, other people Mr., etc.)
o What answers to delete and what answers not is up to you. In general, you
have to make sure that the decision can be justified (based on research or well-
established procedures).
- Multiple choice
o Tight control over answers and structure
o However, asking about income may be problematic. Using ranges is often
more likely to be answered truthfully.
o If you force an answer, people might just drop out (attrition).
o Often you want to add the choice like “don’t want to say” or “don’t know”
- Multiple choice – Likert scale
o This is a subjective scale which is often used in psychometric measurements
o We assume linearity (going from 1 to 2 is just as big as going from 3 to 4)
o However, some people might not want to choose 1 or 5 ever, so the answers
may be biased in the middle.
- Multiple option check boxes
o Tight control, but it can suggest something
o There could be order effect, which necessitates us to randomize items
o If you limit how many boxes they can check, you force them to really select
their highest interests
o Create bottom where they can put their own insight
- Distributing weights
Voordelen van het kopen van samenvattingen bij Stuvia op een rij:
Verzekerd van kwaliteit door reviews
Stuvia-klanten hebben meer dan 700.000 samenvattingen beoordeeld. Zo weet je zeker dat je de beste documenten koopt!
Snel en makkelijk kopen
Je betaalt supersnel en eenmalig met iDeal, creditcard of Stuvia-tegoed voor de samenvatting. Zonder lidmaatschap.
Focus op de essentie
Samenvattingen worden geschreven voor en door anderen. Daarom zijn de samenvattingen altijd betrouwbaar en actueel. Zo kom je snel tot de kern!
Veelgestelde vragen
Wat krijg ik als ik dit document koop?
Je krijgt een PDF, die direct beschikbaar is na je aankoop. Het gekochte document is altijd, overal en oneindig toegankelijk via je profiel.
Tevredenheidsgarantie: hoe werkt dat?
Onze tevredenheidsgarantie zorgt ervoor dat je altijd een studiedocument vindt dat goed bij je past. Je vult een formulier in en onze klantenservice regelt de rest.
Van wie koop ik deze samenvatting?
Stuvia is een marktplaats, je koop dit document dus niet van ons, maar van verkoper britt1910. Stuvia faciliteert de betaling aan de verkoper.
Zit ik meteen vast aan een abonnement?
Nee, je koopt alleen deze samenvatting voor €4,49. Je zit daarna nergens aan vast.
Is Stuvia te vertrouwen?
4,6 sterren op Google & Trustpilot (+1000 reviews)
Afgelopen 30 dagen zijn er 53340 samenvattingen verkocht
Opgericht in 2010, al 14 jaar dé plek om samenvattingen te kopen