Pular para o conteúdo principal
Base de Conhecimento da FocusVision

Quotas Guide: Limit and Monitor Project Sample

Using the survey builder? Learn more about the Quota Element.

1:  Overview

Quotas are often used by researchers to obtain a sample of respondents that is statistically significant to the population that they're analyzing. They are also used to track and monitor the number of qualified completes in a survey.

The quotas for any given project can be viewed in the Quotas tab of the Field Report.

quota_base_example.png

If quotas are used to target a sample of a specific size, then quota limits can be set to prevent an excess number of qualified completes for any given sample group. On the other hand, if quotas are only used to monitor the number of completes for a given category, then quota limits can be set to infinite.

When a quota is called for a respondent, that respondent must fall into a quota bucket to continue the survey. If a respondent does not qualify for a quota bucket, they will be terminated and a special marker named "NQ" will be set.

2:  The Quota System

Quotas are created using Excel spreadsheets. For example, a simple gender quota may look like the following:

  A B
1 # = Gender  
2 Male 500
3 Female 500

Before you can create a quota table, however, you must define all of the quota markers that are available.

2.1:  Step 1: Define the Quota Markers

All quota markers should be specified in a sheet named "defines". The syntax for declaring quota markers is below:

  A B C
1 NAME CONDITION ALT. NAME

For example:

  A B C
1 qual plus  
2 Male Q1.r1  
3 Female Q1.r2  
4 Age_1 Q2.check('18-24') 18-24
5 Age_2 Q2.check('25-34') 25-34
6 Age_3 Q2.check('35-44') 35-44

The marker names are declared in column A. The condition in which a respondent should qualify for a given marker is declared in column B. Optionally, you can specify an alternative name for any marker to appear in the reports in column C.

In addition to the condition logic that you're used to, column B may also be set to "plus" for any marker. A marker with a "plus" condition is a created marker that any respondent can qualify for. Plus markers should be used when you need to synchronize randomly assigned markers across multiple tables; otherwise, a condition of '1 or 'True is standard practice. See the section on "cross-table" markers for more information.

2.2:  Step 2: Create the Quota Tables

Once we have defined all of the possible markers to be used in the survey in a sheet named "defines", we can create additional sheets with any naming convention to add our quota tables to. Quotas may consist of many sheets containing many quota tables. Multiple quota tables in a single sheet should be separated by a blank row.

The syntax for a simple quota table is below:

  A B
1 # = TABLE NAME  
2 MARKER NAME LIMIT

For example:

  A B
1 # = Qualified  
2 qual inf
3    
4 # = Gender  
5 Male 500
6 Female 500
7    
8 # = Age  
9 Age_1 350
10 Age_2 350
11 Age_3 300

A marker's LIMIT can be set to an integer, range, percentage, or "inf" for no upper limit.

Quotas can be nested to further refine the limit in place for sample groups. There are several different ways to nest quotas and the syntax for each is shown below:

  A B C
1 # MARKER NAME = TABLE NAME    
2 MARKER NAME LIMIT  
3 MARKER NAME LIMIT  
4      
5 # = TABLE NAME MARKER NAME MARKER NAME
6 MARKER NAME LIMIT LIMIT
7 MARKER NAME LIMIT LIMIT
8      
9 # = TABLE NAME #  
10 MARKER NAME MARKER NAME LIMIT
11   MARKER NAME LIMIT
12 MARKER NAME MARKER NAME LIMIT
13   MARKER NAME LIMIT

The emboldened MARKER NAME cells above represent markers that a respondent must also qualify for in addition to the regular markers listed in column A.

For example:

  A B C
1 # Male,Female = Age    
2 Age_1 350  
3 Age_2 350  
4 Age_3 300  
5      
6 # = Age x Gender Male Female
7 Age_1 175 175
8 Age_2 175 175
9 Age_3 150 150
10      
11 # = Gender x Age #  
12 Male Age_1 175
13   Age_2 175
14   Age_3 150
15 Female Age_1 175
16   Age_2 175
17   Age_3 150

The "Age" table above will only be called for respondents who qualify as "Male" and "Female". Since this is not possible, this "Age" table will never be called.

This "Age" table above is considered a conditional quota table because it is only called if a respondent is eligible for all of the markers listed between "#" and "=". A single marker may be specified or multiple markers separated by a comma.

The "Age x Gender" table above is set up to evenly distribute the age groups across each gender. For instance, there can only be 175 qualified males within the age range 18 - 24.

The "Gender x Age" table above is set up slightly different than the "Age x Gender" table but the limits are exactly the same.

You may also specify limits as percentages. The syntax for quota tables that use percentages is below:

  A B
1 # TOTAL = TABLE NAME  
2 MARKER NAME LIMIT %

For example:

  A B
1 # 1000 = Age  
2 Age_1 35%
3 Age_2 35%
4 Age_3 30%

In the quota table above, a hard limit of 350 (35% of 1000) is specified for "Age_1" and "Age_2". A hard limit of 300 (30% of 1000) is set for "Age_3".

2.3:  Step 3: Call the Quotas from the Survey

After we've defined all of the quota markers for the survey and created the necessary quota tables, we can save the Excel workbook to a file named "quota.xls" and upload it to the project's directory.

Use the following syntax to add a quota call to your survey:

<quota sheet="SHEETNAME" />

For example:

<quota sheet="age_gender" />

If the quota file is named something other than "quota.xls", then you can use the filename attribute to read a different file:

<quota filename="FILENAME" sheet="SHEETNAME" />

For example:

<quota filename="my-quota.xls" sheet="age_gender" />

This attribute will make the project incompatible with the builder user interface.

If you want the respondent to go somewhere else in the survey if they are overquota, you can specify overquota="LABEL" in the <quota> element to direct them to an element with label="LABEL". For example:

<quota overquota="Outtro_Message" sheet="age_gender" />

Just like survey questions, survey quotas will be called as soon as a respondent reaches the <quota> element within the survey. For example:

<radio label="Q1">
  <title>Are you...</title>
  <row label="r1">Male</row>
  <row label="r2">Female</row>
</radio>
<suspend/>

<number label="Q2" size="3" verify="range(1, 125)">
  <title>Please enter your age below:</title>
</number>
<suspend/>

<term cond="not Q2.check('18-44')">Q2: Age lt 18 or gt 44</term>

<quota sheet="gender_age" />

<html label="Introduction" where="survey">
  Congratulations! You've qualified for this survey.
</html>

If a <quota> element is placed after a question or comment element on the same page, the quota will be called before displaying the element.

3:  Examples

3.1:  A Simple Quota

In this example, we'll create a simple quota to limit respondents based on their gender and age. The quota specifications are below:

  • N = 2000 qualified respondents
  • 50/50 split between males & females
  • Split between the following age groups:
    • 600 for ages 18-24
    • 600 for ages 25-34
    • 400 for ages 35-44
    • 400 for ages 45-54

The gender and age questions that we'll use to create the quota logic are provided below:

<radio label="Q1" optional="0">
  <title>Are you...</title>
  <row label="r1">Male</row>
  <row label="r2">Female</row>
</radio>

<number label="Q2" size="3" verify="range(1,125)">
  <title>Please enter your age below:</title>
</number>
<suspend/>

<term cond="not Q2.check('18-54')">Q2: Age lt 18 or gt 54</term>

Given the questions above, the next step is to define the quota markers. In a new Excel workbook, create a sheet named "defines" and enter the following:

  A B C
1 qual plus  
2 Male Q1.r1  
3 Female Q1.r2  
4 Age_1 Q2.check('18-24') 18-24
5 Age_2 Q2.check('25-34') 25-34
6 Age_3 Q2.check('35-44') 35-44
7 Age_4 Q2.check('45-54') 45-54

With our markers defined, we can move on to creating the quota table named "general" based on the specifications provided:

  A B
1 # = Qualified  
2 qual 2000
3    
4 # 2000 = Gender  
5 Male 50%
6 Female 50%
7    
8 # = Age  
9 Age_1 600
10 Age_2 600
11 Age_3 400
12 Age_4 400

We can save the two sheets above, "defines" and "general", into a workbook named "quota.xls" and upload it to our project. Once it's in place, we can add a <quota> element to our survey and call the quota sheet:

<radio label="Q1" optional="0">
  <title>Are you...</title>
  <row label="r1">Male</row>
  <row label="r2">Female</row>
</radio>

<number label="Q2" size="3" verify="range(1,125)">
  <title>Please enter your age below:</title>
</number>
<suspend/>

<term cond="not Q2.check('18-54')">Q2: Age lt 18 or gt 54</term>

<quota sheet="general" />

Good to go.

3.2:  A Nested Quota

Similar to the previous example, we'll create a quota with the following specifications:

  • N = 2000 qualified respondents
  • Even distribution between genders of all age groups
    • Age groups: 18-24, 24-34, 35-44, 45-54

Using the following questions, we'll create a nested quota table to achieve an even distribution of genders and ages.

<radio label="Q1" optional="0">
  <title>Are you...</title>
  <row label="r1">Male</row>
  <row label="r2">Female</row>
</radio>

<number label="Q2" size="3" verify="range(1,125)">
  <title>Please enter your age below:</title>
</number>
<suspend/>

<term cond="not Q2.check('18-54')">Q2: Age lt 18 or gt 54</term>

The "defines" and "general" sheets with the nested quotas are specified below:

Click to show:   defines     general

  A B C
1 qual plus  
2 Male Q1.r1  
3 Female Q1.r2  
4 Age_1 Q2.check('18-24') 18-24
5 Age_2 Q2.check('25-34') 25-34
6 Age_3 Q2.check('35-44') 35-44
7 Age_4 Q2.check('45-54') 45-54
  A B C D E
1 # = Qualified        
2 qual 2000      
3          
4 # 2000 = Gender x Age Age_1 Age_2 Age_3 Age_4
5 Male 12.5% 12.5% 12.5% 12.5%
6 Female 12.5% 12.5% 12.5% 12.5%

After we save the quota sheets above into a file named "quota.xls" and upload it to our project's directory, we can call the "general" quota sheet from the survey using a <quota> element:

<radio label="Q1" optional="0">
  <title>Are you...</title>
  <row label="r1">Male</row>
  <row label="r2">Female</row>
</radio>

<number label="Q2" size="3" verify="range(1,125)">
  <title>Please enter your age below:</title>
</number>
<suspend/>

<term cond="not Q2.check('18-54')">Q2: Age lt 18 or gt 54</term>

<quota sheet="general" />

The code above produces the following result:

quotas_ex2_result.png

Learn more: Assigning Balanced Concepts in Quotas with Multiple Variables

3.3:  Lowest-Bucket Fill (Concept Picker)

The quota system is used to present concepts evenly to all respondents. In this example, we'll create the necessary quotas to properly assign a respondent one out of the three possible concepts. Additionally, we'll create a question named "vConcept" to track which concept each respondent was assigned.

Specified below are two sheets: the mandatory "defines" sheet and a "concept" sheet to pick one of the concepts for each respondent.

Click to show:   defines     concept

  A B
1 Concept_1 plus
2 Concept_2 plus
3 Concept_3 plus
  A B
1 # = Concept Pick  
2 Concept_1 inf
3 Concept_2 inf
4 Concept_3 inf

All three of the concept quota markers we declared are "plus" markers, which means that all respondents are eligible for each one.

By default, a quota table will assign only one of the markers that a respondent qualifies for. If a respondent qualifies for more than one marker, the quota system will choose the marker with the lowest percentage of completes.

This means that if the limits are the same (as they are in the example above), then the marker with the lowest number of completes will be chosen.

If the limits are different, however, then the marker with the lowest percentage of completes will be chosen. For example, a marker with 10 out of 100 completes has a lot more work to do to fulfill its requirement than a marker with 4 out of 5 completes, so the marker with 10 completes will be chosen.

If the limits are equal and the quota system is choosing markers based on the lowest count, then the quotas will maintain an even distribution throughout the study.

With the quotas specified above uploaded to our project's directory, we can call the "concept" sheet in our survey and create a question to track which concept marker was assigned:

<quota sheet="concept" />

<radio label="vConcept" where="execute">
  <title>HIDDEN: Assigned Concept</title>
  <exec>
for x in xrange(3):
    if hasMarker('Concept_{}'.format(x+1)):
        vConcept.val = x
        break
  </exec>
  <row label="r1">Concept 1</row>
  <row label="r2">Concept 2</row>
  <row label="r3">Concept 3</row>
</radio>
<suspend/>

<pipe label="Concept">
  <case label="c1" cond="vConcept.r1">CONCEPT 1</case>
  <case label="c2" cond="vConcept.r2">CONCEPT 2</case>
  <case label="c3" cond="vConcept.r3">CONCEPT 3</case>
  <case label="c99" cond="1">undefined</case>
</pipe>

<html label="Show_Concept">
    Now you will be asked about [pipe: Concept].
</html>

The code above produces the following result:

quotas_ex3_result.png

Notice that the concepts are being evenly distributed. If the quota system is assigning the next lowest bucket, then "Concept 1" is up next! If we were to run through this survey with QA codes on, we would see the following:

quotas_ex3_hidden.png

The hidden question above is populated to reflect the marker/concept we were assigned.

quotas_ex3_pipe.png

That's it! The proper concept was assigned and displayed to the respondent.

The only missing component of this example is a quota to cap the total number of respondents entering the survey. See the first example where we added a "qual" marker to set an upper-limit.

3.4:  Assigning Multiple Markers

Continuing from the previous example, what if we needed to assign two out of the three possible concepts for each respondent to see?

We can assign multiple quota markers by using a special syntax when we define the quota table. The revised "concept" sheet is below:

Click to show:   defines     concept

  A B
1 Concept_1 plus
2 Concept_2 plus
3 Concept_3 plus
  A B
1 # cells:2 = Concept Pick  
2 Concept_1 inf
3 Concept_2 inf
4 Concept_3 inf

Notice that we added "cells:2" just after the "#" in the "Concept Pick" quota table.

By default, a quota table will choose only one of the qualifying quota markers. Specifically, the one with the lowest count.

If we specify "cells:#" in the quota table, then the quota system will choose # of the qualifying markers. If a respondent qualifies for more than # markers, then the first # markers with the lowest count will be chosen. If a respondent qualifies for less than # markers, then all of the qualifying markers will be chosen.

Since we are choosing multiple concepts to display, we need to update our hidden question and pipes to account for the multiple marker assignment:

<quota sheet="concept" />

<checkbox label="vConcept" exactly="2" shuffle="rows" where="execute">
  <title>HIDDEN: Assigned Concept</title>
  <exec>
for x in xrange(3):
    if hasMarker('Concept_{}'.format(x+1)):
        vConcept.rows[x].val = 1
  </exec>
  <row label="r1">Concept 1</row>
  <row label="r2">Concept 2</row>
  <row label="r3">Concept 3</row>
</checkbox>
<suspend/>


<exec>ConceptLoop_expanded.order = vConcept.rows.order</exec>

<loop label="ConceptLoop" vars="concept" randomizeChildren="1">
  <block label="ConceptStage" cond="vConcept.r[loopvar: label]">

    <html label="Show_Concept_[loopvar: label]">
      Now you will be asked about [loopvar: concept].
    </html>

  </block>
<looprow label="1"><loopvar name="concept">CONCEPT 1</loopvar></looprow>
<looprow label="2"><loopvar name="concept">CONCEPT 2</loopvar></looprow>
<looprow label="3"><loopvar name="concept">CONCEPT 3</loopvar></looprow>
</loop>

The code above produces the following result:

quotas_ex4_result.png

The distribution of concepts is still evenly distributed. With 112 qualified completes, we can see that the quotas are working correctly since the total number of markers distributed shows 224 (e.g. 112 respondents * 2 markers each = 224).

If we were to run through the survey with QA codes turned on, we would see the following:

quotas_ex4_result2.png

The hidden question is a <checkbox> question to account for the number of concepts to be assigned.


The <loop> element will cycle through and show each concept appropriately.

3.5:  Random Selection / Monitoring a Variable

The quota system cannot randomly select quota markers. We can, however, monitor the selections made at a specific question whose answers are randomly populated.

Below is a hidden question whose responses are randomly generated:

<radio label="vRandomPick" where="execute">
  <title>HIDDEN: Random Choice</title>
  <exec>
vRandomPick.val = random.choice(vRandomPick.rows).index
  </exec>
  <row label="r1">1</row>
  <row label="r2">2</row>
  <row label="r3">3</row>
  <row label="r4">4</row>
</radio>

The quota specifications below are set up to monitor the selections made for the question above:

Click to show:   defines     random_monitor

  A B
1 Random_1 vRandomPick.r1
2 Random_2 vRandomPick.r2
3 Random_3 vRandomPick.r3
4 Random_4 vRandomPick.r4
  A B
1 # = Random Monitor  
2 Random_1 inf
3 Random_2 inf
4 Random_3 inf
5 Random_4 inf

We can add a <quota> element to our survey that calls the "random_monitor" sheet above just after the "vRandomPick" question:

<radio label="vRandomPick" where="execute">
  <title>HIDDEN: Random Choice</title>
  <exec>
vRandomPick.val = random.choice(vRandomPick.rows).index
  </exec>
  <row label="r1">1</row>
  <row label="r2">2</row>
  <row label="r3">3</row>
  <row label="r4">4</row>
</radio>
<suspend/>

<quota sheet="random_monitor" />

This quota tracks the random selections made at the "vRandomPick" question, so the data table for this question should be exactly the same as the "Random Monitor" table in the Quotas tab of the Field Report.

For example, the data table for this question looks like this:

quota_ex5_crosstabs.png

And the "Random Monitor" quota table looks like this:

quotas_ex5_field.png

The data is populated in exactly the same way. Even though the quota system didn't randomly assign a marker, we are achieving random-ness by using a hidden question. We are monitoring the selections made at this question by calling the <quota> element immediately after the random question.

If we want to, for instance, assign a random concept based on this selection, we can reference the question itself or the marker that is assigned.
e.g. cond="vRandomPick.r2" or cond="hasMarker('/random_monitor/Random_2')"

3.6:  Cross-Table Markers

By default, the quota system does not allow you to specify the same "plus" marker across multiple tables. For example, the following quota specifications generate an error:

Click to show:   defines     plus_table

  A B
1 MarkerA plus
2 MarkerB plus
3 MarkerC plus
  A B
1 # = Plus Marker Table #1  
2 MarkerA inf
3 MarkerB inf
4 MarkerC inf
5    
6 # = Plus Marker Table #2  
7 MarkerA inf
8 MarkerB inf
9 MarkerC inf

The "plus_table" sheet above generates the following error:

quota_ex6_error.png

In order to force the quota system to allow these "cross-table markers", we need to add a very special attribute to ALL <quota> elements in our survey. Instead of this:

<quota sheet="plus_table" />

We need to set the doit="1" attribute like this:

<quota sheet="plus_table" doit="1" />

With the special override doit="1" in place, our survey and quotas work as expected. The resulting quotas table is shown below:

quotas_ex6_result.png

Cross-table markers are often used to balance concept assignment across multiple variables. For example, if you needed to balance numerous concepts across respondents by age, gender, ethnicity, and any other variable, you must use cross-table markers and the special doit="1" attribute.

Learn more: Assigning Balanced Concepts in Quotas with Multiple Variables

When using doit="1", do not use multiple sets of plus markers in multiple tables (e.g. markers A1, A2, A3 in one dimension and B1, B2, B3 in another). Plus markers will not sync across tables when cells: is used; only one plus marker in each table will be selected and it is possible that multiple non-plus markers can be chosen per table. Please test your project to verify it is working per your expectation.

3.7:  Month-to-Month Tracking

Special surveys, such as tracker studies, sometimes need to field for long periods of time. Quotas often need to be updated after each period in order to establish new or reset existing limits.

In this example, we'll go over a method for creating monthly quotas that can easily switch between limits for each month.

There many ways to accomplish this task. The method described here should be very easy to implement using the survey builder.

At the very top of the survey, we will create an <exec> element that will set a marker relative to the current month:

<exec>
setMarker('May')
</exec>

Optional: We can automate the code above to automatically set the current month marker:

<exec>
current_month = datetime.datetime.now().strftime("%b")
setMarker(current_month)
</exec>

Using the code above, we can declare our quotas for the entire year!

Click to show:   defines     general

  A B
1 Jan hasMarker('Jan')
2 Feb hasMarker('Feb')
3 Mar hasMarker('Mar')
4 Apr hasMarker('Apr')
5 May hasMarker('May')
6 Jun hasMarker('Jun')
7 Jul hasMarker('Jul')
8 Aug hasMarker('Aug')
9 Sep hasMarker('Sep')
10 Oct hasMarker('Oct')
11 Nov hasMarker('Nov')
12 Dec hasMarker('Dec')
13 Male Q1.r1
14 Female Q1.r2
15 qual plus
  A B C D E F G H I J K L M
1 # = Limits Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2 qual 800 800 800 800 800 800 800 800 800 800 800 800
3                          
4 # = Limits Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
5 Male 400 400 400 400 400 400 400 400 400 400 400 400
6 Female 400 400 400 400 400 400 400 400 400 400 400 400

We can add the quotas above to our survey:

<exec>
setMarker('May')
</exec>

<radio label="Q1">
  <title>Are you...</title>
  <row label="r1">Male</row>
  <row label="r2">Female</row>
</radio>
<suspend/>

<quota sheet="general" />

<html label="Introduction" where="survey">
  Congratulations! You've qualified to take this survey.
</html>

The code above produces the following two tables in the Quotas tab of the Field Report:

quotas_ex7_month.png


When we need to begin fielding for a new month, we just need to update the <exec> element to set the new month marker (e.g. change "May" to "Jun").

If you've automated the month marker setting, then you're all set! You can field this survey all year long and never go over 800 completes per month!

3.8:  Soft Quotas

So far we've seen quotas with hard limits set. That is, all of the quotas have a maximum, upper-limit that cannot be exceeded.

It is possible to declare a minimum, lower-limit that a quota must achieve. These are called "soft quotas".

To use soft quotas, you must specify a total for the quota table as well as the minimum and maximum limits for each quota marker.

For example, given the following quota specifications:

  • N = 100
  • At least 40 male respondents
  • At least 50 female respondents

 

We can create a soft quota table to account for the specifications above:

Click to show:   defines     general

  A B
1 qual plus
2 Male Q1.r1
3 Female Q1.r2
  A B
1 # = Qualified  
2 qual 100
3    
4 # 100 = Gender  
5 Male 40-50
6 Female 50-60

Since we need at least 40 males, we specified a soft quota of 40-50. The 40 represents the lower-limit and the 50 is the upper-limit. Why did we choose 50 as the upper limit? Because the upper-limit should be set to a number that allows for all other soft quotas to be reachable.

Use the following formula to figure out the upper limit for a soft quota:

UPPER-LIMIT = TOTAL - THE SUM OF ALL OTHER LOWER-LIMITS

For example, the sum of all other lower limits is 50 for the male marker (because there's one other marker with a lower limit of 50). So 100 - 50 = 50.

For the female marker, the sum of all other lower limits is 40. So 100 - 40 = 60.

The quota table above produces the following result:

quotas_ex8_soft.png

The maximum number of female respondents allowed into this survey is 60 in order to leave room for the minimum amount of male respondents needed, 40.

When using soft quotas, the Quotas tab in the Field Report will automatically update the upper-limits based on how many respondents have entered the survey. As soon as a single bucket meets its lower-limit and begins to go over, the other marker's upper-limits will decrease to account for the decreasing number of buckets available.

3.9:  Priority Quotas

By default, quota markers have a priority level of 0 and they're all considered even. The level of quota marker priority/importance can be set per quota marker using a special syntax. For example:

Click to show:   defines     items

  A B
1 Item_1 Q3.r1
2 Item_2 Q3.r2
3 Item_3 Q3.r3
4 Item_4 Q3.r4
5 Item_5 Q3.r5
6 Item_6 Q3.r6
  A B
1 # cells:3 = Item Picker  
2 Item_1 inf:5
3 Item_2 inf:5
4 Item_3 inf
5 Item_4 250:4
6 Item_5 250
7 Item_6 250

Using the special "LIMIT:#" syntax, we are able to prioritize the assignment of certain markers. Higher priority # markers will be assigned first. Markers without a priority number set default to 0, the lowest.

In the quota above, "Item_1" and "Item_2" will always be chosen if the respondent is eligible for those markers. "Item_4" will always be chosen over "Item_(3, 5, 6)" until it meets its upper-limit of 250.

Assuming none of the markers have met their limit, here are a few examples of which markers would be assigned given the priority quota above:

Eligible for Item 1, 2, 3, 4
Item 1, 2 & 4 are assigned.
Eligible for Item 1, 2, 3
Item 1, 2 & 3 are assigned.
Eligible for Item 3, 4, 5, 6
Item 4 is definitely assigned (unless limit has been met).
The remaining two assignments will be based on the lowest counts for each Item 3, 5 or 6.
Eligible for Item 3, 5, 6
Item 3, 5 & 6 are assigned.
Eligible for Item 1
Item 1 is assigned.

3.10:  Quota Hooks

You can use a quota hook to modify the possible cell assignments for a respondent. 

To make such a custom modification to a quota, you define a quota_hook function inside a <exec when="init">< /exec> code block in the survey.

Within the quota_hook function, you receive the following information:

  •  quota sheet name (e.g. "Sheet1")
  •  table name
  •  list of cells

The list of cells contain the following:

  • fill fraction 
  • set of markers that make up the cell (e.g. male, 18-25 if you had a age*gender quota)
  • the marker that will be set if successful

The quota function can take the cell list and remove, reorder or even add new items. The quota function must return the modified cell list for the quota process to continue. When the quota process continues with the new cell list, for each table: the first marker in the cell list will be set (or the first N markers if using cells:N).

Example

For example, for a survey where a respondent views multiple concepts, you might want to use a quota hook to specify  that the respondent never sees the concept combination of concept1 and concept2.

For the quota hook example, a survey has a defines and two quota sheets as displayed below.

Defines

  A plus
1 one plus
2 two plus
3 three plus
4 never_picked  plus
5 concept1 plus
6 concept2 plus
7 concept3 plus
8 concept4 plus
9 concept5 plus
10 concept6 plus

Sheet1

  A B
1 # = ConceptsA  
2 one 25
3 two 25
4 three 25
5 never_pick 500

Sheet2

  A B
1 # = ConceptsB  
2 concept1 100
3 concept2 100
4 concept3 100
5 concept4 100
6 concept5 100
7 concept6 100

For sheet 1, we never want the never-pick marker to be set.  This is specified in the last line of code in the quota hook below.  We create a new list of cells where x[2] (marker name) is not /never_picked.

For sheet 2, we never want concept1 and concept2 assigned together. To do this, on quota sheet "Sheet2" we go over the eligible list of cells. We find the first of either concept1 or concept2. Then we mark the other quota cell for removal and return all but that quota cell. As a result, only concept1 OR concept2 can ever be picked.

<exec when="init">
C1 = '/Sheet2/concept1'
C2 = '/Sheet2/concept2'
def quota_hook(sheet, table, cells):
  if sheet == "Sheet2":
    remove = None
    for x in cells:
      if x[2] in (C1, C2):
          remove = C1 if x[2] == C2  else C2
          break
    return [x for x in cells if x[2] != remove]
  else:
    return [x for x in cells if x[2] != '/never_picked']
</exec>

4:  Data & Reporting

4.1:  Quota Data Tables

If "quota" is added to the setup attribute of the main <survey> element (e.g. setup="time,term,quota"), then two data tables are automatically generated for every quota table in the project.

Each data table will contain all of the markers for that particular quota table and the number of respondents who qualified for each marker. One data table will reflect the number of overquotas (e.g. voqtable1) and the other will reflect the number of qualified completes (e.g. vqtable1).

For example:

quotas_datatables.png

If you need to manually create these data tables, use the createQuotaTables() mutator function.

4.2:  Editing Quotas Online

The limits for all quota markers can be adjusted from the Quotas tab in the Field Report. Click on the "Edit Quotas" button and you'll have the ability to:

  • raise/lower a marker's limit
  • raise/lower the total limit
  • set a lock on a specific quota maker, stopping it from accepting future respondents

 

Learn more: How to Pause Quotas

4.3:  Accessing Quota Markers from the Survey

Quota markers are named using the following convention: /SHEETNAME/MARKER.

For example:

  • /general/Male
  • /concepts/Female/Age_2/Concept_1

 

If a marker is a "plus" marker, then the name of the marker by itself will also exist in the set of markers (e.g. "Concept_1").

Markers are stored in the persistent list, p.markers. For instance, you can print p.markers to see all of the markers assigned or check if a marker exists in this list with "/general/Female" in p.markers.

In the "Markers" report located in the "Other Reports.." found in Report (2010), you can see all of the markers currently assigned for a project.
e.g. https://v2.decipherinc.com/report/se...1234?markers=1

You can also access information about quota marker limits from within the survey.

For example:

<exec>
cells = gv.survey.root.quota.getQuotaCells()
current, limit, overquota = cells["/general/Male"]
isStopped = "/general/Male" in gv.survey.root.quota.stoppedCells
</exec>

The stoppedCells code above cannot be used in projects located in the /selfserve directory.

The getQuotaCells() function call above returns a dictionary with marker names as the keys and a tuple consisting of (current, limit, overquota) as the values (e.g. {'/general/Male' : (38, 40, 0)}).

If soft limits are in place, then the values will look slightly different (e.g. {'/general/Male' : (38, (40, 40, 50), 0)}).

You can use this information to manually assign quota markers using the setMarker() function without going over the quota limit.

The setMarker() function assigns markers to a respondent while the hasMarker() function checks if a marker already exists for a particular respondent. Learn more.

5:  The Quotas Tab

The Quotas tab can be found inside the Field Report for any project that contains a <quota> element.

Learn more: Quotas Tab

As shown below, there may be several different colored numerical indicators next to some of the marker limits:

quotas_indicators.png

5.1:  In Progress

A green (#) next to any quota marker indicates the number of respondents currently taking the survey who have qualified for that marker. In the screenshot above, there is currently 1 "Female" respondent who is taking the survey.

By default, a respondent can be inactive for 15 minutes before losing their assigned marker. Learn more.

5.2:  Overquotas

A red (#) next to any quota marker indicates the number of overquota respondents for that marker. In the screenshot above, there have been 14 respondents excused from the survey for being overquota.

5.3:  Unable to Qualify

An emboldened red +# next to any quota tables total indicates the number of respondents who were unable to qualify for any quota marker. This is likely caused by a programming error and should be addressed immediately. In the screenshot above, there has been 1 respondent who was unable to qualify for any of the markers in the "Gender" table. This could have happened if the gender question was optional and a respondent chose not to answer it, or if there was a third option (e.g. "Alien") that was left unaccounted for.

  • Este artigo foi útil?