Pular para o conteúdo principal
Base de Conhecimento da FocusVision

Crosstabs Technical Considerations

1: Transferring legacy configuration

The Crosstabs reporting system allows the user to be in control rather than requiring updates to the survey.xml. A separate configuration file is accessible via the web interface. Reports in the Report 2010 system and existing crosstab banners are converted or updated to the new format, the first time the project is accessed in the new Crosstabs reporting system. Additional changes made to the original reports.xml after accessing Crosstabs reporting system are not transferred. Question configuration is continuously converted until the table editor is used to save the settings for that table. After the table editor has settings saved for a table, only settings created in Crosstabs will apply while using the Crosstabs reporting system.

The first access of a project in Crosstabs creates the reports.json file.  When projects are copied, the Crosstabs files reports.json and pinned.json are included.  Any edits made in the table editor are in reports.json and are also transferred to the new project.

The following settings are converted:

  • From the reports.xml, each <report> becomes a saved crosstab with the same name, segments names and conditions (when reports.json doesn't exist in the project folder)
  • Previous Banners, Table Sets and Saved Crosstabs are converted to the new Crosstab format with the same name, segments, conditions, permissions and report settings (If applicable).
  • The value attribute, if set in the survey.xml will determine the initial statistical value when calculating averages (downloads use the value field in the XML).
  • The alt attribute will be used as the reporting text for the item or question.
  • Any <net> will transfer over creating nets within one or more tables or new tables. This includes nets implicitly added for questions using type="rating" in the XML.
  • The adim attribute will determine the grouping of the variables in the question.
  • Questions/Rows using where="survey" or where="none" are converted as not shown but can be enabled using the table editor.
  • Items with aggregate="0" are convert to exclude from base setting in the table editor.
  • Rows using where="none,notdp" cannot be accessed in any way.

The settings are stored in reports.json and can be removed to reset the transfer. 

Following settings are not transferred:

  • Saved crosstabs that were corrupted (i.e. would not run the saved selection).
  • Any chart-related configuration: whether a chart is displayed for a table, any palette or font settings.
  • Custom stat testing levels configured in styles (staff only).
  • Groups (<group>) are not shown in the report.

2:  Obsolete features and limitations

Following features from the Report 2010 system or previous version of Crosstabs no longer work but may be implemented in a future release:

2.1:  General user interface

  • The links shared include applied banners and filters, but not table sorting. Use Saved Reports to allow public and private sharing of Crosstabs. 
  • The inline open ended answers feature is no longer supported.
  • The table sets are, like the data layout manager, only inclusive. Thus a saved table set can only decide what questions and tables to include and will not change as new questions or tables are added.
  • The XML editor limits editing to a single crosstab and can modify the crosstab name, crosstab description, groups and segment logic.
  • There is currently no way to access the report standards (staff only) for the project or directory from crosstabs.
  • Rolling dates can't be added in the segment logic in the UI but can be added in the xml editor 

2.2:  Survey.xml settings

  • The sort attribute on questions could automatically sort table rows by row text (ascending or descending) or request that sorting would be based on counts or percentages. Crosstabs supports sorting only as a view option, and not as a saved setting.
  • The horizontalPercentage does not transfer to Crosstabs. This rare attribute would replace vertical with horizontal percentages while Crosstabs is more flexible.
  • The aggregate tag on questions where functions like stack() are used to create a stacked table is not supported. 
  • Any weighting configuration created via a <weight> is not used. A user interface for RIM weighting is in development.
  • Question nets are not included for selection in the logic editor.

2.3:  Exports

  • There is no print view.
  • There is no email feature.
  • There is no Word (RTF) export.
  • The Excel export generates Excel OOXML files which requires Excel 2007+.
  • The PowerPoint export generates PPTX files which requires PowerPoint 2007+.
  • To control the tables the PPT export generates, a custom table set must be created; the PPTX export will generate what is seen.

2.4:  Customization via styles

  • The user interface cannot be customized via styles (i.e. cells or precision on certain cells or question).
  • The structure or format of exports cannot be customized via styles.
  • The word cloud reporting presentation output is not supported.
  • The extremely custom basing system is not supported (report.main.customfilters style).
  • Local Javascript customizations are not supported (report.config.local style).
  • sigtest.levels (providing default significance testing levels) is not read in.

2.5:  Chart settings

  • Old chart settings and customization via styles carry over.
  • A much smaller set of chart options is available. Please see the user interface for options.

2.6:  Converted reports

  • Reports from the Report 2010 system (in reports.xml) are converted unless they contain any of the following:
    • A rolling date range that does not use days (but months, years, hours).
    • A number/float question restricted by percentiles (e.g. 90%+) or ranges.

3:  Crosstabs Interface Areas

The information in the sections below provide the technical considerations for specific Crosstabs interface areas. The current supported browsers are Chrome, Firefox and IE 11.

3.1:  Landing Page

When first entering Crosstabs, project resources are being loaded and the following functionality will be disabled:

  • Build a Crosstab
  • Quick Split by Question
  • Edit a Crosstab
  • Edit Settings

Users can run crosstabs but with a slight delay while the project resources are loaded. Searching and Exporting data is not affected.  

Crosstabs are currently shown in an alpha ascending sort order.  In a future release, there will be options to display crosstabs in various sort orders similar to the research hub.

3.1.1:  Searching

The search field matches the entered value against the crosstab name and description only.  In a future release matching may extend to segment names and user names.

3.1.2:  Exports

Exports that match the crosstab tables will reflect the selected table set. Raw data downloads do not limit data exported by table set.  To limit the data exported use a custom data layout. 

3.2:  Settings

3.2.1:  Date Filter

Preset date filters are dynamic and will update when running the crosstab.  "All" will update to "First Respondent - Last Respondent" matching the field dates.  Using a custom date range will apply a static filter. When pinning a table only the "All" date filter is dynamic.

3.2.2:  Respondent Status

Surveys with autorecover enabled (autoRecover="1") will add  "All and Partials"

3.2.3:  % Base

Surveys with blank checkbox tracking enabled (trackVars="checkbox") will add "Total Shown Variable"

3.2.4:  Favorites Table Set

When using the Favorites table set the crosstab permission is set to Private. The Favorites table set is a dynamic table set for each user and use in a saved crosstab would create a different report per user. A previous saved Crosstabs that saved a user's Favorite tableset will not run, the crosstab can be duplicated and the table set reapplied which will enforce the required 'private' crosstab permission.

3.3:  Editor

Crosstabs logic supports the dot operator, including .label, .attr(label), .any.all, .but(), .count , .sum, .search('[term]'), .check, .val,.shown and date logic: date_range(start,end), date_days(start,end).shown checks if the question was shown to the user. Other crosstabs logic matches vector logic. This logic is supported in both the segment logic and for the virtual question code used when using "Create Table".  Logic that cannot be shown in the logic editor will display a "complex" message in the logic editor. This includes logic that will run but uses the old Crosstabs syntax Q("[label]").

When a segment is being edited on the stage, each logic node shown creates logic inside a <cond></cond> in the XML.  There are some cases where the logic parser tries to simplify your logic. For example, when using the logic joiners "and not" or "or not", the xml may update the logic to use a joiner of "And"/"Or" and wrap the logic node logic with not().  Or when logic uses "all. Across All" (to add .any) in the UI, the logic may be broken out into several logic nodes for each answer choice selected.

Users can now create precedence in logic through the UI using the "merge" and "unmerge" logic buttons.  Merging follows these rules:

  • You can merge adjacent sibling nodes at the same level.
  • You can unmerge merged sibling nodes.
  • When there are only 2 sibling nodes in a container, you can only unmerge them.
  • When 3 or more node sibling nodes appear in a container, it becomes possible to merge them into a deeper nested container.

When you hover over the merge button, the two logic nodes what will be merged are highlighted with green borders and this creates similar logic to wrapping those nodes with parenthesis.  When you over over the unmerge button, the logic nodes that will be removed from the parent container are highlighted with red borders and this creates similar logic to moved the node outside the parenthesis. There may be multiple "+ Logic" buttons shown when logic nodes are nested.  When you hover over the "+ Logic" the parent container will be highlighted to show where the logic node will be added.  There currently is no drag and drop of the logic nodes but that feature may be added in a future release.

At this time, neither the XML or the logic editor validates segment logic.  An error message is shown when running the crosstab if logic is invalid.

Learn more: Using XML to Edit Crosstabs

3.4:  Report

Edits made to the chart settings (color scheme, data values and which type to display), grid table settings and table settings are global settings for all crosstabs in a project and are stored in reports.json.

The view options “Show/Hide QA Codes”, “Show/Hide Tables with No Respondents”, “Show/Hide Download Data Values” and "Show/Hide Unweighted Total Count” are user preferences that carry over from project to project.

3.4.1:  Table Editor

  • Numeric nets are value ranges.  -100 means a range starting with negative infinitely and ending with +100. Using an invalid numeric net, (i.e. a text value) will create an error in when trying to open the table editor for that question. The net can be removed from reports.json to correct the issue.

3.4.2: Weighting

Crosstabs supports fast per-record weighting (unlike the Report 2010 system, where assigning a weight to each respondent could resulting in reports 500 times as slow).

Currently access to weighting data is hard coded to a variable named nweight. If a nweight.txt file is placed in the survey directory, the weights in that file are matched via uuid.  nweight.txt is a tab-delimited file which contains at least two columns, the key (uuid) in the first column and weights in the column labeled "weight". 

Weighting uses double precision floating binary point; this gives about 15 digits of precision (12.34 requires 4) before any calculations are made.

3.4.3: Aggregate Support - Grid Tables

As of M28, closed end multidimensional questions in newly created projects (or projects access in crosstabs for the first time) will have grid tables automatically enabled by default.  Any projects accessed in crosstabs pre M28 or copied from pre M28 project will not have grid tables enabled.  Grid tables can be enabled or disabled in the table editor, per question or in the "View Options" for all but is limited to the single target format shown below. Three kinds of limited grid tables can be created by modifying the reports.json file located in the project directory. The syntax is below:

"reports": [
    "settings": {
        "aggregate": [
            { AGGREGATE 1 },
            { AGGREGATE 2 },
            ...
            { AGGREGATE 5 }
        ],
    ...
    }
}

The { AGGREGATE 1 } code above is a placeholder for objects of the following form:

{ "fun": "grid", "targets": ["q1"] }
{ "fun": "grid", "targets": ["q1", "q2"] }
{ "fun": "grid", "targets": ["q1", "q2"], "title": "Combined Table Title" }
{ "fun": "grid", "targets": ["q3_[looprow: label]"], "title": "Combined Loop Table", "pipe": "product" }

The "fun" key must be "grid". The "targets" is a list of questions to combine. When a simple/net aggregate table is generated from a list of questions, its columns are extracted and moved into the first matching table. Not all tables with extracted columns will be shown.

If you specify multiple targets, they are all combined into a single table. For example, you could combine q1 and q2 if they were asking about the same rows.

Loop questions can also be aggregated by providing the loop's label template name (e.g. q4_[loopvar: label]) and it will be expanded to include all questions. You may also specify a "pipe" with the name of the variable to be used as the column's title.

The loop syntax above is optional. You may also use the "targets" keyword to specify any of the expanded loop tables. (e.g. q4_1, q4_4, etc.)

The aggregation of tables into the grid table happens after table sets have been applied. This means that if q1 has 5 subtables for 5 attributes but your table set shows only 2 subtables, then aggregated q1 will also display only 2 columns.

When creating aggregate tables in Crosstabs using reports.json, please note that:

  • A 2D single or multi-select question can be turned into a single table where the individual tables are sliced into columns. If the report is split by multiple segments, one grid table is created for each segment.
  • Tables from multiple, unrelated questions can be turned into a single table. This makes sense only if the rows in those tables are identical.
  • All the instances of a looped question can be turned into a single question.
  • Favoriting is not supported.
  • Do not attempt to use this syntax for aggregating text/number questions or questions with with OE data (e.g. open="1"). Only single and multi-select questions are supported (e.g. radio, checkbox, select).
  • If multiple unrelated question tables are merged into one, it will not be possible to use the table editor to modify all questions simultaneously.
  • The Text Highlighter (hottext) DQ chart does not support grid tables.
  • Excel, PPT and PDF exports will correctly generate the tables.

3.4.4:  Virtual Question - Saved Variables

Users with report:edit project permission have the ability to create a virtual question by using the "Create Table" link in the Actions menu. As with Report 2010, this added table can be seen in both reporting applications.

What is happening behind the scenes?
In append.xml, a virtual checkbox question is added with after="survey" to appended after the survey questions and source="crosstabs" to differentiate between questions added via report 2010. The virtual question labels start with vrxq1 and increment. The crosstab name becomes the table question text, each segment of the crosstab becomes a table row and a group becomes a net in the table referencing the grouped segments. If the crosstab is saved, reports.json is updated to set segmentsTable to True for the crosstab.

What does the user see in Crosstabs?
After the virtual question has been added, the "Create Table" link becomes disabled for that saved crosstab. The user will not have an option to remove the virtual question, but has the option to hide the virtual question via table sets. The virtual question will be available to use for logic in the quick split/editor under "Question Logic". The "Create Table" link will re-enable if there is a change that would cause a change in the table, i.e. adding rows or changing logic, but not changes to a report setting.  Once created the virtual question is not modified and any additional use of  the "Create Table" link creates another virtual question.

This feature differs from Report 2010 in that:

  • The virtual question is automatically included in the data downloads
  • There is no way in the user interface to remove the virtual question
  • The virtual questions are appended to append.xml and not rewritten

3.4.5:  Statistical Testing

In the previous Crosstabs, segments using the "Total" or "ALL" logic were excluded from statistical testing. Users now have the option to include "Total" and by default the statistical testing group suggestion does not include "Total".  Statistical testing groups set the sg property for a segment in reports.json.  If a segment belongs to multiple groups, the groups are separated by commas in the sg property. Segments with matching sg values are tested against each other.  For example, a group of segments B,C,D that are being tested against each other have the same sg value of BCD.  In the settings modal this is shown as B-D. A stat grouping of BC,BD would test B in two groups. Segment B's sg value would be BC,BD, Segment C's sg value would be BC, Segment D's sg value would be BD~ is a special sg value which indicates no stat testing.  This sg value is assigned to segments that are not included in the statistical testing groups in the settings modal. For example the auto generated "Total" segment, by default, is assigned an sg value of ~

Statistical testing is currently limited to 26 segments.  A crosstab with more than 26 segments will have statistical testing groups and levels disabled in the settings. 

Statistical testing groups have no affect on statistical testing of the columns in a grid table but will use the same red stat testing identifiers.

3.4.6: Error Messages

When a segment is flagged as invalid one of the following error messages may be shown:

  • Error in segment "[segment name]".
    Error code: "#[CODE]".
    Segment condition: "[segment condition]".
  • "Report configuration is invalid. Contact support. Error: Unknown question '[question label]'
  • "Syntax error in condition for segment #[index] Total: '[bad logic]'

[CODE]: can be SYNTAXERROR, CUSTOM, quota which refer logic that can't be displayed in the UI or has a syntax error in the XML.

[question label]: the question references in the segment logic condition that it not found in the survey

[index] - is the index of the segment as shown in the preview, as with list the index starts with 0.

3.4.7: Restricting Values - minimumSegment

In Crosstabs, minimumSegment support is applied to segments with report filters such as "Date Range", "Respondent Status" and "Additional Filter".  Table filters which may change the segment total for that table only, do not affect the hiding of segment data. For a grid table, each segment is shown as a separate table and when the segment total is less than the minimum segment, the entire table is hidden. There currently is no indication of minimum segment restrictions being applied when running a crosstab or saved report. This will be included in a new crosstab segment summary table as part of the next iteration.

4:  Crosstab & Project Permissions

Most actions require users to have report:edit permission for the project.  Shared user will always have read only permission.

4.1:  Crosstab and table set permissions

Permission Description
Private Default permission. Only the owner is shown the crosstab or table set.
Editable Crosstab or table set is shown to all users. Crosstab tab or table set can be edited or deleted by user with report:edit access.
View Only Crosstab or table set is shown to all users. Only the owner can edit or delete the crosstab or table set.

A crosstab saved with a private table set will show the non-owner the table set in the drop down and run, but the non-owner will not be able to select the private table set in a different crosstab. 

4.2:  Permission needed to perform specific actions

Action Crosstab Permission User Project Permission
Build Crosstab / Create New Crosstab   report:edit
Edit Crosstab (in editor) editable or user is owner report:edit
Save (from settings or report) editable or user is owner report:edit
Duplicate Crosstab   report:edit
Download Raw Data   data:download or edit
Delete Crosstab editable or user is owner report:edit
+ New Table Set   report:edit
Edit Table Set editable or user is owner report:edit
Delete Table Set editable or user is owner report:edit
+ Upload Weighting Scheme   report:edit
Delete Weighting Scheme   report:edit
Create Table   report:edit
View Chart Options   report:edit
Turn All Grid Tables On/Off   report: edit
Table Editor   report:edit
Add / Edit Charts   report:edit
View Open End Data   data: download or edit
Transpose Table   report:edit
View Question   survey:view

5: Configuration beyond the user interface

The user interface does not currently allow access to all report configuration. Use the rconf command to modify the reports.json file in a safe way.

rconf <path/to/survey> [option] [option/value]
rconf path/to/survey verySmallSample

Running the command above will shows current value

rconf path/to/survey verySmallSample 50

Running the command above will set the value to 50

Option Value Description
verySmallSample 30 The threshold when the the ** warning is shown
smallSample 100 The threshold when the the * warning is shown
sigtestMajor 95 The global default testing level (in percentage) of the upper-case statistical testing indicator, used for new crosstabs.  This setting can be also set per crosstab in the settings modal
sigtestMinor 90 The global default testing level (in percentage) of the minor statistical testing indicator used for new crosstabs. This setting can be also set per crosstab in the settings modal
stats.prec 2 precision for statistics
pct 0 (rounded) precision for vertical percentages

verySmallSample and smallSample can be set to 0 to disable showing ** or *.
sigtestMajor or sigtestMinor can be set to 100 to disable.

6:  Misc features

  • All the download URLs that previously took a mysterious ?config=Xq1ew parameter can also take Python expressions, for example ?config=^qualified+and+q3.r1.
  • A number of calculations have been overhauled. Previously, number questions with amount= set on them would recode all NULL values to 0. That is no longer the case.
  • A subset of the Python syntax is used, but Crosstabs does not run it as actual Python code. Rather, it's compiled into a Virtual Machine that can evaluate the conditions at 500x the speed of Python. Each of the table rows and each of the segments has underneath such a logic condition.