Appendix A — Question Bank Items In Wikibase

In this guide you are going to learn how to feed different types of questions and information related to them to Wikibase.

First, you will read about how to load into Wikibase different type of questions. Second, you will learn about different types of information and how to link them to your question. Last, you will see how to add different translations to a question.

The concept behind the workflows follows the DDI-Lifecycle framework. The DDI lifecycle model is designed to support the documentation and management of data throughout its entire lifecycle, ensuring that data can be effectively shared, reused, and preserved. DDI-Lifecycle is not yet available in RDF. Some elements of DDI are standardised; others are not. Whenever possible, we use the standardised DDI-RDF Discovery Vocabulary; when no such ontology is present, we create our interpretation of DDI Hartmann et al. (2024).

DDI is part of the General Statistical Information Model (GSIM)(UNECE 2014), which accompanies the Generic Statistical Business Process Model (GSBPM) as an international standard model that “describes and defines the set of business processes needed to produce official statistics.” We use the conceptualisation of GSIM so that our results will be similar in quality to official statistics; of course, similar processes allow us to create products that combine well with official statistical products.

The two implementation standards, DDI and SDMX ensures that our observatories can work together with statistical offices, or respectable social sciences data repositories.

Applying the Data Documentation Initiatve (DDI) will ensure that we will remain compatible with official statistical microdata and metadata services and other social sciences archives, like GESIS, the official data archive of all European Commission-mandated survey research dating back over 50 years (Vardigan, Heus, and Thomas 2008).

A.1 Need for Questions

The need for questions arises from the fact that you want to collect some data systematically, either in an online or face-to-face questionnaire, in a structured interview, or in making data requests to an API or a form to record repeated answers to this question. After statistical manipulations, such as summarising and averaging, the responses will create empirical variables in a dataset. You can rely on existing data and expand your knowledge utilising already collected (open) data if you use the same questions that other researchers or statisticians have used before you.

Without addressing the theory of data harmonisation, these are the steps you are likely to make:

1. For example, if you need data on reusable plastic bags, you need to find a widely shared definition of plastic, reusability and bags.
2. You should consult a database or a question bank to find out if others have already asked about reusable plastic bags.
3. If you formulate a questionnaire using the exact wording and answering options as earlier surveys on attitudes to reusable plastic bags, you will be able to compare the results. So, you need access to question forms (question texts) and answer options.
4. If you work on an international project or want a global comparison, you will need to ensure that the question texts and answer options are translated very similarly and understood equally in different languages.
5. As a practical last step, the responses must be coded the same way as in international data repositories; for example, female respondents are coded with F in most statistical data repositories, even if the word ‘female’ may start with a different letter in many languages.

Our question bank is designed to be searchable by concepts, question types, question labels or question texts.

A.2 Question Types

What is a survey question after all? DDI organises questions into 3+1 hierarchical levels.

• Question: A question and its answer options formulated in at least one natural language, for example, Please tell me to what extent you agree or disagree with the following statement: “I trust that products carrying the EU ecolabel are environmentally-friendly.” [ ] Totally agree [ ], Tend to agree [ ], Tend to disagree [ ], Totally disagree [ ], DK. In this example DK refers to declined to comment on the question, or a refusal to answer this question.

• QuestionItem: the concept (i.e., unemployment, or plastic bags) being measured by the question, text for the question, response domain information, clarifying instructions, external aids (clarifying objects used in presenting the question to the respondent), Input and Output Parameters and Bindings, allowed response cardinality and estimation of the time required to respond.

• QuestionBlock: This structure is intended to bundle together a set of questions (items and/or grids) that have meaning only about a specified object expressed as the evaluation material. This form of question set is common in educational testing where a text, image, or other material is provided, and the respondent is asked questions specific to the material. For example, a portion of a play script is provided, and the respondent is asked questions concerning the dialogue and/or stage directions provided in the script. Note that the intent of QuestionBlock is not to bundle together a set of questions that are commonly used together or used in a specified order.

• QuestionGroup is only for administrative purposes.

A.2.1 Model question

One other way to make questions and resulting responses and their statistically processed variables comparable is to ask questions about different concepts in a same way? A standard quesiton in a Cultural Access and Participation Survey is:

How many times in the past 12 months have you been to ... ... a concert? ... cinema? ... church?

While people may have recollection biases about the 12 months, and may use a bit differently the concept of concert or cinema, because of the same syntax, context, we can assume that their responses are comparable. In this case, How many times in the past 12 months have you been to ... is a model question.

A model question is a question template that can create simple questions or question items in question grids or blocks.

• URI: Q127
• label: Trust in EU ECOLABEL [model]
• questionText (description): Please tell me to what extent you agree or disagree with the following statement: “I trust that products carrying the EU ecolabel are environmentally-friendly.” [ ] scale

And a question based on a model question, taken from

• QID: https://reprexbase.eu/demowiki/index.php?title=Item:Q111

• label: Trust in EU ECOLABEL

• questionText (description): Please tell me to what extent you agree or disagree with the following statement: “I trust that products carrying the EU ecolabel are environmentally-friendly.” [ ] Totally agree [ ], Tend to agree [ ], Tend to disagree [ ], Totally disagree [ ], DK

• variable representation: scale representation base type

• study (DDI): Eurobarometer 88.1 (2017)

Our model questions follow one of the following formats:

• questionText (description), [ ] concept, where the question connects to a concept, such as environmental protection (Q131).
• questionText (description), [ ] scale, where the answers are on a scale, for example Estimated number of employees in FTE [model] (Q123)
• questionText (description), [ ] category, where the answer options are categories, for example: Reduced use of single use plastic bags [model] (Q112)
• questionText (description), [ ] ranking, where the respondent has to create a rank from the answer options, for example: Important environmental issue [model] (Q143)
• questionText (description), [ ] concept, [ ] scale, where beside the model question there are other sub-questions as well, for example: Important for reduction of plastic [model] (Q128)

Based on the DDI-Lifecycle model we could generate differently structured model questions, and if there will be user need, we will introduce further question templates.

The DDI-Discovery ontology requires the questions to take this format:

Please tell me to what extent you agree or disagree with the following statement: “I trust that products carrying the EU ecolabel are environmentally-friendly.” [ ] Totally agree [ ], Tend to agree [ ], Tend to disagree [ ], Totally disagree [ ], DK.

This is a good representation to for an existing questionnaire, but it is not really good for a questionbank, because in some cases, the agreement scale may be a 3-level, in others, a 5-level agreement scale:

Please tell me to what extent you agree or disagree with the following statement: “I trust that products carrying the EU ecolabel are environmentally-friendly.” [ ], Agree [ ], [ ], Disagree [ ], DK

We can argue that the responses are still comparable, but [ ] Totally agree [ ], Tend to agree [ ] should be added together for a broader [ ] Agree category if one survey uses the 5-scale version of the response scale while the other uses the 3-scale (agree, disagree, decline) version.

This is why we separately record the model question, the subquestions and the answer options.

A.2.2 Simple, Multiple Choice and Matrix Questions

Different question types have different elements. Some questions consist of one question, others have group questions and several sub-questions.

A question might consists of the following elements:

[Model question] + [Question Items] + [Answer options]

• All elements should be added separately to the Wikibase

• The format changes based on the type of the question:

  • simple question: [Model question] + [Answers]
  • multiple choice question: [Model question] + [Question Items]
  • matrix question: [Model question] + [Question Items] + [Answer options]

Let’s see how to load into Wikibase:

• Simple Questions

• Matrix Questions

• Multiple Choice Questions

A.2.2.1 Simple Questions

In case of Simple Questions there’s only one question.

Note

For a clearer definition, see the disco:Question class.

The format of a simple question is: [Model question] + [Answer options]

Let’s see how to create a simple question entry in Wikibase. Go to “Special Pages”

Scroll down and select: Create a New Item

Fill the form with the question’s data:

• Language ▷ Choose the language (en)

• Label - Give a short name for the question

• Description - Enter the question itself in the format specified above.

• Aliases - leave it empty

Click Create.

The question now is created on Wikibase.

Note

Note: The system assigns a unique ID to every entry. In our example the ID is Q111.

A.2.2.2 Matrix Questions

Matrix questions have:

• a model question

• several question items

• answer options

This question is taken from the question block D (QD) from the Eurobarometer 88.1 study.

Following the functions “Special Pages” ▷ “Create New Item” you should feed into Wibikbase separately the model question and the questions items.

Q128 is the model question, which follows the structure:

• Language ▷ Choose the language (en)

• Label - questionName + [model] - Important for reduction of plastic [model]

• questionText (description)

  • In your opinion, how important is each of the following in reducing plastic waste and littering?

  • [ ] concept - stands for the question items

  • [ ] scale - stands for the answer options, which follow a scale

• Aliases - leave it empty

Q140 is a question item, which follows the structure:

• Language ▷ Choose the language (en)
• label: Important for reduction of plastic - collection facilities
• questionText (description)
  • model question - In your opinion, how important is each of the following in reducing plastic waste and littering?
  • question item: [ ] Local authorities should provide more and better collection facilities for plastic waste
  • [ ] scale - stands for the answer options, which follow a scale
• Aliases - leave it empty

When creating a “question item”, using statements, always connect the “question item” to the “model question”

A.2.2.3 Multiple Choice Questions

Multiple Choice questions have:

• a model question

• several question items

This question is taken from the question block D (QD) from the Eurobarometer 88.1 study.

Following the functions “Special Pages” ▷ “Create New Item” you should feed into Wibikbase separately the model question and the questions items.

Q143 is the model question, which follows the structure:

• Language - Choose the language (en)

• Label - questionName + [model] - Important environmental issue``[model] 

• questionText (description)

  • From the following list, please pick the four environmental issues which you consider the most important.

  • [ ] ranking - stands for the question items

• Aliases: leave it empty

Q144 is a question item, which follows the structure:

• Language ▷ Choose the language (en)
• label: Important for reduction of plastic - collection facilities
• questionText (description)
  • model question - From the following list, please pick the four environmental issues which you consider the most important.
  • question item - [ ] Decline or extinction of species and habitats, and of natural ecosystems (forests, fertile soils)
• Aliases - leave it empty

When creating a “question item”, using statements, always connect the “question item” to the “model question”

A.3 Add Metadata Statements to your Questions

Using Wikibase’s “statements” feature you can link different type of information to your questions.

You need to add further metadata statements to the question bank item. Metadata is a statement about the data. We are adding standard, basic statements in subject, predicate, and object (triplet) format to each question bank item.

IN the following this guide explain how to add information about:

• questionnaire classes

• variable representation (P265): a DDI-Lifecycle category for the creation of variables from the answer options, for example

• study (DDI) P270: the study where you can find this (model) question (item). In DDI, a study represents the process by which a data set was generated or collected (in a survey). For example, Eurobarometer 88.1 (2017) Q139

• related survey concept (P267): a concept that a study (group), question (group) or question item aims to measure, for example environmental protection (Q131).

A.3.1 Questionnaire Classes

Let’s start by specifying the entry we created as model question or question item.

Specify the entries created as model question or question item.

• Select +add statement.

• Using the instance of property, which is defining the taxonomical class of the entered item (in this case, a question.)

• In case of model questions, define them as model question.

• In case of question item, define them as question items.

In the case of questions items, always link them to the appropriate model questions using the model question (P266) property.

A.3.2 Variable Representation

When the questionnaire will be filled out in a raw dataset, each response of a question(item) will be translated into a variable. We need to define how we want to represent those answers in the resulting output dataset. (See DDI 3.3 (2020) documentation - Variable Value Representation and Question Response Domain)

Using statements you can define the representation of the variables. You can choose from the following categories:

• category representation base type: if the answers are categories (for example: [ ] Female, [ ] Male, [ ] Prefer not to say)

• category representation base type with a scale: if the answers are categories and follow a scale (for example: [] Very important, [ ] Fairly important, [ ] Not very important, [ ] Not at all important. )

• ranking representation base type: the respondant must rank the answer options, like 1st, 2nd, 3rd, etc.

• numeric variable representation base type: the answer should be a number, for example, the age of the respondant as an integer number or a postal code in a country where postal codes contain only numeric digits, f.e., 1051.

• textual variable representation base type: the answer should be some text, for example, and open answer, or a geographical location typed as a simple text, for example, Bratislava.

A.3.3 Define the source study

Tip

For further details, please check the disco:Study class.

With the study (DDI) P270 property you must link as a statement the study where you found the (model) question (item).

An example for a study: Eurobarometer 88.1 (2017) Q139

Note

Note: If the study is not yet in Wikibase, you can create an entry for it using the Create a New Item function.

A.3.4 Add related concept

With the related survey concept (P267) property you can link concepts, that a study (group), question (group) or question item aims to measure, for example environmental protection (Q131).

Where are the related concepts coming from?

1. The best case is that you use a widely accepted conceptualisation (ontology item) of your domain. For example, we took the study (Q149) concept from the DDI-Discovery (disco) ontology. You can connect statements of equivalence to a well-defined ontology via equivalent class (P69). In other words, our Q149 entity is equivalent to DDI’s Study.

2. If there are no accepted ontologies or you are uncertain, it is a very good practice to use a concept definition from Wikidata. Even in the case of an ontological definition, adding the Wikidata QID is a great idea because Wikidata connects equivalent definitions across various domains’ ontologies. You can make a statement about an equivalent Wikidata URI (for an item) by Wikibase URI (P73). See, for example plastic (Q148), Wikibase URI (P73), https://www.wikidata.org/wiki/Q11474, meaning that our plastic definition is equivalent to the Wikidata definition of plastic.

3. You can create your definition if you are still looking for a suitable definition in an accepted ontology or on Wikidata. For this, you should create a definition in Wikibase (as a new item.) See, for example, model question (Q126), which is our own proprietary definition until we find a more consensual one.

A.4 Add the questionText translations

On Wikibase you can add different language versions to the same question.

To do so, go to “Special Pages”

Scroll down and select: “Set Item/Property Description”

Fill the form:

• ID - The QiD of the question

• Language code - the new language you want to input the question

• Description - The question itself in the new language

Select “Set Description”.

The entry is now updated with another language.

Data Documentation Initiative. 2020. “DDI Lifecycle (3.3) Documentation.” https://ddi-lifecycle-documentation.readthedocs.io/en/latest/index.html.

Hartmann, Thomas, Sarven Capadisli, Franck Cotton, Richard Cyganiak, Arofan Gregory, Benedikt Kämpgen, Olof Olsson, Heiko Paulheim, Joachim Wackerow, and Benjamin Zapilko. 2024. “DDI-RDF Discovery Vocabulary. A Vocabulary for Publishing Metadata about Data Sets (Research and Survey Data) into the Web of Linked Data.” Edited by Thomas Hartmann, Richard Cyganiak, Joachim Wackerow, and Benjamin Zapilko. W3C. https://rdf-vocabulary.ddialliance.org/discovery.html.

UNECE. 2014. “Generic Statistical Information Model. GSIM V2.0 Documents. UNECE Statswiki.” 2014. https://statswiki.unece.org/display/gsim/GSIM+v2.0+documents.

Vardigan, Mary, Pascal Heus, and Wendy Thomas. 2008. “Data Documentation Initiative: Toward a Standard for the Social Sciences.” International Journal of Digital Curation 3 (1): 107–13.