LLMPrompt Tutorial

This tutorial explains how LLMPrompt is assembled and rendered.

It intentionally does not run end-to-end inference. The goal is to build intuition for prompt construction first, then move to workflow tutorials.

This tutorial is split into small blocks.

In each block, we look at one piece of prompt construction and when you would reach for it in practice.

import pandas as pd

from qstn.inference.response_generation import JSONSingleResponseGenerationMethod
from qstn.prompt_builder import LLMPrompt, generate_likert_options
from qstn.utilities import placeholder
from qstn.utilities.constants import QuestionnairePresentation

1. Minimal LLMPrompt Setup

Start with the smallest useful setup:

• questionnaire items (questionnaire_source)

• a global system_prompt

• a reusable user prompt template (prompt)

Use this base setup whenever you want one consistent prompt structure for many questionnaire items.

questionnaire_df = pd.DataFrame(
    [
        {"questionnaire_item_id": 1, "question_content": "The Democratic Party?"},
        {"questionnaire_item_id": 2, "question_content": "The Republican Party?"},
    ]
)

system_prompt = "You are a respondent in a survey."
prompt_template = f"Please answer the following item:\n{placeholder.PROMPT_QUESTIONS}"

# Keep one shared option set so examples stay comparable across sections.
option_labels = [
    "Strongly Dislike",
    "Dislike",
    "Neither Dislike nor Like",
    "Like",
    "Strongly Like",
]

questionnaire_base = LLMPrompt(
    questionnaire_name="Parties",
    questionnaire_source=questionnaire_df,
    system_prompt=system_prompt,
    prompt=prompt_template,
)

2. Placeholder: PROMPT_QUESTIONS

placeholder.PROMPT_QUESTIONS marks where the rendered question text is inserted in the user prompt.

You will use this placeholder almost always, because it is the anchor that turns your generic prompt template into an item-specific prompt.

system_text, user_text = questionnaire_base.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.SINGLE_ITEM,
    item_position=0,
)

print(f"SYSTEM PROMPT:\n{system_text}\n")
print(f"USER PROMPT:\n{user_text}")

SYSTEM PROMPT:
You are a respondent in a survey.

USER PROMPT:
Please answer the following item:
The Democratic Party?

3. Placeholder: QUESTION_CONTENT (via prepare_prompt)

prepare_prompt(...) lets you define item-level formatting with question_stem.

placeholder.QUESTION_CONTENT inserts each item’s raw question text into that stem.

This is useful when you want every item to follow the same wording style (for example, “How do you feel about …”) without manually writing each full question.

questionnaire_qcontent = questionnaire_base.duplicate()

questionnaire_qcontent.prepare_prompt(
    question_stem=f"How do you feel about {placeholder.QUESTION_CONTENT}",
)

_, user_text = questionnaire_qcontent.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.SINGLE_ITEM,
    item_position=1,
)

print(user_text)

Please answer the following item:
How do you feel about The Republican Party?

4. Placeholder: PROMPT_OPTIONS

placeholder.PROMPT_OPTIONS controls where answer options are inserted.

Use this when you need explicit answer choices in the prompt (Likert, multiple choice, constrained labels), and want to control exactly where those options appear relative to the question text.

A key benefit is that you can change your option setup in one place and keep the rest of the prompt logic unchanged.

options = generate_likert_options(
    n=len(option_labels),
    answer_texts=option_labels,
    options_separator=" | ",
)

questionnaire_options = questionnaire_base.duplicate()
questionnaire_options.prepare_prompt(
    question_stem=(
        f"How do you feel about {placeholder.QUESTION_CONTENT} "
        f"{placeholder.PROMPT_OPTIONS}"
    ),
    answer_options=options,
)

_, user_text = questionnaire_options.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.SINGLE_ITEM,
    item_position=0,
)

print(user_text)

Please answer the following item:
How do you feel about The Democratic Party? Options are: 1: Strongly Dislike | 2: Dislike | 3: Neither Dislike nor Like | 4: Like | 5: Strongly Like

Quick reconfiguration example:

If you only change the option labels, you can keep the same prompt template and question stem.

alternative_option_labels = [
    "Very Negative",
    "Negative",
    "Neutral",
    "Positive",
    "Very Positive",
]

alternative_options = generate_likert_options(
    n=len(alternative_option_labels),
    answer_texts=alternative_option_labels,
    options_separator=" | ",
)

questionnaire_options_changed = questionnaire_base.duplicate()
questionnaire_options_changed.prepare_prompt(
    question_stem=(
        f"How do you feel about {placeholder.QUESTION_CONTENT} "
        f"{placeholder.PROMPT_OPTIONS}"
    ),
    answer_options=alternative_options,
)

_, changed_user_text = questionnaire_options_changed.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.SINGLE_ITEM,
    item_position=0,
)

print(changed_user_text)

Please answer the following item:
How do you feel about The Democratic Party? Options are: 1: Very Negative | 2: Negative | 3: Neutral | 4: Positive | 5: Very Positive

5. Difference between different questionnaire presentation modes

QSTN supports multiple ways to present questions.

A practical way to think about it:

• SINGLE_ITEM: one question per call, fresh context

• SEQUENTIAL: multi-turn conversation where history is kept

• BATTERY: all questions in one prompt

Choose based on your study design and how much context you want the model to carry across items.

print("=== SINGLE_ITEM ===")
_, single_item_turn1 = questionnaire_options.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.SINGLE_ITEM,
    item_position=0,
)
print(f"\n[item 1]\n{single_item_turn1}")

print("\n=== SEQUENTIAL Turns ===")
first_turn = single_item_turn1
second_turn = questionnaire_options.generate_question_prompt(questionnaire_options.get_question(1))
print(f"\n[turn 1: full template]\n{first_turn}")
print(f"\n[turn 2: question only]\n{second_turn}")

_, battery_prompt = questionnaire_options.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.BATTERY,
    item_position=0,
    item_separator="\n---\n",
)
print(f"\n=== BATTERY (all questions in one prompt) ===\n{battery_prompt}")

=== SINGLE_ITEM ===

[item 1]
Please answer the following item:
How do you feel about The Democratic Party? Options are: 1: Strongly Dislike | 2: Dislike | 3: Neither Dislike nor Like | 4: Like | 5: Strongly Like

=== SEQUENTIAL Turns ===

[turn 1: full template]
Please answer the following item:
How do you feel about The Democratic Party? Options are: 1: Strongly Dislike | 2: Dislike | 3: Neither Dislike nor Like | 4: Like | 5: Strongly Like

[turn 2: question only]
How do you feel about The Republican Party? Options are: 1: Strongly Dislike | 2: Dislike | 3: Neither Dislike nor Like | 4: Like | 5: Strongly Like

=== BATTERY (all questions in one prompt) ===
Please answer the following item:
How do you feel about The Democratic Party? Options are: 1: Strongly Dislike | 2: Dislike | 3: Neither Dislike nor Like | 4: Like | 5: Strongly Like
---
How do you feel about The Republican Party? Options are: 1: Strongly Dislike | 2: Dislike | 3: Neither Dislike nor Like | 4: Like | 5: Strongly Like

6. Placeholder: PROMPT_AUTOMATIC_OUTPUT_INSTRUCTIONS (RGM tie-in)

Response Generation Methods (RGM) can inject output-format instructions automatically.

When you include placeholder.PROMPT_AUTOMATIC_OUTPUT_INSTRUCTIONS in the system prompt or user prompt, QSTN fills it using the configured response generation method.

Use this when you want output constraints (for example JSON format) to stay coupled to the chosen response-generation method instead of hardcoding instructions manually.

options_with_rgm = generate_likert_options(
    n=len(option_labels),
    answer_texts=option_labels,
    response_generation_method=JSONSingleResponseGenerationMethod(),
)

questionnaire_rgm = LLMPrompt(
    questionnaire_name="Parties",
    questionnaire_source=questionnaire_df,
    system_prompt=(
        f"You are a respondent in a survey. {placeholder.PROMPT_AUTOMATIC_OUTPUT_INSTRUCTIONS}"
    ),
    prompt=(
        f"Answer this item:\n{placeholder.PROMPT_QUESTIONS}\n"
        f"{placeholder.PROMPT_OPTIONS}\n"
        f"{placeholder.PROMPT_AUTOMATIC_OUTPUT_INSTRUCTIONS}"
    ),
)

questionnaire_rgm.prepare_prompt(
    question_stem=f"How do you feel about {placeholder.QUESTION_CONTENT}",
    answer_options=options_with_rgm,
)

system_text, user_text = questionnaire_rgm.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.SINGLE_ITEM,
    item_position=0,
)

print(f"SYSTEM PROMPT WITH AUTOMATIC INSTRUCTIONS:\n\n{system_text}\n")
print(f"USER PROMPT WITH AUTOMATIC INSTRUCTIONS:\n\n{user_text}")

SYSTEM PROMPT WITH AUTOMATIC INSTRUCTIONS:

You are a respondent in a survey. You only respond with the most probable answer option in the following JSON format:
{
  "answer": "choose one of: Options are: 1: Strongly Dislike, 2: Dislike, 3: Neither Dislike nor Like, 4: Like, 5: Strongly Like"
}

USER PROMPT WITH AUTOMATIC INSTRUCTIONS:

Answer this item:
How do you feel about The Democratic Party?
Options are: 1: Strongly Dislike, 2: Dislike, 3: Neither Dislike nor Like, 4: Like, 5: Strongly Like
You only respond with the most probable answer option in the following JSON format:
{
  "answer": "choose one of: Options are: 1: Strongly Dislike, 2: Dislike, 3: Neither Dislike nor Like, 4: Like, 5: Strongly Like"
}

7. Full Prompt Setup: All Blocks Together

This final example combines all core building blocks in one place:

• PROMPT_QUESTIONS

• QUESTION_CONTENT

• PROMPT_OPTIONS

• PROMPT_AUTOMATIC_OUTPUT_INSTRUCTIONS

Use this as a template when you set up a real questionnaire: define the global prompt once, attach options/stems with prepare_prompt(...), and render per presentation mode.

full_setup = LLMPrompt(
    questionnaire_name="PartiesFull",
    questionnaire_source=questionnaire_df,
    system_prompt=(
        f"You are a careful survey respondent. "
        f"{placeholder.PROMPT_AUTOMATIC_OUTPUT_INSTRUCTIONS}"
    ),
    prompt=(
        "Task:\n"
        f"{placeholder.PROMPT_QUESTIONS}\n"
        f"{placeholder.PROMPT_AUTOMATIC_OUTPUT_INSTRUCTIONS}"
    ),
)

full_options = generate_likert_options(
    n=len(option_labels),
    answer_texts=option_labels,
    response_generation_method=JSONSingleResponseGenerationMethod(),
)

full_setup.prepare_prompt(
    question_stem=(
        f"How do you feel towards {placeholder.QUESTION_CONTENT}\n"
        f"{placeholder.PROMPT_OPTIONS}"
    ),
    answer_options=full_options,
)

system_turn1, user_turn1 = full_setup.get_prompt_for_questionnaire_type(
    questionnaire_type=QuestionnairePresentation.SEQUENTIAL,
    item_position=0,
)
turn2_question_only = full_setup.generate_question_prompt(full_setup.get_question(1))

print(f"FIRST TURN (template + placeholders):\n\nSYSTEM:\n{system_turn1}\n")
print(f"USER:\n{user_turn1}\n")
print("SECOND TURN IN SEQUENTIAL (question-only follow-up):\n")
print(turn2_question_only)

FIRST TURN (template + placeholders):

SYSTEM:
You are a careful survey respondent. You only respond with the most probable answer option in the following JSON format:
{
  "answer": "choose one of: Options are: 1: Strongly Dislike, 2: Dislike, 3: Neither Dislike nor Like, 4: Like, 5: Strongly Like"
}

USER:
Task:
How do you feel towards The Democratic Party?
Options are: 1: Strongly Dislike, 2: Dislike, 3: Neither Dislike nor Like, 4: Like, 5: Strongly Like
You only respond with the most probable answer option in the following JSON format:
{
  "answer": "choose one of: Options are: 1: Strongly Dislike, 2: Dislike, 3: Neither Dislike nor Like, 4: Like, 5: Strongly Like"
}

SECOND TURN IN SEQUENTIAL (question-only follow-up):

How do you feel towards The Republican Party?
Options are: 1: Strongly Dislike, 2: Dislike, 3: Neither Dislike nor Like, 4: Like, 5: Strongly Like

8. Quick Recap

• LLMPrompt(...) defines reusable global prompt structure.

• prepare_prompt(...) is where item-level stems and options are attached.

• Placeholders control insertion points, so you can refactor prompts without rewriting question content.

• Sequential surveys usually start with a full first turn and continue with question-only follow-ups.

• get_prompt_for_questionnaire_type(...) is the central renderer for all presentation modes.

Next step: continue with Tutorial 1: Multiple Choice Questionnaires.