Step Assessments: Advanced Multiple Answer Multiple Choice (AMAMC)

Learners interact with the Advanced Multiple Answer Multiple Choice step assessment type in exactly the same way as the standard MAMC, but this assessment type provides an additional, flexible set of evaluation features for authors to utilize.

The basic operation of adding choices and configuring the funneled results is the same as the MAMC assessment type. Please review the MAMC tutorial for details on these assessment elements.

The AMAMC assessment type introduces the following features :

Vital for Correct Result (Choice option toggle)
Any choices with a Correct status will also have a toggle option available “Vital for Correct Result”.
  • If NO choices have this option toggled “on”, then ALL correct choices MUST be selected by the learner to arrive at the Correct funneled result. (Vital patterns are also checked for matching results, described in the Patterns section below)
  • If ONE OR MORE choices have this option toggled “on” :
    • ALL of the choices with this option toggled “on” MUST be selected by the learner to arrive at the Correct funneled result.
    • Any Correct choices that DO NOT have this option toggled “on” should be considered valid Correct choices, but are NOT vitally required to arrive at the Correct funneled result. Keeping these “non-vital” choices status as Correct allows for choices that may be “technically correct”, but should not affect the outcome as the Vital Correct choices (should not cause a “Partial” funneled outcome, for example.)

RULES
Minimum and Maximum Selected Requirement option toggles :
  • If Minimum Selected Requirement is toggled “off”, the minimum defaults to 1.
  • If Minimum Selected Requirement is toggled “on” :
    • The minimum value field is made available, defaults to 1, and must be between 1 and the number of available choices.
    • A “Forced” option toggle is made available for this requirement. This blocks the learner from checking their assessment results if the number of selected options is lower than the min. value.
  • If Maximum Selected Requirement is toggled “off”, the maximum defaults to the number of available choices.
  • If Maximum Selected Requirement is toggled “on” :
    • The maximum value field is made available, defaults to the number of available choices, and must be between the Minimum Selected Requirement value (1, if Minimum Selected Requirement is toggled “off”) and the number of available choices.
    • A “Forced” option toggle is made available for this requirement. This blocks the learner from checking their assessment results if the number of selected options is lower than the min. value.
  • If the “Forced” option is not toggled “on” for a min/max rule, and the number of learner’s assessment selections is breaches that min/max value, then the funneled result will be forced to Nullify (see more about the Nullify funneled result below)

PATTERNS
AMAMC Patterns provide an additional layer of controls for evaluating learner selections over the standard MAMC. The influence each Pattern on the funneled result can be set with fine detail, including independently-toggleable status and score values for each Pattern’s multiple evaluation results (a particular Pattern match result can influence status but not scoring, or scoring but not status, or both, or neither, etc. .) For convenience, each Pattern provides an optional field for labeling the Pattern within the authoring environment (non-learner facing.)


If a Pattern’s current member list or other configuration conditions create an evaluation conflict for the Pattern, the “offending” Pattern’s Pattern Members list will display RED and that Pattern will be excluded from the Pattern evaluation process in the Player (the assessment will otherwise operate normally, collecting and tallying all other valid choice and/or Pattern evaluation results into the funneled outcome.) If such a conflict is detected in the AMAMC editor, an orange conflict notice will appear in the Step editor header area, and the step on the map will gain an orange border.


There are three Pattern types available in the AMAMC assessment type :

“And” Patterns :
  • Require at least one member to be a valid Pattern.
  • Provide a mandatory “All Selected” Pattern result, triggered if a learner has selected ALL of the Pattern members (or the single choice in a single-member Pattern.) Typically this is the used as the “optimal” Pattern result for this Pattern type.
  • Provide an optional “None Selected” Pattern result, triggered if a learner has selected NONE of the Pattern members. Typically this is the used as the “worst” Pattern result for this Pattern type.
  • If two or more choices are in the Pattern’s member list, provide an optional “One+ Selected” Pattern result for that And Pattern. This Pattern result is triggered if one or more, but not ALL of the Pattern’s members are selected by the learner. Typically this is used as the “partially-acceptable” result for this Pattern type.
“Or” Patterns :
  • Require at least two members to be a valid Pattern.
  • Provide a mandatory “One Selected” Pattern result, triggered if a learner has selected ANY ONE of the Pattern members. Typically this is the used as the “optimal” Pattern result for this Pattern type.
  • Provide an optional “Two+ Selected” Pattern result, triggered if a learner has selected TWO OR MORE of the Pattern members. Typically this is used as the “partially-acceptable” result for this Pattern type.
  • Provide an optional “None Selected” Pattern result, triggered if a learner has selected NONE of the Pattern members. Typically this is the used as the “worst” Pattern result for this Pattern type.
“Avoid” Patterns :
  • Require at least one member to be a valid Pattern.
  • Provide a mandatory “None Selected” Pattern result, triggered if a learner has NOT selected ANY of the Pattern members (or the single choice in a single-member Pattern.) Typically this is the used as the “optimal” Pattern result for this Pattern type.
  • Provide an optional “All Selected” Pattern result, triggered if a learner has selected ALL of the Pattern members. Typically this is the used as the “worst” Pattern result for this Pattern type.
  • If two or more choices are in the Pattern’s member list, provide an optional “One+ Selected” Pattern result for that Avoid Pattern. This Pattern result is triggered if one or more, but not ALL of the Pattern’s members are selected by the learner. Typically this is used as the “partially-acceptable” result for this Pattern type.
Like choices, Patterns also have a Vital for Correct Result (Pattern option toggle) :
  • Any Patterns bearing a (toggled “on”) Correct status will also have a toggle option available “Vital for Correct Result”. If toggled “on”, any of the Pattern’s Correct results will be evaluated toward the funneled handler’s Correct outcome.
  • This combines with any non-Patterned choices that also have a Vital for Correct Result option toggled “on” (If ANY choices and/or Patterns are Vitally-Correct, then ALL Vitally-Correct choies/Pattern-match-results MUST be achieved for a Correct funneled result.)
Adding Patterns :
  • Each Pattern type has an Add Pattern button above the top-most Pattern box of that Pattern type.
  • New Patterns are appended below existing Patterns in the respective Pattern type list.
  • New Patterns will have no members assigned, and thus will be in “conflict mode” until they have choices assigned to them after creation.
  • Pattern boxes cannot be reordered in the editor panel.
Deleting Patterns :
  • Each Pattern box has an X (Delete) button in the upper right corner.
  • Deleting a Pattern will sever the linkage to all choices in the deleted Pattern’s member list. Choices that are not linked to any other Patterns will regain a “non-Patterned” condition and have the status selection (and scoring, if applicable) controls re-enabled.
Selecting Pattern Members :
Each Pattern has a Choose button above its Pattern Members list. This will enter the Pattern Member Selection mode :
  • All Choice boxes gain a control strip within the header area :
    • Each choice with the “Add To Pattern” option toggled “on” will be designated as a Pattern Member once the Pattern is applied.
    • The down-pointing arrow button will conveniently scroll the editor panel down to the active Pattern box.
  • Once all of the choices that should be members of the active Pattern are selected, return to the active Pattern (identified with a colored “halo”) and use the “Apply Selected Choices” button to complete the member selection and exit this mode.
    • Any choices that are members of any Pattern will display “Pattern” as the choice status and behavior. Pattern-assigned choices will remain as such until removed from all Patterns that include them as members.
    • Choices can be members of one or more Patterns. Meaningful and relevant Pattern membership is left to the Author to determine.
    • Score input values will be removed from any choices that are linked to one or more Patterns (as the Pattern-match results assume responsibility of scoring results for any Patterned choices.)
    • If the Pattern member selection process causes any choices to be removed from a Pattern, any of those choices that are not linked to any other Patterns will regain a “non-Patterned” condition and have the status selection (and scoring, if applicable) controls re-enabled.
  • If a Pattern has members applied, the Pattern member numbers can be clicked to conveniently scroll the editor panel up to the respective choice box in the choice stack.


Clearing Pattern Members :
  • Each Pattern has a Clear button above its Pattern Members list. This will remove all Pattern members from a Pattern.
  • Clearing all Pattern members will sever the linkage to all choices in the deleted Pattern’s member list. Choices that are not linked to any other Patterns will regain a “non-Patterned” condition and have the status selection (and scoring, if applicable) controls re-enabled.
FUNNELED RESULT HANDLING
Scroll down the editor panel to see how the assessment's responses are funneled to different results. Under Funneled Result Handling, you may see one or more drop-down options for handling certain response conditions :
  • Dirty-Correct: Learner provides one or more Correct AND one or more Non-Correct responses (only available if both Correct and Non-Correct responses are possible.)
  • Quasi-Correct: Learner provides one or more, but not all, necessary Correct responses AND no Non-Correct responses (only available if two or more Correct responses are possible.)
Funneling rules are as follows:
Response CombinationFunneled Result
Correct (no Non-Corrects, all necessary Corrects)Correct
Correct (no Non-Corrects, but not all necessary Corrects)Quasi-Correct
Partial (only)Partial
Incorrect (only)Incorrect
Neutral (only)Neutral
Correct + IncorrectDirty-Correct
Correct + PartialDirty-Correct
Correct + NeutralDirty-Correct
Correct + Incorrect + PartialDirty-Correct
Correct + Incorrect + NeutralDirty-Correct
Correct + Partial + NeutralDirty-Correct
Correct + Incorrect + Partial + NeutralDirty-Correct
Incorrect + PartialPartial
Incorrect + NeutralIncorrect
Incorrect + Partial + NeutralPartial
Partial + NeutralPartial
Nullify (if applicable, regardless of combination with any other status type)Nullify
If there are no responses with a status set to Partial or Neutral, there will not be a Partial or Neutral funneled result available.  You may manually add these results to the funneled handling stack by checking the Add Partial Result or Add Neutral Result, respectively.


For each funneled result, add the text you wish the students to see by typing in the Feedback Text box. If the feedback text is left blank, the student will still see a popup with the resulting status (Correct, Incorrect, etc.)  If you do not wish to have any feedback popup for a given funneled result, use the “Clear Feedback” button — with no feedback applied to the funneled result, the learner will simply move forward to the funneled result's target step. Test your changes and settings by viewing the step’s Preview.

It is possible that one or more existing funneled results may not be reachable by the learner due to assessment structure or the available response combinations, etc.. This is normal and should not be an issue.

Scoring AMAMC Assessments
If scoring is enabled, an option toggle “Score Funneled” is available in the CHOICES options.
  • If toggled “off” :
    • Scoring fields are placed in all non-Patterned, non-Neutral, non-Nullify choices. Each non-Patterned choice can have an optional “Unselected” scoring result added (via the “Add Unselected” button next to the score field), to allow for additional scoring conditions such as a “penalty” (negative) score value applied for Correct choices that the learner fails to select.
    • Scoring fields are placed in all non-Neutral, non-Nullify Pattern results. All Pattern scoring fields provide a toggle to enable/disable scoring for each respective Pattern result.
    • Scoring is tallied from the learner’s selections. All score-bearing, non-patterned choices and all score-bearing (and enabled) pattern match results are summed.
  • If toggled “on” :
    • All scoring fields are removed from the stack of Choices and Patterns, and instead scoring fields are placed in all non-Neutral-status, non-Retry-behavior funneled handler result boxes.
    • No score tally is performed for the assessment, all scoring is set by the funneled results.
Regardless of the “Score Funneled” toggle state, the Nullify funneled handler will retain a scoring field (if scoring is enabled.)

AMAMC Example
Here we are presenting the learner with a stack of 36 choices: The letters A-Z followed by numbers 0-9. A combination of “regular” Correct choices and AMAMC Patterns are used for determining funneled results. The author can decide if the learner is made aware of the “behind the scenes” evaluation Patterns via the given assessment instruction, etc. In this case, the configuration for resolving to the Correct funneled result is as follows:
  • Letters O, R, U selected (Non-Patterned, “regular” Correct choices)
  • Number 9 selected (Non-Patterned, “regular” Correct choice)
  • First 3 vowels in the alphabet selected (“And” Pattern 1 Correct result — One+ selected = Partial, None selected = Incorrect)
  • First 9 consonants in the alphabet selected (“And” Pattern 2 Correct result — One+ selected = Partial, None selected = Incorrect)
  • Letter W OR X selected (“Or” Pattern 1 Correct result — Two+ selected = Incorrect, None selected = Incorrect)
  • Letter Z AND Number 0 NOT selected (“Avoid” Pattern 1 Correct result — One+ selected, All selected are not enabled as status results for this pattern)
  • All other choices not noted above are set to Incorrect status