ExpLookitDialoguePage Class
Frame to implement a storybook page with dialogue spoken by characters. First, characters appear and any main narration audio is played. Next, the user can click on the characters to play additional audio associated with each character image, or (for a choice trial) the user clicks one of the images to select it as an answer. Once main narration audio has been played and either a selection has been made (for a choice trial, isChoiceFrame: true) or all required character audio has been played (for a non-choice trial), the user can proceed by pressing 'next'. (A trial with only main narration audio can also simply auto-proceed when audio is finished.)
Recording is optional. If webcam recording is conducted (doRecording: true) then audio does not start until recording does, to ensure the entire trial is recorded.
The character images are specified in 'images', including an image source, positioning on the screen, any animation at the start of the trial, any associated audio, and whether that audio is required.
This frame extends ExpFrameBaseUnsafe because it is displayed fullscreen and expected to be repeated.
The examples below show a few expected uses of this frame. In phase-2, two characters are shown; the protagonist is already present, and speaker1 flies in from the left. Speaker1 has associated audio (dialogue). After the narrative audio, the user can click on speaker1 to play the audio, and will then be able to proceed.
phase-5 is a choice trial, where the user has to click on one of the two images before proceeding.
"frames":
"phase-2": {
"kind": "exp-lookit-dialogue-page",
"id": "phase-2",
"baseDir": "https://s3.amazonaws.com/lookitcontents/politeness/",
"audioTypes": ["mp3", "ogg"],
"backgroundImage": "order1_test1_background.png",
"doRecording": false,
"autoProceed": false,
"parentTextBlock": {
"title": "Parents!",
"text": "some instructions",
"emph": true
},
"images": [
{
"id": "protagonist",
"src": "order1_test1_listener1.png",
"left": "40",
"bottom": "2",
"height": "60"
},
{
"id": "speaker1",
"text": "Click to hear what he said!",
"src": "order1_test1_speaker1.png",
"left": "20",
"bottom": "2",
"height": "60",
"animate": "flyleft",
"requireAudio": true,
"imageAudio": [
{
"stub": "polcon_example_2_2speaker1polite"
}
]
}
],
"audioSources": [
{
"audioId": "firstAudio",
"sources": [
{
"stub": "polcon_example_2_1intro"
}
]
}
]
},
"phase-5": {
"kind": "exp-lookit-dialogue-page",
"id": "phase-5",
"baseDir": "https://s3.amazonaws.com/lookitcontents/politeness/",
"audioTypes": ["mp3", "ogg"],
"backgroundImage": "order1_test1_background.png",
"doRecording": false,
"autoProceed": false,
"isChoiceFrame": true,
"parentTextBlock": {
"title": "Parents:",
"text": "click on the character your child selects.",
"emph": true
},
"images": [
{
"id": "speaker1",
"src": "order1_test1_speaker1.png",
"left": "20",
"bottom": "2",
"height": "60"
},
{
"id": "speaker2",
"src": "order1_test1_speaker2.png",
"left": "60",
"bottom": "2",
"height": "60"
}
],
"audioSources": [
{
"audioId": "firstAudio",
"sources": [
{
"stub": "polcon_example_5q1"
}
]
}
]
}
}
Item Index
Properties
Methods
exitFullscreen
()
serializeContent
-
videoID
-
eventTimings
-
currentlyHighlighted
Parameters captured and sent to the server
Parameters:
Returns:
The payload sent to the server
showFullscreen
()
Properties
audioSources
Object[]
Array of objects describing audio to play at the start of this frame. Each element describes a separate audio segment.
Sub-properties:
-
audioId
Stringunique string identifying this audio segment
-
sources
Object[]Array of {src: 'url', type: 'MIMEtype'} objects with audio sources for this segment.
Can also give a single element {stub: 'filename'}, which will be expanded out to the appropriate array based on
baseDir
andaudioTypes
values; seeaudioTypes
. -
highlights
Object[]Array of {'range': [startT, endT], 'image': 'imageId'} objects, where the imageId values correspond to the ids given in images
audioTypes
String[]
List of audio types to expect for any audio specified just with a string rather than with a list of src/type pairs. If audioTypes is ['typeA', 'typeB'] and an audio source is given as [{'stub': 'intro'}], the audio source will be expanded out to
[
{
src: 'baseDir' + 'typeA/intro.typeA',
type: 'audio/typeA'
},
{
src: 'baseDir' + 'typeB/intro.typeB',
type: 'audio/typeB'
}
]
Default: ['mp3', 'ogg']
baseDir
String
Base directory for where to find stimuli. Any image src
values that are not full paths will be expanded by prefixing
with baseDir
+ img/
. Any audio/video src values that give
a value for 'stub' rather than 'src' and 'type' will be
expanded out to
baseDir/avtype/[stub].avtype
, where the potential avtypes
are given by audioTypes and videoTypes.
Note that baseDir SHOULD include a trailing slash
(e.g., http://stimuli.org/myexperiment/
, not
http://stimuli.org/myexperiment
)
Default: ''
doRecording
Boolean
Whether to do webcam recording (will wait for webcam connection before starting audio if so)
fullScreenElementId
String
images
Object[]
Array of images to display and information about their placement
Sub-properties:
-
id
Stringunique ID for this image. This will be used to refer to the choice made by the user, if any.
-
src
StringURL of image source (can be full URL, or stub to append to baseDir; see
baseDir
) -
left
Stringleft margin, as percentage of story area width
-
height
Stringimage height, as percentage of story area height
-
bottom
Stringbottom margin, as percentage of story area height
-
animate
Stringanimation to use at start of trial on this image, if any. If not provided, image is shown throughout trial. Options are 'fadein', 'fadeout', 'flyleft' (fly from left), and 'flyright'.
-
text
Stringtext to display above image, e.g. 'Click to hear what he said!' If omitted, no text is shown.
-
imageAudio
Object[]sources Array of {src: 'url', type: 'MIMEtype'} objects with audio sources for audio to play when this image is clicked, if any. (Omit to not associate audio with this image.)
Can also give a single element {stub: 'filename'}, which will be expanded out to the appropriate array based on
baseDir
andaudioTypes
values; seeaudioTypes
. -
requireAudio
Booleanwhether to require the user to click this image and complete the audio associated before proceeding to the next trial. (Incompatible with autoProceed.)
isChoiceFrame
Boolean
Whether this is a frame where the user needs to click to select one of the images before proceeding
Default: false
nPhase
Number
Phase number (just included as a convenience & sent to server, to make handling collected data simpler)
Default: 0
nTrial
Number
Trial number (just included as a convenience & sent to server, to make handling collected data simpler)
Default: 0
Events
completeMainAudio
When narration audio is completed
completeSpeakerAudio
When image audio is completed (not recorded if interrupted)
Event Payload:
-
imageId
String
enteredFullscreen
Upon detecting change to fullscreen mode
leftFullscreen
Upon detecting change out of fullscreen mode
nextFrame
Move to next frame
stoppingCapture
Just before stopping webcam video capture
videoStreamConnection
When recorder detects a change in video stream connection status
Event Payload:
-
status
Stringstatus of video stream connection, e.g. 'NetConnection.Connect.Success' if successful