API Docs for: 0.5.1
Show:

ExpLookitDialoguePage Class

Frame to implement a storybook page with dialogue spoken by characters. First, characters appear and any main narration audio is played. Next, the user can click on the characters to play additional audio associated with each character image, or (for a choice trial) the user clicks one of the images to select it as an answer. Once main narration audio has been played and either a selection has been made (for a choice trial, isChoiceFrame: true) or all required character audio has been played (for a non-choice trial), the user can proceed by pressing 'next'. (A trial with only main narration audio can also simply auto-proceed when audio is finished.)

Recording is optional. If webcam recording is conducted (doRecording: true) then audio does not start until recording does, to ensure the entire trial is recorded.

The character images are specified in 'images', including an image source, positioning on the screen, any animation at the start of the trial, any associated audio, and whether that audio is required.

This frame extends ExpFrameBaseUnsafe because it is displayed fullscreen and expected to be repeated.

The examples below show a few expected uses of this frame. In phase-2, two characters are shown; the protagonist is already present, and speaker1 flies in from the left. Speaker1 has associated audio (dialogue). After the narrative audio, the user can click on speaker1 to play the audio, and will then be able to proceed.

phase-5 is a choice trial, where the user has to click on one of the two images before proceeding.

"frames":
       "phase-2": {
           "kind": "exp-lookit-dialogue-page",
           "id": "phase-2",
           "baseDir": "https://s3.amazonaws.com/lookitcontents/politeness/",
           "audioTypes": ["mp3", "ogg"],
           "backgroundImage": "order1_test1_background.png",
           "doRecording": false,
           "autoProceed": false,
           "parentTextBlock": {
               "title": "Parents!",
               "text": "some instructions",
               "emph": true
           },
           "images": [
               {
                   "id": "protagonist",
                   "src": "order1_test1_listener1.png",
                   "left": "40",
                   "bottom": "2",
                   "height": "60"
               },
               {
                   "id": "speaker1",
                   "text": "Click to hear what he said!",
                   "src": "order1_test1_speaker1.png",
                   "left": "20",
                   "bottom": "2",
                   "height": "60",
                   "animate": "flyleft",
                   "requireAudio": true,
                   "imageAudio": [
                       {
                           "stub": "polcon_example_2_2speaker1polite"
                       }
                   ]
               }
           ],
           "audioSources": [
               {
                   "audioId": "firstAudio",
                   "sources": [
                       {
                           "stub": "polcon_example_2_1intro"
                       }
                   ]
               }
           ]
       },
       "phase-5": {
           "kind": "exp-lookit-dialogue-page",
           "id": "phase-5",
           "baseDir": "https://s3.amazonaws.com/lookitcontents/politeness/",
           "audioTypes": ["mp3", "ogg"],
           "backgroundImage": "order1_test1_background.png",
           "doRecording": false,
           "autoProceed": false,
           "isChoiceFrame": true,
           "parentTextBlock": {
               "title": "Parents:",
               "text": "click on the character your child selects.",
               "emph": true
           },
           "images": [
               {
                   "id": "speaker1",
                   "src": "order1_test1_speaker1.png",
                   "left": "20",
                   "bottom": "2",
                   "height": "60"
               },
               {
                   "id": "speaker2",
                   "src": "order1_test1_speaker2.png",
                   "left": "60",
                   "bottom": "2",
                   "height": "60"
               }
           ],
           "audioSources": [
               {
                   "audioId": "firstAudio",
                   "sources": [
                       {
                           "stub": "polcon_example_5q1"
                       }
                   ]
               }
           ]
       }
}

Methods

exitFullscreen

()
Exit fullscreen mode

serializeContent

(
  • videoID
  • eventTimings
  • currentlyHighlighted
)
Object

Parameters captured and sent to the server

Parameters:

  • videoID String

    The ID of any video recorded during this frame

  • eventTimings Object
  • currentlyHighlighted String

    which image is selected at the end of the trial, or null if none is. This indicates the final selected choice for a choice trial.

Returns:

Object:

The payload sent to the server

showFullscreen

()
Make a specified element fullscreen

Properties

audioSources

Object[]

Array of objects describing audio to play at the start of this frame. Each element describes a separate audio segment.

Sub-properties:

  • audioId String

    unique string identifying this audio segment

  • sources Object[]

    Array of {src: 'url', type: 'MIMEtype'} objects with audio sources for this segment.

    Can also give a single element {stub: 'filename'}, which will be expanded out to the appropriate array based on baseDir and audioTypes values; see audioTypes.

  • highlights Object[]

    Array of {'range': [startT, endT], 'image': 'imageId'} objects, where the imageId values correspond to the ids given in images

audioTypes

String[]

List of audio types to expect for any audio specified just with a string rather than with a list of src/type pairs. If audioTypes is ['typeA', 'typeB'] and an audio source is given as [{'stub': 'intro'}], the audio source will be expanded out to

                [
                       {
                           src: 'baseDir' + 'typeA/intro.typeA',
                           type: 'audio/typeA'
                       },
                       {
                           src: 'baseDir' + 'typeB/intro.typeB',
                           type: 'audio/typeB'
                       }
               ]

Default: ['mp3', 'ogg']

autoProceed

Boolean

Whether to proceed automatically after audio (and hide replay/next buttons)

backgroundImage

String

URL of background image; will be stretched to width of page

baseDir

String

Base directory for where to find stimuli. Any image src values that are not full paths will be expanded by prefixing with baseDir + img/. Any audio/video src values that give a value for 'stub' rather than 'src' and 'type' will be expanded out to baseDir/avtype/[stub].avtype, where the potential avtypes are given by audioTypes and videoTypes.

Note that baseDir SHOULD include a trailing slash (e.g., http://stimuli.org/myexperiment/, not http://stimuli.org/myexperiment)

Default: ''

doRecording

Boolean

Whether to do webcam recording (will wait for webcam connection before starting audio if so)

fsButtonID

String
The element ID of a button to show if the user leaves fullscreen mode

fullScreenElementId

String
The element ID of the thing to make full screen (video element, div, etc)

images

Object[]

Array of images to display and information about their placement

Sub-properties:

  • id String

    unique ID for this image. This will be used to refer to the choice made by the user, if any.

  • src String

    URL of image source (can be full URL, or stub to append to baseDir; see baseDir)

  • left String

    left margin, as percentage of story area width

  • height String

    image height, as percentage of story area height

  • bottom String

    bottom margin, as percentage of story area height

  • animate String

    animation to use at start of trial on this image, if any. If not provided, image is shown throughout trial. Options are 'fadein', 'fadeout', 'flyleft' (fly from left), and 'flyright'.

  • text String

    text to display above image, e.g. 'Click to hear what he said!' If omitted, no text is shown.

  • imageAudio Object[]

    sources Array of {src: 'url', type: 'MIMEtype'} objects with audio sources for audio to play when this image is clicked, if any. (Omit to not associate audio with this image.)

    Can also give a single element {stub: 'filename'}, which will be expanded out to the appropriate array based on baseDir and audioTypes values; see audioTypes.

  • requireAudio Boolean

    whether to require the user to click this image and complete the audio associated before proceeding to the next trial. (Incompatible with autoProceed.)

isChoiceFrame

Boolean

Whether this is a frame where the user needs to click to select one of the images before proceeding

Default: false

nPhase

Number

Phase number (just included as a convenience & sent to server, to make handling collected data simpler)

Default: 0

nTrial

Number

Trial number (just included as a convenience & sent to server, to make handling collected data simpler)

Default: 0

parentTextBlock

Object

Text block to display to parent.

Sub-properties:

  • title String

    title to display

  • text String

    paragraph of text

  • emph Boolean

    whether to bold this paragraph

Events

clickSpeaker

When one of the images is clicked during a choice frame

Event Payload:

completeMainAudio

When narration audio is completed

completeSpeakerAudio

When image audio is completed (not recorded if interrupted)

Event Payload:

enteredFullscreen

Upon detecting change to fullscreen mode

leftFullscreen

Upon detecting change out of fullscreen mode

nextFrame

Move to next frame

onCamAccess

When recorder detects a change in camera access

Event Payload:

startSpeakerAudio

When image audio is started

Event Payload:

stoppingCapture

Just before stopping webcam video capture

videoStreamConnection

When recorder detects a change in video stream connection status

Event Payload:

  • status String

    status of video stream connection, e.g. 'NetConnection.Connect.Success' if successful