TheBigExperiment

Examining the tapes

• All digitized videos are now on one external hard drive - the new "My Book" that is now in the VPL. However, the videos were not deleted from the other external hard drives yet....just in case.
  • There are several iMovie files with TBE participant recordings - I'm not sure of total number of participants in the study
• 82 images were shown to each participant in the first part, where they just had to examine the pictures
  • these need to be compared with all the images shown below

• TBEimagestuff.xls: excel sheet with categories and names
• TBE_All_Images_Classified.xls: ALL the images categorized

Description of TBE from Jeff

Each S (we probably have 15) looked at 82 images on the plasma display "before the experiment began." They were told that they were looking at examples of the kind of images they would be seeing in the experiments.

The free view period was self paced, but people took ~10 minutes to look at the 82 images. We can decide how many of the fixations to look at on each. There was no initial fixation point, so the "first fixation" is the first one after the image changed. I think there's enough to be learned by looking at the first 3-5 fixations on each - both for agreement between observers and for the target itself.

Observer data:

• tbe_observer_data.mat: tbe_observer_data.mat

The .mat file has observer/experiment data for 10.5 observers (10 free-view & multitask, 1 free-view only) in a cell array. Load the data into the Matlab workspace with the 'load' command and show the structure with the 'whos' command:

>> load tbe_observer_data.mat

>> whos
Name Size Bytes Class Attributes

tbe_observer_data 1x21 279254 cell
---------------------------------------------------------------------------------------------------------

To show information for a single trial, indicate the cell element in {curly brackets}. For example, to show the first element:

this_observer = tbe_observer_data{1}

this_observer =

init

'CAR'
age: '33'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '06_Nov_2002_16_12_44'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]
---------------------------------------------------------------------------------------------------------

The .freeview_set field indicates that the observer saw images from the 'freeview_multiview_setA.txt' set (vs. 'freeview_multiview_setB.txt' [ freeview_multiview_setA.txt: freeview_multiview_setB.txt: ]), and the .exp field indicates that this was the free_view set (vs. the multiview):
>> this_observer.freeview_set

ans =
freeview_multiview_setA.txt

>> this_observer.exp

ans =
free_view
---------------------------------------------------------------------------------------------------------

(Note that tbe_observer_data{1}.exp is equivalent to this_observer.exp)

To show the list of images in the order viewed:

>> tbe_observer_data{1}.image_list

ans =

'search_iconic_11_image9_letterR.jpg 22'
'Scaling_DSCN0177.jpg 40'
'search_verbal_14_image18_trashcan.jpg 61'
'Scaling_172022.jpg 29'
'action_bell_plain.jpg 5'
'search_verbal_03_image37_christmastreeshape.jpg 50'
'search_iconic_14_Stockolm.jpg 25'
'search_iconic_03_DSC00002.jpg 14'
'Scaling_215000.jpg 36'
'same_diff_DCP01876.JPG 67'
'Scaling_215075.jpg 43'
'search_iconic_12_P2010123.jpg 23'
'search_verbal_10_215065.jpg 57'
'same_diff_101031_1.jpg 74'
'search_verbal_04_lerner_image_6.jpg 51'
'nat_task_DCSN0252.jpg 2'
'same_diff_Pale_Painted_Lady_no_3.jpg 76'
'same_diff_Eubie_again.jpg 75'
'Scaling_215036.jpg 39'
'search_verbal_08_DSCN0265.jpg 55'
'Scaling_172088.jpg 33'
'action_wheel_action.jpg 88'
'search_iconic_06_image2_greencar.jpg 17'
'action_bell_action.jpg 84'
'same_diff_101016_1.jpg 72'
'search_iconic_01_279067.jpg 12'
'same_diff_DCP01386.JPG 78'
'search_verbal_02_image6_waterfaucet.jpg 49'
'Scaling_215058.jpg 42'
'search_verbal_12_215095.jpg 59'
'action_DSCN0323.jpg 7'
'action_cupboard_action.jpg 6'
'action_DSCN0287.jpg 8'
'action_DSCN0324.jpg 86'
'same_diff_DSCN0309.jpg 79'
'search_verbal_06_P7080413.jpg 53'
'Scaling_172043.jpg 30'
'action_DSCN0284.jpg 87'
'same_diff_DSC00444.JPG 69'
'search_verbal_01_image5_phonesign.jpg 48'
'search_verbal_13_279064.jpg 60'
'Scaling_215004.jpg 37'
'search_verbal_15_image21_letterC.jpg 62'
'same_diff_215039.jpg 64'
'same_diff_Mercedes.jpg 71'
'action_wheel_plain.jpg 9'
'same_diff_P7080417.jpg 68'
'action_lamp_plain.jpg 11'
'search_iconic_16_279029.jpg 27'
'same_diff_DCP01648.JPG 65'
'Scaling_172089.jpg 34'
'search_verbal_07_DSCN0254.jpg 54'
'same_diff_DSC00723.JPG 66'
'Scaling_172003.jpg 28'
'nat_task_DCSN0261.jpg 3'
'search_verbal_09_215062.jpg 56'
'Scaling_172096.jpg 35'
'Scaling_215040.jpg 41'
'action_goggle_plain.jpg 89'
'action_cupboard_plain.jpg 85'
'same_diff_gears_IV.jpg 73'
'same_diff_DCP01641.JPG 70'
'action_goggle_action.jpg 10'
'search_iconic_02_279089.jpg 13'
'search_iconic_09_image4_stool.jpg 20'
'search_iconic_04_applecomp.jpg 15'
'search_iconic_13_P7070402.jpg 24'
'search_verbal_16_image28_clock.jpg 63'
'search_iconic_05_image1_letterB.jpg 16'
'action_lamp_action.jpg 90'
'nat_task_DCSN0274.jpg 4'
'search_verbal_11_215090.jpg 58'
'search_iconic_10_image8_watch.jpg 21'
'search_iconic_07_image20_fedexbox.jpg 18'
'Scaling_172081.jpg 31'
'same_diff_stock_-_technicolor_9.jpg 77'
'Scaling_172083.jpg 32'
'Scaling_215035.jpg 38'
'search_iconic_08_image24_letterC.jpg 19'
'search_verbal_05_DSC00006.jpg 52'
'search_iconic_15_279000.jpg 26'
'nat_task_DCSN0250.jpg 1'
--------------------------------------------------------------------------------------------------------- ######################################################################################
%%
for i = 1:n
display(['tbe_observer_data{', num2str(i), '} = ']);
display(tbe_observer_data{i});
display('=========================================================');
end

=========================================================
tbe_observer_data{1} =
init: 'CAR'
age: '33'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '06_Nov_2002_16_12_44'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{2} =
init: 'CAR'
age: '33'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '06_Nov_2002_16_26_58'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'B'

=========================================================
tbe_observer_data{3} =
init: 'CLS'
age: '22'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '20_Aug_2002_14_11_20'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{4} =
init: 'CLS'
age: '22'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '20_Aug_2002_14_21_39'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'A'

=========================================================
tbe_observer_data{5} =

init: 'EMO'
age: '22'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '20_Aug_2002_15_14_32'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{6} =

init: 'EMO'
age: '22'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '20_Aug_2002_15_27_05'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'B'

=========================================================
tbe_observer_data{7} =

init: 'ET'
age: '19'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '11_Oct_2002_14_35_28'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{8} =

init: 'ET'
age: '19'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '11_Oct_2002_14_53_20'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'A'

=========================================================
tbe_observer_data{9} =

init: 'FL'
age: '18'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '11_Oct_2002_13_42_42'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{10} =
init: 'FL'
age: '18'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '11_Oct_2002_13_54_59'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'B'

=========================================================
tbe_observer_data{11} =

init: 'JLB'
age: '22'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '19_Aug_2002_12_33_14'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{12} =

init: 'JLB'
age: '22'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '19_Aug_2002_12_42_49'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'B'

=========================================================
tbe_observer_data{13} =

init: 'JSK'
age: '24'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '20_Aug_2002_12_13_19'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{14} =

init: 'JSK'
age: '24'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '20_Aug_2002_12_26_10'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'A'

=========================================================
tbe_observer_data{15} =

init: 'JST'
age: '19'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '04_Oct_2002_13_31_16'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{16} =

init: 'JST'
age: '19'
tc: '00:00:00:00'
sex: 'f'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '04_Oct_2002_13_50_11'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'A'

=========================================================
tbe_observer_data{17} =

init: 'MDB'
age: '19'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '21_Aug_2002_11_17_12'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{18} =

init: 'MDB'
age: '19'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '21_Aug_2002_11_36_56'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'B'

=========================================================
tbe_observer_data{19} =

init: 'RCB'
age: '18'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setB.txt'
date: '21_Aug_2002_12_09_59'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=========================================================
tbe_observer_data{20} =

init: 'RCB'
age: '18'
tc: '00:00:00:00'
sex: 'm'
level: 'Naive'
freeview_set: 'freeview_multiview_setA.txt'
date: '21_Aug_2002_12_17_17'
exp: 'multi_view'
random_order: [90x1 double]
image_list: {90x1 cell}
time: [90x1 double]
xy_coordinates: {90x1 cell}
swap: 'A'

=========================================================
tbe_observer_data{21} =

init: 'RE'
age: '45'
tc: '00:00:00:00'
sex: 'm'
level: 'Expert'
freeview_set: 'freeview_multiview_setA.txt'
date: '11_Oct_2002_15_37_43'
exp: 'free_view'
random_order: [82x1 double]
image_list: {82x1 cell}
time: [82x1 double]

=====================================================

-- JeffPelz - 18 Apr 2007

ASL fixation data (Excel files):

The following zip file contains individual .XLS files for each image.

• CindyScigajA3_multi_index_parse.zip: CindyScigajA3^? _multi_index_parse.zip

Individual filenames, such as: MikeBostickB3^? _multi_search_iconic_20_rolls-royces.xls and JaredKayB2^? _free_search_iconic_32_binoculars.xls indicate [observer name], [freeview or multiview ('free'/'multi')], and [image filename (search_iconic_20_rolls-royces & search_iconic_32_binoculars)]. Each Excel file has header information with display/viewing information, the name of the raw ASL ASC data file name (e.g., JoshuaBrownB3.ASC) and the image filename (e.g., action_door_action.jpg), followed by the raw eye and head position data:

display	screenW_pix	screenH_pix	screenW_in	screenH_in	pix_per_inW	pix_per_inH	centerW	centerH	ES_DIS_from_centerscreen	filename	imagename
PioneerPlasma?	1280	768	43	24.25	29.767442	31.670103	641	385	41.349924	JoshuaBrownB3.ASC	action_door_action.jpg

FLD	R	S	E	X	SEG	OT	HR	MN	SEC	TOTAL_SECS	VPOS	HPOS	PDIAM	XD	MK	ES_DIS	SCN	MHTX	MHTY	MHTZ	AZM	ELV	ROLL
47762	0	0	0	0	1	0	12	56	42.883	46602.883	-1.035	2.935	74	189	-1	41.955	0	3232	1556	892	-972	500	248
47763	0	0	0	0	1	0	12	56	42.9	46602.9	-1.32	2.82	74	189	-1	41.935	0	3232	1560	896	-968	500	252
47764	0	0	0	0	1	0	12	56	42.917	46602.917	-0.945	2.81	74	189	-1	41.955	0	3232	1560	896	-964	492	256
47765	0	0	0	0	1	0	12	56	42.933	46602.933	-1.255	2.79	75	189	-1	41.935	0	3232	1560	896	-964	496	256
47766	0	0	0	0	1	0	12	56	42.95	46602.95	-1.035	2.785	74	189	-1	41.96	0	3228	1564	896	-960	492	256
47767	0	0	0	0	1	0	12	56	42.967	46602.967	-1.23	2.82	75	189	-1	41.95	0	3228	1564	896	-956	496	260
47768	0	0	0	0	1	0	12	56	42.983	46602.983	-0.99	2.81	75	189	-1	41.96	0	3228	1564	896	-956	500	260
47769	0	0	0	0	1	0	12	56	43	46603	-1.325	2.79	75	189	-1	41.94	0	3228	1564	896	-952	504	260
47770	0	0	0	0	1	0	12	56	43.017	46603.017	-1.03	2.01	77	189	-1	41.925	0	3228	1568	896	-948	504	268
47771	0	0	0	0	1	0	12	56	43.033	46603.033	-2.205	-0.175	76	189	-1	41.895	0	3224	1568	896	-940	500	276
47772	0	0	0	0	1	0	12	56	43.05	46603.05	-2.475	-1.845	74	189	-1	41.97	0	3224	1568	896	-940	508	272

.
.
.
And the individual fixations and saccades are listed:

begin (ms)	end (ms)	duration (ms)	HPOS	VPOS	HPOS_pixels	VPOS_pixels	HPOS_gaze_deg	VPOS_gaze_deg	delta_H_angle	delta_V_angle	saccade_size	saccade_direction
0	134	134	2.82	-1.14	725	349	3.90	-1.58	0.00	0.00	0.00	0.0
200	434	234	-2.21	-2.67	575	300	-3.06	-3.70	-6.97	-2.12	7.28	-163.1
501	685	184	3.96	-3.62	759	270	5.47	-5.00	8.53	-1.30	8.63	-8.7
752	1002	251	11.73	0.26	990	393	15.84	0.36	10.37	5.37	11.68	27.4
1035	1303	267	9.73	-0.81	931	359	13.24	-1.13	-2.60	-1.49	3.00	-150.2
1353	1587	234	14.87	-3.33	1083	280	19.77	-4.60	6.53	-3.47	7.40	-28.0
1637	2171	534	12.36	1.44	1009	431	16.64	2.00	-3.14	6.60	7.31	115.4
2221	2572	351	15.01	-1.48	1088	338	19.95	-2.06	3.31	-4.05	5.23	-50.7
2822	2939	117	14.18	8.11	1063	642	18.93	11.09	-1.02	13.15	13.19	94.4
2989	3624	635	11.63	1.68	987	438	15.71	2.33	-3.22	-8.76	9.33	-110.2
3691	3941	251	2.22	-0.36	707	373	3.07	-0.50	-12.64	-2.83	12.95	-167.4
4025	4208	184	-12.48	1.69	270	439	-16.79	2.34	-19.86	2.85	20.07	171.8
4275	4476	200	-14.12	5.65	221	564	-18.85	7.79	-2.06	5.44	5.82	110.7
4542	4643	100	-5.78	4.82	469	538	-7.95	6.65	10.90	-1.14	10.96	-6.0
4743	4876	134	-19.68	-2.66	55	301	-25.45	-3.68	-17.50	-10.33	20.32	-149.5
4927	5010	84	-20.10	-3.50	43	274	-25.93	-4.84	-0.47	-1.16	1.25	-112.3
5094	5344	251	-10.82	-3.95	319	260	-14.66	-5.46	11.27	-0.62	11.28	-3.1
5428	5628	200	-19.21	-8.30	69	122	-24.92	-11.35	-10.26	-5.90	11.84	-150.1
5661	5845	184	-18.37	-6.23	94	188	-23.95	-8.57	0.97	2.78	2.95	70.8
5945	6446	501	5.83	-7.72	814	141	8.02	-10.57	31.98	-2.00	32.04	-3.6
6530	6663	134	-6.73	-5.19	441	221	-9.25	-7.16	-17.27	3.41	17.61	168.8
6713	6847	134	-3.18	-5.77	546	202	-4.40	-7.94	4.85	-0.79	4.91	-9.2
6931	7181	251	3.34	-3.65	741	270	4.62	-5.04	9.02	2.91	9.48	17.9
7248	7432	184	5.37	-7.84	801	137	7.40	-10.74	2.77	-5.70	6.34	-64.1
7498	7782	284	2.12	-6.57	704	177	2.94	-9.03	-4.46	1.71	4.78	159.1
7882	8150	267	-5.08	6.48	490	590	-7.01	8.90	-9.94	17.94	20.51	119.0
8183	8350	167	-6.82	5.78	438	568	-9.36	7.95	-2.35	-0.95	2.54	-158.1
8417	8868	451	-14.20	4.96	218	542	-18.95	6.83	-9.59	-1.12	9.66	-173.3
9001	9252	251	12.21	3.71	1005	503	16.45	5.13	35.41	-1.70	35.45	-2.8
9319	9886	568	13.28	7.16	1036	612	17.80	9.83	1.35	4.70	4.89	74.0
9970	10170	200	16.66	-0.45	1137	371	21.94	-0.62	4.14	-10.45	11.24	-68.4
10237	10471	234	17.66	-4.61	1167	239	23.13	-6.36	1.19	-5.74	5.87	-78.3
10538	10738	200	17.99	-9.88	1177	72	23.52	-13.44	0.38	-7.08	7.09	-86.9

The following zip file has the raw ASL ASC data files.

• SamanthaSmithB3.zip: SamanthaSmithB3^?.zip

-- JeffPelz - 18 Apr 2007

• • Set ALLOWTOPICCHANGE = VplGroup

-- LeanneStefano - 13 Apr 2007

• TBE_All_Images_Classified.xls: ALL the images categorized