pymovements in 10 minutes#
What you will learn in this tutorial:#
how to download one of the publicly available datasets
how to load a subset of the data into your memory
how to transform pixel coordinates into degrees of visual angle
how to transform positional data into velocity data
how to detect fixations by using the I-VT algorithm
how to detect saccades by using the microsaccades algorithm
how to compute additional event properties for your analysis
how to save your preprocessed data
how to plot the main saccadic sequence from your data
Downloading one of the public datasets#
We import pymovements
as the alias pm
for convenience.
[1]:
import polars as pl
import pymovements as pm
pymovements provides a library of publicly available datasets.
You can browse through the available dataset definitions here: Datasets
For this tutorial we will limit ourselves to the ToyDataset
due to its minimal space requirements.
Other datasets can be downloaded by simply replacing ToyDataset
with one of the other available datasets.
We can initialize and download by passing the desired dataset name as a string argument.
Additionally we need the root directory path of your data.
[2]:
dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')
dataset.download()
INFO:pymovements.dataset.dataset:
You are downloading the pymovements Toy Dataset. Please be aware that pymovements does not
host or distribute any dataset resources and only provides a convenient interface to
download the public dataset resources that were published by their respective authors.
Please cite the referenced publication if you intend to use the dataset in your research.
Downloading http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/ to data/ToyDataset/downloads/pymovements-toy-dataset.zip
Checking integrity of pymovements-toy-dataset.zip
Extracting pymovements-toy-dataset.zip to data/ToyDataset/raw
100%|██████████| 23/23 [00:00<00:00, 305.17it/s]
[2]:
-
DatasetDefinitionDatasetDefinition
-
NoneNone
-
dict (0 items)
-
dict (1 items)
-
dict (4 items)
-
list (5 items)
- 'timestamp'
- 'x'
- (3 more)
-
dict (5 items)
-
Float64Float64
-
Float64Float64
- (3 more)
-
- (2 more)
-
-
-
NoneNone
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
-
NoneNone
-
dict (1 items)
-
'trial_{text_id:d}_{page_id:d}.csv''trial_{text_id:d}_{page_id:d}.csv'
-
-
dict (1 items)
-
dict (2 items)
-
<class 'int'><class 'int'>
-
<class 'int'><class 'int'>
-
-
-
TrueTrue
-
'pymovements Toy Dataset''pymovements Toy Dataset'
-
dict (0 items)
-
'ToyDataset''ToyDataset'
-
list (2 items)
- 'x'
- 'y'
-
NoneNone
-
list (1 items)
-
ResourceDefinition
-
'gaze''gaze'
-
'pymovements-toy-dataset.zip''pymovements-toy-dataset.zip'
-
'trial_{text_id:d}_{page_id:d}.csv''trial_{text_id:d}_{page_id:d}.csv'
-
dict (2 items)
-
<class 'int'><class 'int'>
-
<class 'int'><class 'int'>
-
-
'4da622457637a8181d86601fe17f3aa8''4da622457637a8181d86601fe17f3aa8'
-
str'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
-
-
ResourceDefinition
-
'timestamp''timestamp'
-
'ms''ms'
-
NoneNone
-
NoneNone
-
-
list (0 items)
-
DataFrame (0 columns, 0 rows)shape: (0, 0)
-
list (0 items)
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
DatasetPathsDatasetPaths
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
PosixPath('data/ToyDataset/downloads')PosixPath('data/ToyDataset/downloads')
-
PosixPath('data/ToyDataset/events')PosixPath('data/ToyDataset/events')
-
PosixPath('data/ToyDataset/precomputed_events')PosixPath('data/ToyDataset/precomputed_events')
-
PosixPathPosixPath('data/ToyDataset/precomputed_reading_measures')
-
PosixPath('data/ToyDataset/preprocessed')PosixPath('data/ToyDataset/preprocessed')
-
PosixPath('data/ToyDataset/raw')PosixPath('data/ToyDataset/raw')
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
-
list (0 items)
-
list (0 items)
Our downloaded dataset will be placed in new a directory with the name of the dataset:
[3]:
dataset.path
[3]:
PosixPath('data/ToyDataset')
Archive files are automatically extracted into the path specified by Dataset.paths.raw
:
[4]:
dataset.paths.raw
[4]:
PosixPath('data/ToyDataset/raw')
Loading in your data into memory#
Next we load our dataset into memory to be able to work with it:
[5]:
dataset.load()
[5]:
-
DatasetDefinitionDatasetDefinition
-
NoneNone
-
dict (0 items)
-
dict (1 items)
-
dict (4 items)
-
list (5 items)
- 'timestamp'
- 'x'
- (3 more)
-
dict (5 items)
-
Float64Float64
-
Float64Float64
- (3 more)
-
- (2 more)
-
-
-
NoneNone
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
-
NoneNone
-
dict (1 items)
-
'trial_{text_id:d}_{page_id:d}.csv''trial_{text_id:d}_{page_id:d}.csv'
-
-
dict (1 items)
-
dict (2 items)
-
<class 'int'><class 'int'>
-
<class 'int'><class 'int'>
-
-
-
TrueTrue
-
'pymovements Toy Dataset''pymovements Toy Dataset'
-
dict (0 items)
-
'ToyDataset''ToyDataset'
-
list (2 items)
- 'x'
- 'y'
-
NoneNone
-
list (1 items)
-
ResourceDefinition
-
'gaze''gaze'
-
'pymovements-toy-dataset.zip''pymovements-toy-dataset.zip'
-
'trial_{text_id:d}_{page_id:d}.csv''trial_{text_id:d}_{page_id:d}.csv'
-
dict (2 items)
-
<class 'int'><class 'int'>
-
<class 'int'><class 'int'>
-
-
'4da622457637a8181d86601fe17f3aa8''4da622457637a8181d86601fe17f3aa8'
-
str'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
-
-
ResourceDefinition
-
'timestamp''timestamp'
-
'ms''ms'
-
NoneNone
-
NoneNone
-
-
list (0 items)
-
dict (1 items)
-
DataFrame (3 columns, 20 rows)shape: (20, 3)
text_id page_id filepath i64 i64 str 0 1 "aeye-lab-pymovements-toy-datas… 0 2 "aeye-lab-pymovements-toy-datas… 0 3 "aeye-lab-pymovements-toy-datas… 0 4 "aeye-lab-pymovements-toy-datas… 0 5 "aeye-lab-pymovements-toy-datas… … … … 3 1 "aeye-lab-pymovements-toy-datas… 3 2 "aeye-lab-pymovements-toy-datas… 3 3 "aeye-lab-pymovements-toy-datas… 3 4 "aeye-lab-pymovements-toy-datas… 3 5 "aeye-lab-pymovements-toy-datas…
-
-
list (20 items)
-
Gaze
-
DataFrame (6 columns, 17223 rows)shape: (17_223, 6)
time stimuli_x stimuli_y text_id page_id pixel i64 f64 f64 i64 i64 list[f64] 1988145 -1.0 -1.0 0 1 [206.8, 152.4] 1988146 -1.0 -1.0 0 1 [206.9, 152.1] 1988147 -1.0 -1.0 0 1 [207.0, 151.8] 1988148 -1.0 -1.0 0 1 [207.1, 151.7] 1988149 -1.0 -1.0 0 1 [207.0, 151.5] … … … … … … 2005363 -1.0 -1.0 0 1 [361.0, 415.4] 2005364 -1.0 -1.0 0 1 [358.0, 414.5] 2005365 -1.0 -1.0 0 1 [355.8, 413.8] 2005366 -1.0 -1.0 0 1 [353.1, 413.2] 2005367 -1.0 -1.0 0 1 [351.2, 412.9] -
EventsEvents
-
DataFrame (6 columns, 0 rows)shape: (0, 6)
text_id page_id name onset offset duration i64 i64 str i64 i64 i64 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
-
-
Gaze
-
DataFrame (6 columns, 29799 rows)shape: (29_799, 6)
time stimuli_x stimuli_y text_id page_id pixel i64 f64 f64 i64 i64 list[f64] 2008305 -1.0 -1.0 0 2 [141.4, 153.6] 2008306 -1.0 -1.0 0 2 [141.1, 153.2] 2008307 -1.0 -1.0 0 2 [140.7, 152.8] 2008308 -1.0 -1.0 0 2 [140.6, 152.7] 2008309 -1.0 -1.0 0 2 [140.5, 152.6] … … … … … … 2038099 -1.0 -1.0 0 2 [273.8, 773.8] 2038100 -1.0 -1.0 0 2 [273.8, 774.1] 2038101 -1.0 -1.0 0 2 [273.9, 774.5] 2038102 -1.0 -1.0 0 2 [274.0, 774.4] 2038103 -1.0 -1.0 0 2 [274.0, 773.9] -
EventsEvents
-
DataFrame (6 columns, 0 rows)shape: (0, 6)
text_id page_id name onset offset duration i64 i64 str i64 i64 i64 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
-
- (18 more)
-
Gaze
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
DatasetPathsDatasetPaths
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
PosixPath('data/ToyDataset/downloads')PosixPath('data/ToyDataset/downloads')
-
PosixPath('data/ToyDataset/events')PosixPath('data/ToyDataset/events')
-
PosixPath('data/ToyDataset/precomputed_events')PosixPath('data/ToyDataset/precomputed_events')
-
PosixPathPosixPath('data/ToyDataset/precomputed_reading_measures')
-
PosixPath('data/ToyDataset/preprocessed')PosixPath('data/ToyDataset/preprocessed')
-
PosixPath('data/ToyDataset/raw')PosixPath('data/ToyDataset/raw')
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
-
list (0 items)
-
list (0 items)
This way we fill two attributes with data. First we have the fileinfo
attribute which holds all the basic information for files:
[6]:
dataset.fileinfo['gaze'].head()
[6]:
text_id | page_id | filepath |
---|---|---|
i64 | i64 | str |
0 | 1 | "aeye-lab-pymovements-toy-datas… |
0 | 2 | "aeye-lab-pymovements-toy-datas… |
0 | 3 | "aeye-lab-pymovements-toy-datas… |
0 | 4 | "aeye-lab-pymovements-toy-datas… |
0 | 5 | "aeye-lab-pymovements-toy-datas… |
We notice that for each filepath a text_id
and page_id
is specified.
We have also loaded our gaze data into the dataframes in the gaze
attribute:
[7]:
dataset.gaze[0]
[7]:
-
DataFrame (6 columns, 17223 rows)shape: (17_223, 6)
time stimuli_x stimuli_y text_id page_id pixel i64 f64 f64 i64 i64 list[f64] 1988145 -1.0 -1.0 0 1 [206.8, 152.4] 1988146 -1.0 -1.0 0 1 [206.9, 152.1] 1988147 -1.0 -1.0 0 1 [207.0, 151.8] 1988148 -1.0 -1.0 0 1 [207.1, 151.7] 1988149 -1.0 -1.0 0 1 [207.0, 151.5] … … … … … … 2005363 -1.0 -1.0 0 1 [361.0, 415.4] 2005364 -1.0 -1.0 0 1 [358.0, 414.5] 2005365 -1.0 -1.0 0 1 [355.8, 413.8] 2005366 -1.0 -1.0 0 1 [353.1, 413.2] 2005367 -1.0 -1.0 0 1 [351.2, 412.9] -
EventsEvents
-
DataFrame (6 columns, 0 rows)shape: (0, 6)
text_id page_id name onset offset duration i64 i64 str i64 i64 i64 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
Apart from some trial identifier columns we see the columns time
and pixel
.
The last two columns refer to the pixel coordinates at the timestep specified by time
.
We are also able to just take a subset of the data by specifying values of the fileinfo columns. The key refers to the column in the fileinfo
dataframe. The values in the dictionary can be of type bool
, int
, float
or str
, but also lists and ranges
[8]:
subset = {
'text_id': 0,
'page_id': [0, 1],
}
dataset.load(subset=subset)
dataset.fileinfo
[8]:
{'gaze': shape: (1, 3)
┌─────────┬─────────┬─────────────────────────────────┐
│ text_id ┆ page_id ┆ filepath │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ str │
╞═════════╪═════════╪═════════════════════════════════╡
│ 0 ┆ 1 ┆ aeye-lab-pymovements-toy-datas… │
└─────────┴─────────┴─────────────────────────────────┘}
Now we selected only a small subset of our data.
Preprocessing raw gaze data#
We now want to preprocess our gaze data by transforming pixel coordinates into degrees of visual angle and then computing velocity data from our positional data.
[9]:
dataset.pix2deg()
dataset.gaze[0]
[9]:
-
DataFrame (7 columns, 17223 rows)shape: (17_223, 7)
time stimuli_x stimuli_y text_id page_id pixel position i64 f64 f64 i64 i64 list[f64] list[f64] 1988145 -1.0 -1.0 0 1 [206.8, 152.4] [-10.697598, -8.852399] 1988146 -1.0 -1.0 0 1 [206.9, 152.1] [-10.695183, -8.859678] 1988147 -1.0 -1.0 0 1 [207.0, 151.8] [-10.692768, -8.866956] 1988148 -1.0 -1.0 0 1 [207.1, 151.7] [-10.690352, -8.869381] 1988149 -1.0 -1.0 0 1 [207.0, 151.5] [-10.692768, -8.874233] … … … … … … … 2005363 -1.0 -1.0 0 1 [361.0, 415.4] [-6.932438, -2.386672] 2005364 -1.0 -1.0 0 1 [358.0, 414.5] [-7.006376, -2.408998] 2005365 -1.0 -1.0 0 1 [355.8, 413.8] [-7.060582, -2.426362] 2005366 -1.0 -1.0 0 1 [353.1, 413.2] [-7.12709, -2.441245] 2005367 -1.0 -1.0 0 1 [351.2, 412.9] [-7.173881, -2.448686] -
EventsEvents
-
DataFrame (6 columns, 0 rows)shape: (0, 6)
text_id page_id name onset offset duration i64 i64 str i64 i64 i64 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
We notice that a new column has appeared: position
. This column specifies the position coordinates in degrees of visual angle (dva).
For transforming our positional data into velocity data we will use the Savitzky-Golay differentiation filter.
We can also specify some additional parameters for this method:
[10]:
dataset.pos2vel(method='savitzky_golay', degree=2, window_length=7)
dataset.gaze[0]
[10]:
-
DataFrame (8 columns, 17223 rows)shape: (17_223, 8)
time stimuli_x stimuli_y text_id page_id pixel position velocity i64 f64 f64 i64 i64 list[f64] list[f64] list[f64] 1988145 -1.0 -1.0 0 1 [206.8, 152.4] [-10.697598, -8.852399] [1.207641, -3.119165] 1988146 -1.0 -1.0 0 1 [206.9, 152.1] [-10.695183, -8.859678] [1.20764, -4.072198] 1988147 -1.0 -1.0 0 1 [207.0, 151.8] [-10.692768, -8.866956] [1.035119, -4.765267] 1988148 -1.0 -1.0 0 1 [207.1, 151.7] [-10.690352, -8.869381] [1.207654, -4.245382] 1988149 -1.0 -1.0 0 1 [207.0, 151.5] [-10.692768, -8.874233] [1.552735, -2.339263] … … … … … … … … 2005363 -1.0 -1.0 0 1 [361.0, 415.4] [-6.932438, -2.386672] [-62.062479, -20.465552] 2005364 -1.0 -1.0 0 1 [358.0, 414.5] [-7.006376, -2.408998] [-61.343786, -18.073031] 2005365 -1.0 -1.0 0 1 [355.8, 413.8] [-7.060582, -2.426362] [-53.501231, -14.617634] 2005366 -1.0 -1.0 0 1 [353.1, 413.2] [-7.12709, -2.441245] [-41.879965, -10.276475] 2005367 -1.0 -1.0 0 1 [351.2, 412.9] [-7.173881, -2.448686] [-27.710881, -6.112645] -
EventsEvents
-
DataFrame (6 columns, 0 rows)shape: (0, 6)
text_id page_id name onset offset duration i64 i64 str i64 i64 i64 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
There is also the more general apply() method, which can be used to apply both transformation and event detection methods.
[11]:
dataset.apply('pos2acc', degree=2, window_length=7)
dataset.gaze[0]
[11]:
-
DataFrame (9 columns, 17223 rows)shape: (17_223, 9)
time stimuli_x stimuli_y text_id page_id pixel position velocity acceleration i64 f64 f64 i64 i64 list[f64] list[f64] list[f64] list[f64] 1988145 -1.0 -1.0 0 1 [206.8, 152.4] [-10.697598, -8.852399] [1.207641, -3.119165] [690.085837, -1501.799767] 1988146 -1.0 -1.0 0 1 [206.9, 152.1] [-10.695183, -8.859678] [1.20764, -4.072198] [0.001831, -866.371365] 1988147 -1.0 -1.0 0 1 [207.0, 151.8] [-10.692768, -8.866956] [1.035119, -4.765267] [-575.06741, -57.655244] 1988148 -1.0 -1.0 0 1 [207.1, 151.7] [-10.690352, -8.869381] [1.207654, -4.245382] [-230.013049, 1328.57081] 1988149 -1.0 -1.0 0 1 [207.0, 151.5] [-10.692768, -8.874233] [1.552735, -2.339263] [690.12611, 2021.586565] … … … … … … … … … 2005363 -1.0 -1.0 0 1 [361.0, 415.4] [-6.932438, -2.386672] [-62.062479, -20.465552] [-1099.087619, 1477.17518] 2005364 -1.0 -1.0 0 1 [358.0, 414.5] [-7.006376, -2.408998] [-61.343786, -18.073031] [1834.348384, 2599.156806] 2005365 -1.0 -1.0 0 1 [355.8, 413.8] [-7.060582, -2.426362] [-53.501231, -14.617634] [9396.15507, 4547.960553] 2005366 -1.0 -1.0 0 1 [353.1, 413.2] [-7.12709, -2.441245] [-41.879965, -10.276475] [16194.183852, 5079.286997] 2005367 -1.0 -1.0 0 1 [351.2, 412.9] [-7.173881, -2.448686] [-27.710881, -6.112645] [16598.914618, 4193.246498] -
EventsEvents
-
DataFrame (6 columns, 0 rows)shape: (0, 6)
text_id page_id name onset offset duration i64 i64 str i64 i64 i64 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
Detecting events#
Now let’s detect some events.
First we will detect fixations using the I-VT algorithm using its default parameters:
[12]:
dataset.detect_events('ivt')
dataset.events[0]
[12]:
-
DataFrame (6 columns, 72 rows)shape: (72, 6)
text_id page_id name onset offset duration i64 i64 str i64 i64 i64 0 1 "fixation" 1988145 1988322 177 0 1 "fixation" 1988351 1988546 195 0 1 "fixation" 1988592 1988736 144 0 1 "fixation" 1988788 1989012 224 0 1 "fixation" 1989044 1989170 126 … … … … … … 0 1 "fixation" 2004114 2004349 235 0 1 "fixation" 2004399 2004687 288 0 1 "fixation" 2004714 2004878 164 0 1 "fixation" 2004930 2005109 179 0 1 "fixation" 2005138 2005286 148 -
list (2 items)
- 'text_id'
- 'page_id'
Next we detect some saccades. This time we don’t use the default parameters but specify our own:
[13]:
dataset.detect_events('microsaccades', minimum_duration=8)
dataset.events[0].frame.filter(pl.col('name') == 'saccade').head()
[13]:
text_id | page_id | name | onset | offset | duration |
---|---|---|---|---|---|
i64 | i64 | str | i64 | i64 | i64 |
0 | 1 | "saccade" | 1988323 | 1988337 | 14 |
0 | 1 | "saccade" | 1988341 | 1988351 | 10 |
0 | 1 | "saccade" | 1988546 | 1988567 | 21 |
0 | 1 | "saccade" | 1988570 | 1988583 | 13 |
0 | 1 | "saccade" | 1988736 | 1988760 | 24 |
We can also use the more general interface of the apply() method:
[14]:
dataset.apply('idt', dispersion_threshold=2.7, name='fixation.ivt')
dataset.events[0].frame.filter(pl.col('name') == 'fixation.ivt').head()
[14]:
text_id | page_id | name | onset | offset | duration |
---|---|---|---|---|---|
i64 | i64 | str | i64 | i64 | i64 |
0 | 1 | "fixation.ivt" | 1988145 | 1988563 | 418 |
0 | 1 | "fixation.ivt" | 1988564 | 1988750 | 186 |
0 | 1 | "fixation.ivt" | 1988751 | 1989178 | 427 |
0 | 1 | "fixation.ivt" | 1989179 | 1989436 | 257 |
0 | 1 | "fixation.ivt" | 1989437 | 1989600 | 163 |
Computing event properties#
The event dataframe currently only holds the name
, onset
, offset
and duration
of an event (additionally we have some more identifier columns at the beginning).
We now want to compute some additional properties for each event. Event properties are things like peak velocity, amplitude and dispersion during an event.
We start out with computing the dispersion:
[15]:
dataset.compute_event_properties("dispersion")
dataset.events[0]
[15]:
-
DataFrame (7 columns, 264 rows)shape: (264, 7)
text_id page_id name onset offset duration dispersion i64 i64 str i64 i64 i64 f64 0 1 "fixation" 1988145 1988322 177 0.154958 0 1 "fixation" 1988351 1988546 195 0.291833 0 1 "fixation" 1988592 1988736 144 0.296297 0 1 "fixation" 1988788 1989012 224 0.271854 0 1 "fixation" 1989044 1989170 126 0.349 … … … … … … … 0 1 "fixation.ivt" 2003929 2004090 161 2.814851 0 1 "fixation.ivt" 2004091 2004363 272 2.819008 0 1 "fixation.ivt" 2004364 2004883 519 2.768099 0 1 "fixation.ivt" 2004885 2005116 231 2.805674 0 1 "fixation.ivt" 2005117 2005298 181 2.744494 -
list (2 items)
- 'text_id'
- 'page_id'
We notice that a new column with the name dispersion
has appeared in the event dataframe.
We can also pass a list of properties to compute all of our desired properties in a single run. Let’s add the amplitude and peak velocity:
[16]:
dataset.compute_event_properties(["amplitude", "peak_velocity"])
dataset.events[0]
[16]:
-
DataFrame (9 columns, 264 rows)shape: (264, 9)
text_id page_id name onset offset duration dispersion amplitude peak_velocity i64 i64 str i64 i64 i64 f64 f64 f64 0 1 "fixation" 1988145 1988322 177 0.154958 0.110074 16.24151 0 1 "fixation" 1988351 1988546 195 0.291833 0.206397 18.88542 0 1 "fixation" 1988592 1988736 144 0.296297 0.209546 17.690373 0 1 "fixation" 1988788 1989012 224 0.271854 0.192719 19.130211 0 1 "fixation" 1989044 1989170 126 0.349 0.304362 18.616167 … … … … … … … … … 0 1 "fixation.ivt" 2003929 2004090 161 2.814851 2.527788 212.117446 0 1 "fixation.ivt" 2004091 2004363 272 2.819008 2.518967 244.333244 0 1 "fixation.ivt" 2004364 2004883 519 2.768099 2.46208 194.527643 0 1 "fixation.ivt" 2004885 2005116 231 2.805674 2.507902 203.067333 0 1 "fixation.ivt" 2005117 2005298 181 2.744494 2.578767 329.741947 -
list (2 items)
- 'text_id'
- 'page_id'
Plotting our data#
pymovements provides a range of plotting functions.
You can browse through the available plotting functions here: Plotting
In this this tutorial we will plot the saccadic main sequence of our data.
[17]:
pm.plotting.main_sequence_plot(dataset.events[0])

Saving and loading your dataframes#
If we want to save interim results we can simply use the save()
method like this:
[18]:
dataset.save()
[18]:
-
DatasetDefinitionDatasetDefinition
-
NoneNone
-
dict (0 items)
-
dict (1 items)
-
dict (4 items)
-
list (5 items)
- 'timestamp'
- 'x'
- (3 more)
-
dict (5 items)
-
Float64Float64
-
Float64Float64
- (3 more)
-
- (2 more)
-
-
-
NoneNone
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
-
NoneNone
-
dict (1 items)
-
'trial_{text_id:d}_{page_id:d}.csv''trial_{text_id:d}_{page_id:d}.csv'
-
-
dict (1 items)
-
dict (2 items)
-
<class 'int'><class 'int'>
-
<class 'int'><class 'int'>
-
-
-
TrueTrue
-
'pymovements Toy Dataset''pymovements Toy Dataset'
-
dict (0 items)
-
'ToyDataset''ToyDataset'
-
list (2 items)
- 'x'
- 'y'
-
NoneNone
-
list (1 items)
-
ResourceDefinition
-
'gaze''gaze'
-
'pymovements-toy-dataset.zip''pymovements-toy-dataset.zip'
-
'trial_{text_id:d}_{page_id:d}.csv''trial_{text_id:d}_{page_id:d}.csv'
-
dict (2 items)
-
<class 'int'><class 'int'>
-
<class 'int'><class 'int'>
-
-
'4da622457637a8181d86601fe17f3aa8''4da622457637a8181d86601fe17f3aa8'
-
str'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
-
-
ResourceDefinition
-
'timestamp''timestamp'
-
'ms''ms'
-
NoneNone
-
NoneNone
-
-
list (1 items)
-
Events
-
DataFrame (9 columns, 264 rows)shape: (264, 9)
text_id page_id name onset offset duration dispersion amplitude peak_velocity i64 i64 str i64 i64 i64 f64 f64 f64 0 1 "fixation" 1988145 1988322 177 0.154958 0.110074 16.24151 0 1 "fixation" 1988351 1988546 195 0.291833 0.206397 18.88542 0 1 "fixation" 1988592 1988736 144 0.296297 0.209546 17.690373 0 1 "fixation" 1988788 1989012 224 0.271854 0.192719 19.130211 0 1 "fixation" 1989044 1989170 126 0.349 0.304362 18.616167 … … … … … … … … … 0 1 "fixation.ivt" 2003929 2004090 161 2.814851 2.527788 212.117446 0 1 "fixation.ivt" 2004091 2004363 272 2.819008 2.518967 244.333244 0 1 "fixation.ivt" 2004364 2004883 519 2.768099 2.46208 194.527643 0 1 "fixation.ivt" 2004885 2005116 231 2.805674 2.507902 203.067333 0 1 "fixation.ivt" 2005117 2005298 181 2.744494 2.578767 329.741947 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
Events
-
dict (1 items)
-
DataFrame (3 columns, 1 rows)shape: (1, 3)
text_id page_id filepath i64 i64 str 0 1 "aeye-lab-pymovements-toy-datas…
-
-
list (1 items)
-
Gaze
-
DataFrame (9 columns, 17223 rows)shape: (17_223, 9)
time stimuli_x stimuli_y text_id page_id pixel position velocity acceleration i64 f64 f64 i64 i64 list[f64] list[f64] list[f64] list[f64] 1988145 -1.0 -1.0 0 1 [206.8, 152.4] [-10.697598, -8.852399] [1.207641, -3.119165] [690.085837, -1501.799767] 1988146 -1.0 -1.0 0 1 [206.9, 152.1] [-10.695183, -8.859678] [1.20764, -4.072198] [0.001831, -866.371365] 1988147 -1.0 -1.0 0 1 [207.0, 151.8] [-10.692768, -8.866956] [1.035119, -4.765267] [-575.06741, -57.655244] 1988148 -1.0 -1.0 0 1 [207.1, 151.7] [-10.690352, -8.869381] [1.207654, -4.245382] [-230.013049, 1328.57081] 1988149 -1.0 -1.0 0 1 [207.0, 151.5] [-10.692768, -8.874233] [1.552735, -2.339263] [690.12611, 2021.586565] … … … … … … … … … 2005363 -1.0 -1.0 0 1 [361.0, 415.4] [-6.932438, -2.386672] [-62.062479, -20.465552] [-1099.087619, 1477.17518] 2005364 -1.0 -1.0 0 1 [358.0, 414.5] [-7.006376, -2.408998] [-61.343786, -18.073031] [1834.348384, 2599.156806] 2005365 -1.0 -1.0 0 1 [355.8, 413.8] [-7.060582, -2.426362] [-53.501231, -14.617634] [9396.15507, 4547.960553] 2005366 -1.0 -1.0 0 1 [353.1, 413.2] [-7.12709, -2.441245] [-41.879965, -10.276475] [16194.183852, 5079.286997] 2005367 -1.0 -1.0 0 1 [351.2, 412.9] [-7.173881, -2.448686] [-27.710881, -6.112645] [16598.914618, 4193.246498] -
EventsEvents
-
DataFrame (9 columns, 264 rows)shape: (264, 9)
text_id page_id name onset offset duration dispersion amplitude peak_velocity i64 i64 str i64 i64 i64 f64 f64 f64 0 1 "fixation" 1988145 1988322 177 0.154958 0.110074 16.24151 0 1 "fixation" 1988351 1988546 195 0.291833 0.206397 18.88542 0 1 "fixation" 1988592 1988736 144 0.296297 0.209546 17.690373 0 1 "fixation" 1988788 1989012 224 0.271854 0.192719 19.130211 0 1 "fixation" 1989044 1989170 126 0.349 0.304362 18.616167 … … … … … … … … … 0 1 "fixation.ivt" 2003929 2004090 161 2.814851 2.527788 212.117446 0 1 "fixation.ivt" 2004091 2004363 272 2.819008 2.518967 244.333244 0 1 "fixation.ivt" 2004364 2004883 519 2.768099 2.46208 194.527643 0 1 "fixation.ivt" 2004885 2005116 231 2.805674 2.507902 203.067333 0 1 "fixation.ivt" 2005117 2005298 181 2.744494 2.578767 329.741947 -
list (2 items)
- 'text_id'
- 'page_id'
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
-
-
Gaze
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
DatasetPathsDatasetPaths
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
PosixPath('data/ToyDataset/downloads')PosixPath('data/ToyDataset/downloads')
-
PosixPath('data/ToyDataset/events')PosixPath('data/ToyDataset/events')
-
PosixPath('data/ToyDataset/precomputed_events')PosixPath('data/ToyDataset/precomputed_events')
-
PosixPathPosixPath('data/ToyDataset/precomputed_reading_measures')
-
PosixPath('data/ToyDataset/preprocessed')PosixPath('data/ToyDataset/preprocessed')
-
PosixPath('data/ToyDataset/raw')PosixPath('data/ToyDataset/raw')
-
PosixPath('data/ToyDataset')PosixPath('data/ToyDataset')
-
-
list (0 items)
-
list (0 items)
Let’s test this out by initializing a new PublicDataset
object in the same directory and loading in the preprocessed gaze and event data.
This time we don’t need to download anything.
[19]:
preprocessed_dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')
dataset.load(events=True, preprocessed=True, subset=subset)
display(dataset.gaze[0])
display(dataset.events[0])
-
DataFrame (9 columns, 17223 rows)shape: (17_223, 9)
time stimuli_x stimuli_y pixel position velocity acceleration text_id page_id i64 f64 f64 list[f64] list[f64] list[f64] list[f64] i64 i64 1988145 -1.0 -1.0 [206.8, 152.4] [-10.697598, -8.852399] [1.207641, -3.119165] [690.085837, -1501.799767] 0 1 1988146 -1.0 -1.0 [206.9, 152.1] [-10.695183, -8.859678] [1.20764, -4.072198] [0.001831, -866.371365] 0 1 1988147 -1.0 -1.0 [207.0, 151.8] [-10.692768, -8.866956] [1.035119, -4.765267] [-575.06741, -57.655244] 0 1 1988148 -1.0 -1.0 [207.1, 151.7] [-10.690352, -8.869381] [1.207654, -4.245382] [-230.013049, 1328.57081] 0 1 1988149 -1.0 -1.0 [207.0, 151.5] [-10.692768, -8.874233] [1.552735, -2.339263] [690.12611, 2021.586565] 0 1 … … … … … … … … … 2005363 -1.0 -1.0 [361.0, 415.4] [-6.932438, -2.386672] [-62.062479, -20.465552] [-1099.087619, 1477.17518] 0 1 2005364 -1.0 -1.0 [358.0, 414.5] [-7.006376, -2.408998] [-61.343786, -18.073031] [1834.348384, 2599.156806] 0 1 2005365 -1.0 -1.0 [355.8, 413.8] [-7.060582, -2.426362] [-53.501231, -14.617634] [9396.15507, 4547.960553] 0 1 2005366 -1.0 -1.0 [353.1, 413.2] [-7.12709, -2.441245] [-41.879965, -10.276475] [16194.183852, 5079.286997] 0 1 2005367 -1.0 -1.0 [351.2, 412.9] [-7.173881, -2.448686] [-27.710881, -6.112645] [16598.914618, 4193.246498] 0 1 -
EventsEvents
-
DataFrame (9 columns, 264 rows)shape: (264, 9)
text_id page_id name onset offset duration dispersion amplitude peak_velocity i64 i64 str i64 i64 i64 f64 f64 f64 0 1 "fixation" 1988145 1988322 177 0.154958 0.110074 16.24151 0 1 "fixation" 1988351 1988546 195 0.291833 0.206397 18.88542 0 1 "fixation" 1988592 1988736 144 0.296297 0.209546 17.690373 0 1 "fixation" 1988788 1989012 224 0.271854 0.192719 19.130211 0 1 "fixation" 1989044 1989170 126 0.349 0.304362 18.616167 … … … … … … … … … 0 1 "fixation.ivt" 2003929 2004090 161 2.814851 2.527788 212.117446 0 1 "fixation.ivt" 2004091 2004363 272 2.819008 2.518967 244.333244 0 1 "fixation.ivt" 2004364 2004883 519 2.768099 2.46208 194.527643 0 1 "fixation.ivt" 2004885 2005116 231 2.805674 2.507902 203.067333 0 1 "fixation.ivt" 2005117 2005298 181 2.744494 2.578767 329.741947 -
NoneNone
-
-
list (2 items)
- 'text_id'
- 'page_id'
-
ExperimentExperiment
-
EyeTrackerEyeTracker
-
NoneNone
-
NoneNone
-
NoneNone
-
NoneNone
-
10001000
-
NoneNone
-
NoneNone
-
-
10001000
-
ScreenScreen
-
6868
-
30.230.2
-
10241024
-
'upper left''upper left'
-
3838
-
12801280
-
15.59938648778295315.599386487782953
-
-15.599386487782953-15.599386487782953
-
12.50804441088254612.508044410882546
-
-12.508044410882546-12.508044410882546
-
-
-
DataFrame (9 columns, 264 rows)shape: (264, 9)
text_id page_id name onset offset duration dispersion amplitude peak_velocity i64 i64 str i64 i64 i64 f64 f64 f64 0 1 "fixation" 1988145 1988322 177 0.154958 0.110074 16.24151 0 1 "fixation" 1988351 1988546 195 0.291833 0.206397 18.88542 0 1 "fixation" 1988592 1988736 144 0.296297 0.209546 17.690373 0 1 "fixation" 1988788 1989012 224 0.271854 0.192719 19.130211 0 1 "fixation" 1989044 1989170 126 0.349 0.304362 18.616167 … … … … … … … … … 0 1 "fixation.ivt" 2003929 2004090 161 2.814851 2.527788 212.117446 0 1 "fixation.ivt" 2004091 2004363 272 2.819008 2.518967 244.333244 0 1 "fixation.ivt" 2004364 2004883 519 2.768099 2.46208 194.527643 0 1 "fixation.ivt" 2004885 2005116 231 2.805674 2.507902 203.067333 0 1 "fixation.ivt" 2005117 2005298 181 2.744494 2.578767 329.741947 -
NoneNone