pymovements in 10 minutes#

What you will learn in this tutorial:#

  • how to download one of the publicly available datasets

  • how to load a subset of the data into your memory

  • how to transform pixel coordinates into degrees of visual angle

  • how to transform positional data into velocity data

  • how to detect fixations by using the I-VT algorithm

  • how to detect saccades by using the microsaccades algorithm

  • how to compute additional event properties for your analysis

  • how to save your preprocessed data

  • how to plot the main saccadic sequence from your data

Downloading one of the public datasets#

We import pymovements as the alias pm for convenience.

[1]:
import polars as pl

import pymovements as pm

pymovements provides a library of publicly available datasets.

You can browse through the available dataset definitions here: Datasets

For this tutorial we will limit ourselves to the ToyDataset due to its minimal space requirements.

Other datasets can be downloaded by simply replacing ToyDataset with one of the other available datasets.

We can initialize and download by passing the desired dataset name as a string argument.

Additionally we need the root directory path of your data.

[2]:
dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')
dataset.download()
INFO:pymovements.dataset.dataset:
        You are downloading the pymovements Toy Dataset. Please be aware that pymovements does not
        host or distribute any dataset resources and only provides a convenient interface to
        download the public dataset resources that were published by their respective authors.

        Please cite the referenced publication if you intend to use the dataset in your research.

Downloading http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/ to data/ToyDataset/downloads/pymovements-toy-dataset.zip
Checking integrity of pymovements-toy-dataset.zip
Extracting pymovements-toy-dataset.zip to data/ToyDataset/raw
100%|██████████| 23/23 [00:00<00:00, 305.17it/s]
[2]:
Dataset
  • DatasetDefinition
    DatasetDefinition
    • None
      None
    • dict (0 items)
      • dict (1 items)
        • dict (4 items)
          • list (5 items)
            • 'timestamp'
            • 'x'
            • (3 more)
          • dict (5 items)
            • Float64
              Float64
            • Float64
              Float64
            • (3 more)
          • (2 more)
      • None
        None
      • Experiment
        Experiment
        • EyeTracker
          EyeTracker
          • None
            None
          • None
            None
          • None
            None
          • None
            None
          • 1000
            1000
          • None
            None
          • None
            None
        • 1000
          1000
        • Screen
          Screen
          • 68
            68
          • 30.2
            30.2
          • 1024
            1024
          • 'upper left'
            'upper left'
          • 38
            38
          • 1280
            1280
          • 15.599386487782953
            15.599386487782953
          • -15.599386487782953
            -15.599386487782953
          • 12.508044410882546
            12.508044410882546
          • -12.508044410882546
            -12.508044410882546
      • None
        None
      • dict (1 items)
        • 'trial_{text_id:d}_{page_id:d}.csv'
          'trial_{text_id:d}_{page_id:d}.csv'
      • dict (1 items)
        • dict (2 items)
          • <class 'int'>
            <class 'int'>
          • <class 'int'>
            <class 'int'>
      • True
        True
      • 'pymovements Toy Dataset'
        'pymovements Toy Dataset'
      • dict (0 items)
        • 'ToyDataset'
          'ToyDataset'
        • list (2 items)
          • 'x'
          • 'y'
        • None
          None
        • list (1 items)
          • ResourceDefinition
            • 'gaze'
              'gaze'
            • 'pymovements-toy-dataset.zip'
              'pymovements-toy-dataset.zip'
            • 'trial_{text_id:d}_{page_id:d}.csv'
              'trial_{text_id:d}_{page_id:d}.csv'
            • dict (2 items)
              • <class 'int'>
                <class 'int'>
              • <class 'int'>
                <class 'int'>
            • '4da622457637a8181d86601fe17f3aa8'
              '4da622457637a8181d86601fe17f3aa8'
            • str
              'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
        • 'timestamp'
          'timestamp'
        • 'ms'
          'ms'
        • None
          None
        • None
          None
      • list (0 items)
        • DataFrame (0 columns, 0 rows)
          shape: (0, 0)
        • list (0 items)
          • PosixPath('data/ToyDataset')
            PosixPath('data/ToyDataset')
          • DatasetPaths
            DatasetPaths
            • PosixPath('data/ToyDataset')
              PosixPath('data/ToyDataset')
            • PosixPath('data/ToyDataset/downloads')
              PosixPath('data/ToyDataset/downloads')
            • PosixPath('data/ToyDataset/events')
              PosixPath('data/ToyDataset/events')
            • PosixPath('data/ToyDataset/precomputed_events')
              PosixPath('data/ToyDataset/precomputed_events')
            • PosixPath
              PosixPath('data/ToyDataset/precomputed_reading_measures')
            • PosixPath('data/ToyDataset/preprocessed')
              PosixPath('data/ToyDataset/preprocessed')
            • PosixPath('data/ToyDataset/raw')
              PosixPath('data/ToyDataset/raw')
            • PosixPath('data/ToyDataset')
              PosixPath('data/ToyDataset')
          • list (0 items)
            • list (0 items)

              Our downloaded dataset will be placed in new a directory with the name of the dataset:

              [3]:
              
              dataset.path
              
              [3]:
              
              PosixPath('data/ToyDataset')
              

              Archive files are automatically extracted into the path specified by Dataset.paths.raw:

              [4]:
              
              dataset.paths.raw
              
              [4]:
              
              PosixPath('data/ToyDataset/raw')
              

              Loading in your data into memory#

              Next we load our dataset into memory to be able to work with it:

              [5]:
              
              dataset.load()
              
              [5]:
              
              Dataset
              • DatasetDefinition
                DatasetDefinition
                • None
                  None
                • dict (0 items)
                  • dict (1 items)
                    • dict (4 items)
                      • list (5 items)
                        • 'timestamp'
                        • 'x'
                        • (3 more)
                      • dict (5 items)
                        • Float64
                          Float64
                        • Float64
                          Float64
                        • (3 more)
                      • (2 more)
                  • None
                    None
                  • Experiment
                    Experiment
                    • EyeTracker
                      EyeTracker
                      • None
                        None
                      • None
                        None
                      • None
                        None
                      • None
                        None
                      • 1000
                        1000
                      • None
                        None
                      • None
                        None
                    • 1000
                      1000
                    • Screen
                      Screen
                      • 68
                        68
                      • 30.2
                        30.2
                      • 1024
                        1024
                      • 'upper left'
                        'upper left'
                      • 38
                        38
                      • 1280
                        1280
                      • 15.599386487782953
                        15.599386487782953
                      • -15.599386487782953
                        -15.599386487782953
                      • 12.508044410882546
                        12.508044410882546
                      • -12.508044410882546
                        -12.508044410882546
                  • None
                    None
                  • dict (1 items)
                    • 'trial_{text_id:d}_{page_id:d}.csv'
                      'trial_{text_id:d}_{page_id:d}.csv'
                  • dict (1 items)
                    • dict (2 items)
                      • <class 'int'>
                        <class 'int'>
                      • <class 'int'>
                        <class 'int'>
                  • True
                    True
                  • 'pymovements Toy Dataset'
                    'pymovements Toy Dataset'
                  • dict (0 items)
                    • 'ToyDataset'
                      'ToyDataset'
                    • list (2 items)
                      • 'x'
                      • 'y'
                    • None
                      None
                    • list (1 items)
                      • ResourceDefinition
                        • 'gaze'
                          'gaze'
                        • 'pymovements-toy-dataset.zip'
                          'pymovements-toy-dataset.zip'
                        • 'trial_{text_id:d}_{page_id:d}.csv'
                          'trial_{text_id:d}_{page_id:d}.csv'
                        • dict (2 items)
                          • <class 'int'>
                            <class 'int'>
                          • <class 'int'>
                            <class 'int'>
                        • '4da622457637a8181d86601fe17f3aa8'
                          '4da622457637a8181d86601fe17f3aa8'
                        • str
                          'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
                    • 'timestamp'
                      'timestamp'
                    • 'ms'
                      'ms'
                    • None
                      None
                    • None
                      None
                  • list (0 items)
                    • dict (1 items)
                      • DataFrame (3 columns, 20 rows)
                        shape: (20, 3)
                        text_idpage_idfilepath
                        i64i64str
                        01"aeye-lab-pymovements-toy-datas…
                        02"aeye-lab-pymovements-toy-datas…
                        03"aeye-lab-pymovements-toy-datas…
                        04"aeye-lab-pymovements-toy-datas…
                        05"aeye-lab-pymovements-toy-datas…
                        31"aeye-lab-pymovements-toy-datas…
                        32"aeye-lab-pymovements-toy-datas…
                        33"aeye-lab-pymovements-toy-datas…
                        34"aeye-lab-pymovements-toy-datas…
                        35"aeye-lab-pymovements-toy-datas…
                    • list (20 items)
                      • Gaze
                        • DataFrame (6 columns, 17223 rows)
                          shape: (17_223, 6)
                          timestimuli_xstimuli_ytext_idpage_idpixel
                          i64f64f64i64i64list[f64]
                          1988145-1.0-1.001[206.8, 152.4]
                          1988146-1.0-1.001[206.9, 152.1]
                          1988147-1.0-1.001[207.0, 151.8]
                          1988148-1.0-1.001[207.1, 151.7]
                          1988149-1.0-1.001[207.0, 151.5]
                          2005363-1.0-1.001[361.0, 415.4]
                          2005364-1.0-1.001[358.0, 414.5]
                          2005365-1.0-1.001[355.8, 413.8]
                          2005366-1.0-1.001[353.1, 413.2]
                          2005367-1.0-1.001[351.2, 412.9]
                        • Events
                          Events
                          • DataFrame (6 columns, 0 rows)
                            shape: (0, 6)
                            text_idpage_idnameonsetoffsetduration
                            i64i64stri64i64i64
                          • list (2 items)
                            • 'text_id'
                            • 'page_id'
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'
                        • Experiment
                          Experiment
                          • EyeTracker
                            EyeTracker
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • 1000
                              1000
                            • None
                              None
                            • None
                              None
                          • 1000
                            1000
                          • Screen
                            Screen
                            • 68
                              68
                            • 30.2
                              30.2
                            • 1024
                              1024
                            • 'upper left'
                              'upper left'
                            • 38
                              38
                            • 1280
                              1280
                            • 15.599386487782953
                              15.599386487782953
                            • -15.599386487782953
                              -15.599386487782953
                            • 12.508044410882546
                              12.508044410882546
                            • -12.508044410882546
                              -12.508044410882546
                      • Gaze
                        • DataFrame (6 columns, 29799 rows)
                          shape: (29_799, 6)
                          timestimuli_xstimuli_ytext_idpage_idpixel
                          i64f64f64i64i64list[f64]
                          2008305-1.0-1.002[141.4, 153.6]
                          2008306-1.0-1.002[141.1, 153.2]
                          2008307-1.0-1.002[140.7, 152.8]
                          2008308-1.0-1.002[140.6, 152.7]
                          2008309-1.0-1.002[140.5, 152.6]
                          2038099-1.0-1.002[273.8, 773.8]
                          2038100-1.0-1.002[273.8, 774.1]
                          2038101-1.0-1.002[273.9, 774.5]
                          2038102-1.0-1.002[274.0, 774.4]
                          2038103-1.0-1.002[274.0, 773.9]
                        • Events
                          Events
                          • DataFrame (6 columns, 0 rows)
                            shape: (0, 6)
                            text_idpage_idnameonsetoffsetduration
                            i64i64stri64i64i64
                          • list (2 items)
                            • 'text_id'
                            • 'page_id'
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'
                        • Experiment
                          Experiment
                          • EyeTracker
                            EyeTracker
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • 1000
                              1000
                            • None
                              None
                            • None
                              None
                          • 1000
                            1000
                          • Screen
                            Screen
                            • 68
                              68
                            • 30.2
                              30.2
                            • 1024
                              1024
                            • 'upper left'
                              'upper left'
                            • 38
                              38
                            • 1280
                              1280
                            • 15.599386487782953
                              15.599386487782953
                            • -15.599386487782953
                              -15.599386487782953
                            • 12.508044410882546
                              12.508044410882546
                            • -12.508044410882546
                              -12.508044410882546
                      • (18 more)
                    • PosixPath('data/ToyDataset')
                      PosixPath('data/ToyDataset')
                    • DatasetPaths
                      DatasetPaths
                      • PosixPath('data/ToyDataset')
                        PosixPath('data/ToyDataset')
                      • PosixPath('data/ToyDataset/downloads')
                        PosixPath('data/ToyDataset/downloads')
                      • PosixPath('data/ToyDataset/events')
                        PosixPath('data/ToyDataset/events')
                      • PosixPath('data/ToyDataset/precomputed_events')
                        PosixPath('data/ToyDataset/precomputed_events')
                      • PosixPath
                        PosixPath('data/ToyDataset/precomputed_reading_measures')
                      • PosixPath('data/ToyDataset/preprocessed')
                        PosixPath('data/ToyDataset/preprocessed')
                      • PosixPath('data/ToyDataset/raw')
                        PosixPath('data/ToyDataset/raw')
                      • PosixPath('data/ToyDataset')
                        PosixPath('data/ToyDataset')
                    • list (0 items)
                      • list (0 items)

                        This way we fill two attributes with data. First we have the fileinfo attribute which holds all the basic information for files:

                        [6]:
                        
                        dataset.fileinfo['gaze'].head()
                        
                        [6]:
                        
                        shape: (5, 3)
                        text_idpage_idfilepath
                        i64i64str
                        01"aeye-lab-pymovements-toy-datas…
                        02"aeye-lab-pymovements-toy-datas…
                        03"aeye-lab-pymovements-toy-datas…
                        04"aeye-lab-pymovements-toy-datas…
                        05"aeye-lab-pymovements-toy-datas…

                        We notice that for each filepath a text_id and page_id is specified.

                        We have also loaded our gaze data into the dataframes in the gaze attribute:

                        [7]:
                        
                        dataset.gaze[0]
                        
                        [7]:
                        
                        Gaze
                        • DataFrame (6 columns, 17223 rows)
                          shape: (17_223, 6)
                          timestimuli_xstimuli_ytext_idpage_idpixel
                          i64f64f64i64i64list[f64]
                          1988145-1.0-1.001[206.8, 152.4]
                          1988146-1.0-1.001[206.9, 152.1]
                          1988147-1.0-1.001[207.0, 151.8]
                          1988148-1.0-1.001[207.1, 151.7]
                          1988149-1.0-1.001[207.0, 151.5]
                          2005363-1.0-1.001[361.0, 415.4]
                          2005364-1.0-1.001[358.0, 414.5]
                          2005365-1.0-1.001[355.8, 413.8]
                          2005366-1.0-1.001[353.1, 413.2]
                          2005367-1.0-1.001[351.2, 412.9]
                        • Events
                          Events
                          • DataFrame (6 columns, 0 rows)
                            shape: (0, 6)
                            text_idpage_idnameonsetoffsetduration
                            i64i64stri64i64i64
                          • list (2 items)
                            • 'text_id'
                            • 'page_id'
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'
                        • Experiment
                          Experiment
                          • EyeTracker
                            EyeTracker
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • 1000
                              1000
                            • None
                              None
                            • None
                              None
                          • 1000
                            1000
                          • Screen
                            Screen
                            • 68
                              68
                            • 30.2
                              30.2
                            • 1024
                              1024
                            • 'upper left'
                              'upper left'
                            • 38
                              38
                            • 1280
                              1280
                            • 15.599386487782953
                              15.599386487782953
                            • -15.599386487782953
                              -15.599386487782953
                            • 12.508044410882546
                              12.508044410882546
                            • -12.508044410882546
                              -12.508044410882546

                        Apart from some trial identifier columns we see the columns time and pixel.

                        The last two columns refer to the pixel coordinates at the timestep specified by time.

                        We are also able to just take a subset of the data by specifying values of the fileinfo columns. The key refers to the column in the fileinfo dataframe. The values in the dictionary can be of type bool, int, float or str, but also lists and ranges

                        [8]:
                        
                        subset = {
                            'text_id': 0,
                            'page_id': [0, 1],
                        }
                        dataset.load(subset=subset)
                        
                        dataset.fileinfo
                        
                        [8]:
                        
                        {'gaze': shape: (1, 3)
                         ┌─────────┬─────────┬─────────────────────────────────┐
                         │ text_id ┆ page_id ┆ filepath                        │
                         │ ---     ┆ ---     ┆ ---                             │
                         │ i64     ┆ i64     ┆ str                             │
                         ╞═════════╪═════════╪═════════════════════════════════╡
                         │ 0       ┆ 1       ┆ aeye-lab-pymovements-toy-datas… │
                         └─────────┴─────────┴─────────────────────────────────┘}
                        

                        Now we selected only a small subset of our data.

                        Preprocessing raw gaze data#

                        We now want to preprocess our gaze data by transforming pixel coordinates into degrees of visual angle and then computing velocity data from our positional data.

                        [9]:
                        
                        dataset.pix2deg()
                        
                        dataset.gaze[0]
                        
                        [9]:
                        
                        Gaze
                        • DataFrame (7 columns, 17223 rows)
                          shape: (17_223, 7)
                          timestimuli_xstimuli_ytext_idpage_idpixelposition
                          i64f64f64i64i64list[f64]list[f64]
                          1988145-1.0-1.001[206.8, 152.4][-10.697598, -8.852399]
                          1988146-1.0-1.001[206.9, 152.1][-10.695183, -8.859678]
                          1988147-1.0-1.001[207.0, 151.8][-10.692768, -8.866956]
                          1988148-1.0-1.001[207.1, 151.7][-10.690352, -8.869381]
                          1988149-1.0-1.001[207.0, 151.5][-10.692768, -8.874233]
                          2005363-1.0-1.001[361.0, 415.4][-6.932438, -2.386672]
                          2005364-1.0-1.001[358.0, 414.5][-7.006376, -2.408998]
                          2005365-1.0-1.001[355.8, 413.8][-7.060582, -2.426362]
                          2005366-1.0-1.001[353.1, 413.2][-7.12709, -2.441245]
                          2005367-1.0-1.001[351.2, 412.9][-7.173881, -2.448686]
                        • Events
                          Events
                          • DataFrame (6 columns, 0 rows)
                            shape: (0, 6)
                            text_idpage_idnameonsetoffsetduration
                            i64i64stri64i64i64
                          • list (2 items)
                            • 'text_id'
                            • 'page_id'
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'
                        • Experiment
                          Experiment
                          • EyeTracker
                            EyeTracker
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • 1000
                              1000
                            • None
                              None
                            • None
                              None
                          • 1000
                            1000
                          • Screen
                            Screen
                            • 68
                              68
                            • 30.2
                              30.2
                            • 1024
                              1024
                            • 'upper left'
                              'upper left'
                            • 38
                              38
                            • 1280
                              1280
                            • 15.599386487782953
                              15.599386487782953
                            • -15.599386487782953
                              -15.599386487782953
                            • 12.508044410882546
                              12.508044410882546
                            • -12.508044410882546
                              -12.508044410882546

                        We notice that a new column has appeared: position. This column specifies the position coordinates in degrees of visual angle (dva).

                        For transforming our positional data into velocity data we will use the Savitzky-Golay differentiation filter.

                        We can also specify some additional parameters for this method:

                        [10]:
                        
                        dataset.pos2vel(method='savitzky_golay', degree=2, window_length=7)
                        
                        dataset.gaze[0]
                        
                        [10]:
                        
                        Gaze
                        • DataFrame (8 columns, 17223 rows)
                          shape: (17_223, 8)
                          timestimuli_xstimuli_ytext_idpage_idpixelpositionvelocity
                          i64f64f64i64i64list[f64]list[f64]list[f64]
                          1988145-1.0-1.001[206.8, 152.4][-10.697598, -8.852399][1.207641, -3.119165]
                          1988146-1.0-1.001[206.9, 152.1][-10.695183, -8.859678][1.20764, -4.072198]
                          1988147-1.0-1.001[207.0, 151.8][-10.692768, -8.866956][1.035119, -4.765267]
                          1988148-1.0-1.001[207.1, 151.7][-10.690352, -8.869381][1.207654, -4.245382]
                          1988149-1.0-1.001[207.0, 151.5][-10.692768, -8.874233][1.552735, -2.339263]
                          2005363-1.0-1.001[361.0, 415.4][-6.932438, -2.386672][-62.062479, -20.465552]
                          2005364-1.0-1.001[358.0, 414.5][-7.006376, -2.408998][-61.343786, -18.073031]
                          2005365-1.0-1.001[355.8, 413.8][-7.060582, -2.426362][-53.501231, -14.617634]
                          2005366-1.0-1.001[353.1, 413.2][-7.12709, -2.441245][-41.879965, -10.276475]
                          2005367-1.0-1.001[351.2, 412.9][-7.173881, -2.448686][-27.710881, -6.112645]
                        • Events
                          Events
                          • DataFrame (6 columns, 0 rows)
                            shape: (0, 6)
                            text_idpage_idnameonsetoffsetduration
                            i64i64stri64i64i64
                          • list (2 items)
                            • 'text_id'
                            • 'page_id'
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'
                        • Experiment
                          Experiment
                          • EyeTracker
                            EyeTracker
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • 1000
                              1000
                            • None
                              None
                            • None
                              None
                          • 1000
                            1000
                          • Screen
                            Screen
                            • 68
                              68
                            • 30.2
                              30.2
                            • 1024
                              1024
                            • 'upper left'
                              'upper left'
                            • 38
                              38
                            • 1280
                              1280
                            • 15.599386487782953
                              15.599386487782953
                            • -15.599386487782953
                              -15.599386487782953
                            • 12.508044410882546
                              12.508044410882546
                            • -12.508044410882546
                              -12.508044410882546

                        There is also the more general apply() method, which can be used to apply both transformation and event detection methods.

                        [11]:
                        
                        dataset.apply('pos2acc', degree=2, window_length=7)
                        
                        dataset.gaze[0]
                        
                        [11]:
                        
                        Gaze
                        • DataFrame (9 columns, 17223 rows)
                          shape: (17_223, 9)
                          timestimuli_xstimuli_ytext_idpage_idpixelpositionvelocityacceleration
                          i64f64f64i64i64list[f64]list[f64]list[f64]list[f64]
                          1988145-1.0-1.001[206.8, 152.4][-10.697598, -8.852399][1.207641, -3.119165][690.085837, -1501.799767]
                          1988146-1.0-1.001[206.9, 152.1][-10.695183, -8.859678][1.20764, -4.072198][0.001831, -866.371365]
                          1988147-1.0-1.001[207.0, 151.8][-10.692768, -8.866956][1.035119, -4.765267][-575.06741, -57.655244]
                          1988148-1.0-1.001[207.1, 151.7][-10.690352, -8.869381][1.207654, -4.245382][-230.013049, 1328.57081]
                          1988149-1.0-1.001[207.0, 151.5][-10.692768, -8.874233][1.552735, -2.339263][690.12611, 2021.586565]
                          2005363-1.0-1.001[361.0, 415.4][-6.932438, -2.386672][-62.062479, -20.465552][-1099.087619, 1477.17518]
                          2005364-1.0-1.001[358.0, 414.5][-7.006376, -2.408998][-61.343786, -18.073031][1834.348384, 2599.156806]
                          2005365-1.0-1.001[355.8, 413.8][-7.060582, -2.426362][-53.501231, -14.617634][9396.15507, 4547.960553]
                          2005366-1.0-1.001[353.1, 413.2][-7.12709, -2.441245][-41.879965, -10.276475][16194.183852, 5079.286997]
                          2005367-1.0-1.001[351.2, 412.9][-7.173881, -2.448686][-27.710881, -6.112645][16598.914618, 4193.246498]
                        • Events
                          Events
                          • DataFrame (6 columns, 0 rows)
                            shape: (0, 6)
                            text_idpage_idnameonsetoffsetduration
                            i64i64stri64i64i64
                          • list (2 items)
                            • 'text_id'
                            • 'page_id'
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'
                        • Experiment
                          Experiment
                          • EyeTracker
                            EyeTracker
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • None
                              None
                            • 1000
                              1000
                            • None
                              None
                            • None
                              None
                          • 1000
                            1000
                          • Screen
                            Screen
                            • 68
                              68
                            • 30.2
                              30.2
                            • 1024
                              1024
                            • 'upper left'
                              'upper left'
                            • 38
                              38
                            • 1280
                              1280
                            • 15.599386487782953
                              15.599386487782953
                            • -15.599386487782953
                              -15.599386487782953
                            • 12.508044410882546
                              12.508044410882546
                            • -12.508044410882546
                              -12.508044410882546

                        Detecting events#

                        Now let’s detect some events.

                        First we will detect fixations using the I-VT algorithm using its default parameters:

                        [12]:
                        
                        dataset.detect_events('ivt')
                        
                        dataset.events[0]
                        
                        [12]:
                        
                        Events
                        • DataFrame (6 columns, 72 rows)
                          shape: (72, 6)
                          text_idpage_idnameonsetoffsetduration
                          i64i64stri64i64i64
                          01"fixation"19881451988322177
                          01"fixation"19883511988546195
                          01"fixation"19885921988736144
                          01"fixation"19887881989012224
                          01"fixation"19890441989170126
                          01"fixation"20041142004349235
                          01"fixation"20043992004687288
                          01"fixation"20047142004878164
                          01"fixation"20049302005109179
                          01"fixation"20051382005286148
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'

                        Next we detect some saccades. This time we don’t use the default parameters but specify our own:

                        [13]:
                        
                        dataset.detect_events('microsaccades', minimum_duration=8)
                        
                        dataset.events[0].frame.filter(pl.col('name') == 'saccade').head()
                        
                        [13]:
                        
                        shape: (5, 6)
                        text_idpage_idnameonsetoffsetduration
                        i64i64stri64i64i64
                        01"saccade"1988323198833714
                        01"saccade"1988341198835110
                        01"saccade"1988546198856721
                        01"saccade"1988570198858313
                        01"saccade"1988736198876024

                        We can also use the more general interface of the apply() method:

                        [14]:
                        
                        dataset.apply('idt', dispersion_threshold=2.7, name='fixation.ivt')
                        
                        dataset.events[0].frame.filter(pl.col('name') == 'fixation.ivt').head()
                        
                        [14]:
                        
                        shape: (5, 6)
                        text_idpage_idnameonsetoffsetduration
                        i64i64stri64i64i64
                        01"fixation.ivt"19881451988563418
                        01"fixation.ivt"19885641988750186
                        01"fixation.ivt"19887511989178427
                        01"fixation.ivt"19891791989436257
                        01"fixation.ivt"19894371989600163

                        Computing event properties#

                        The event dataframe currently only holds the name, onset, offset and duration of an event (additionally we have some more identifier columns at the beginning).

                        We now want to compute some additional properties for each event. Event properties are things like peak velocity, amplitude and dispersion during an event.

                        We start out with computing the dispersion:

                        [15]:
                        
                        dataset.compute_event_properties("dispersion")
                        
                        dataset.events[0]
                        
                        [15]:
                        
                        Events
                        • DataFrame (7 columns, 264 rows)
                          shape: (264, 7)
                          text_idpage_idnameonsetoffsetdurationdispersion
                          i64i64stri64i64i64f64
                          01"fixation"198814519883221770.154958
                          01"fixation"198835119885461950.291833
                          01"fixation"198859219887361440.296297
                          01"fixation"198878819890122240.271854
                          01"fixation"198904419891701260.349
                          01"fixation.ivt"200392920040901612.814851
                          01"fixation.ivt"200409120043632722.819008
                          01"fixation.ivt"200436420048835192.768099
                          01"fixation.ivt"200488520051162312.805674
                          01"fixation.ivt"200511720052981812.744494
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'

                        We notice that a new column with the name dispersion has appeared in the event dataframe.

                        We can also pass a list of properties to compute all of our desired properties in a single run. Let’s add the amplitude and peak velocity:

                        [16]:
                        
                        dataset.compute_event_properties(["amplitude", "peak_velocity"])
                        
                        dataset.events[0]
                        
                        [16]:
                        
                        Events
                        • DataFrame (9 columns, 264 rows)
                          shape: (264, 9)
                          text_idpage_idnameonsetoffsetdurationdispersionamplitudepeak_velocity
                          i64i64stri64i64i64f64f64f64
                          01"fixation"198814519883221770.1549580.11007416.24151
                          01"fixation"198835119885461950.2918330.20639718.88542
                          01"fixation"198859219887361440.2962970.20954617.690373
                          01"fixation"198878819890122240.2718540.19271919.130211
                          01"fixation"198904419891701260.3490.30436218.616167
                          01"fixation.ivt"200392920040901612.8148512.527788212.117446
                          01"fixation.ivt"200409120043632722.8190082.518967244.333244
                          01"fixation.ivt"200436420048835192.7680992.46208194.527643
                          01"fixation.ivt"200488520051162312.8056742.507902203.067333
                          01"fixation.ivt"200511720052981812.7444942.578767329.741947
                        • list (2 items)
                          • 'text_id'
                          • 'page_id'

                        Plotting our data#

                        pymovements provides a range of plotting functions.

                        You can browse through the available plotting functions here: Plotting

                        In this this tutorial we will plot the saccadic main sequence of our data.

                        [17]:
                        
                        pm.plotting.main_sequence_plot(dataset.events[0])
                        
                        ../_images/tutorials_pymovements-in-10-minutes_43_0.png

                        Saving and loading your dataframes#

                        If we want to save interim results we can simply use the save() method like this:

                        [18]:
                        
                        dataset.save()
                        
                        [18]:
                        
                        Dataset
                        • DatasetDefinition
                          DatasetDefinition
                          • None
                            None
                          • dict (0 items)
                            • dict (1 items)
                              • dict (4 items)
                                • list (5 items)
                                  • 'timestamp'
                                  • 'x'
                                  • (3 more)
                                • dict (5 items)
                                  • Float64
                                    Float64
                                  • Float64
                                    Float64
                                  • (3 more)
                                • (2 more)
                            • None
                              None
                            • Experiment
                              Experiment
                              • EyeTracker
                                EyeTracker
                                • None
                                  None
                                • None
                                  None
                                • None
                                  None
                                • None
                                  None
                                • 1000
                                  1000
                                • None
                                  None
                                • None
                                  None
                              • 1000
                                1000
                              • Screen
                                Screen
                                • 68
                                  68
                                • 30.2
                                  30.2
                                • 1024
                                  1024
                                • 'upper left'
                                  'upper left'
                                • 38
                                  38
                                • 1280
                                  1280
                                • 15.599386487782953
                                  15.599386487782953
                                • -15.599386487782953
                                  -15.599386487782953
                                • 12.508044410882546
                                  12.508044410882546
                                • -12.508044410882546
                                  -12.508044410882546
                            • None
                              None
                            • dict (1 items)
                              • 'trial_{text_id:d}_{page_id:d}.csv'
                                'trial_{text_id:d}_{page_id:d}.csv'
                            • dict (1 items)
                              • dict (2 items)
                                • <class 'int'>
                                  <class 'int'>
                                • <class 'int'>
                                  <class 'int'>
                            • True
                              True
                            • 'pymovements Toy Dataset'
                              'pymovements Toy Dataset'
                            • dict (0 items)
                              • 'ToyDataset'
                                'ToyDataset'
                              • list (2 items)
                                • 'x'
                                • 'y'
                              • None
                                None
                              • list (1 items)
                                • ResourceDefinition
                                  • 'gaze'
                                    'gaze'
                                  • 'pymovements-toy-dataset.zip'
                                    'pymovements-toy-dataset.zip'
                                  • 'trial_{text_id:d}_{page_id:d}.csv'
                                    'trial_{text_id:d}_{page_id:d}.csv'
                                  • dict (2 items)
                                    • <class 'int'>
                                      <class 'int'>
                                    • <class 'int'>
                                      <class 'int'>
                                  • '4da622457637a8181d86601fe17f3aa8'
                                    '4da622457637a8181d86601fe17f3aa8'
                                  • str
                                    'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
                              • 'timestamp'
                                'timestamp'
                              • 'ms'
                                'ms'
                              • None
                                None
                              • None
                                None
                            • list (1 items)
                              • Events
                                • DataFrame (9 columns, 264 rows)
                                  shape: (264, 9)
                                  text_idpage_idnameonsetoffsetdurationdispersionamplitudepeak_velocity
                                  i64i64stri64i64i64f64f64f64
                                  01"fixation"198814519883221770.1549580.11007416.24151
                                  01"fixation"198835119885461950.2918330.20639718.88542
                                  01"fixation"198859219887361440.2962970.20954617.690373
                                  01"fixation"198878819890122240.2718540.19271919.130211
                                  01"fixation"198904419891701260.3490.30436218.616167
                                  01"fixation.ivt"200392920040901612.8148512.527788212.117446
                                  01"fixation.ivt"200409120043632722.8190082.518967244.333244
                                  01"fixation.ivt"200436420048835192.7680992.46208194.527643
                                  01"fixation.ivt"200488520051162312.8056742.507902203.067333
                                  01"fixation.ivt"200511720052981812.7444942.578767329.741947
                                • list (2 items)
                                  • 'text_id'
                                  • 'page_id'
                            • dict (1 items)
                              • DataFrame (3 columns, 1 rows)
                                shape: (1, 3)
                                text_idpage_idfilepath
                                i64i64str
                                01"aeye-lab-pymovements-toy-datas…
                            • list (1 items)
                              • Gaze
                                • DataFrame (9 columns, 17223 rows)
                                  shape: (17_223, 9)
                                  timestimuli_xstimuli_ytext_idpage_idpixelpositionvelocityacceleration
                                  i64f64f64i64i64list[f64]list[f64]list[f64]list[f64]
                                  1988145-1.0-1.001[206.8, 152.4][-10.697598, -8.852399][1.207641, -3.119165][690.085837, -1501.799767]
                                  1988146-1.0-1.001[206.9, 152.1][-10.695183, -8.859678][1.20764, -4.072198][0.001831, -866.371365]
                                  1988147-1.0-1.001[207.0, 151.8][-10.692768, -8.866956][1.035119, -4.765267][-575.06741, -57.655244]
                                  1988148-1.0-1.001[207.1, 151.7][-10.690352, -8.869381][1.207654, -4.245382][-230.013049, 1328.57081]
                                  1988149-1.0-1.001[207.0, 151.5][-10.692768, -8.874233][1.552735, -2.339263][690.12611, 2021.586565]
                                  2005363-1.0-1.001[361.0, 415.4][-6.932438, -2.386672][-62.062479, -20.465552][-1099.087619, 1477.17518]
                                  2005364-1.0-1.001[358.0, 414.5][-7.006376, -2.408998][-61.343786, -18.073031][1834.348384, 2599.156806]
                                  2005365-1.0-1.001[355.8, 413.8][-7.060582, -2.426362][-53.501231, -14.617634][9396.15507, 4547.960553]
                                  2005366-1.0-1.001[353.1, 413.2][-7.12709, -2.441245][-41.879965, -10.276475][16194.183852, 5079.286997]
                                  2005367-1.0-1.001[351.2, 412.9][-7.173881, -2.448686][-27.710881, -6.112645][16598.914618, 4193.246498]
                                • Events
                                  Events
                                  • DataFrame (9 columns, 264 rows)
                                    shape: (264, 9)
                                    text_idpage_idnameonsetoffsetdurationdispersionamplitudepeak_velocity
                                    i64i64stri64i64i64f64f64f64
                                    01"fixation"198814519883221770.1549580.11007416.24151
                                    01"fixation"198835119885461950.2918330.20639718.88542
                                    01"fixation"198859219887361440.2962970.20954617.690373
                                    01"fixation"198878819890122240.2718540.19271919.130211
                                    01"fixation"198904419891701260.3490.30436218.616167
                                    01"fixation.ivt"200392920040901612.8148512.527788212.117446
                                    01"fixation.ivt"200409120043632722.8190082.518967244.333244
                                    01"fixation.ivt"200436420048835192.7680992.46208194.527643
                                    01"fixation.ivt"200488520051162312.8056742.507902203.067333
                                    01"fixation.ivt"200511720052981812.7444942.578767329.741947
                                  • list (2 items)
                                    • 'text_id'
                                    • 'page_id'
                                • list (2 items)
                                  • 'text_id'
                                  • 'page_id'
                                • Experiment
                                  Experiment
                                  • EyeTracker
                                    EyeTracker
                                    • None
                                      None
                                    • None
                                      None
                                    • None
                                      None
                                    • None
                                      None
                                    • 1000
                                      1000
                                    • None
                                      None
                                    • None
                                      None
                                  • 1000
                                    1000
                                  • Screen
                                    Screen
                                    • 68
                                      68
                                    • 30.2
                                      30.2
                                    • 1024
                                      1024
                                    • 'upper left'
                                      'upper left'
                                    • 38
                                      38
                                    • 1280
                                      1280
                                    • 15.599386487782953
                                      15.599386487782953
                                    • -15.599386487782953
                                      -15.599386487782953
                                    • 12.508044410882546
                                      12.508044410882546
                                    • -12.508044410882546
                                      -12.508044410882546
                            • PosixPath('data/ToyDataset')
                              PosixPath('data/ToyDataset')
                            • DatasetPaths
                              DatasetPaths
                              • PosixPath('data/ToyDataset')
                                PosixPath('data/ToyDataset')
                              • PosixPath('data/ToyDataset/downloads')
                                PosixPath('data/ToyDataset/downloads')
                              • PosixPath('data/ToyDataset/events')
                                PosixPath('data/ToyDataset/events')
                              • PosixPath('data/ToyDataset/precomputed_events')
                                PosixPath('data/ToyDataset/precomputed_events')
                              • PosixPath
                                PosixPath('data/ToyDataset/precomputed_reading_measures')
                              • PosixPath('data/ToyDataset/preprocessed')
                                PosixPath('data/ToyDataset/preprocessed')
                              • PosixPath('data/ToyDataset/raw')
                                PosixPath('data/ToyDataset/raw')
                              • PosixPath('data/ToyDataset')
                                PosixPath('data/ToyDataset')
                            • list (0 items)
                              • list (0 items)

                                Let’s test this out by initializing a new PublicDataset object in the same directory and loading in the preprocessed gaze and event data.

                                This time we don’t need to download anything.

                                [19]:
                                
                                preprocessed_dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')
                                
                                dataset.load(events=True, preprocessed=True, subset=subset)
                                
                                display(dataset.gaze[0])
                                display(dataset.events[0])
                                
                                Gaze
                                • DataFrame (9 columns, 17223 rows)
                                  shape: (17_223, 9)
                                  timestimuli_xstimuli_ypixelpositionvelocityaccelerationtext_idpage_id
                                  i64f64f64list[f64]list[f64]list[f64]list[f64]i64i64
                                  1988145-1.0-1.0[206.8, 152.4][-10.697598, -8.852399][1.207641, -3.119165][690.085837, -1501.799767]01
                                  1988146-1.0-1.0[206.9, 152.1][-10.695183, -8.859678][1.20764, -4.072198][0.001831, -866.371365]01
                                  1988147-1.0-1.0[207.0, 151.8][-10.692768, -8.866956][1.035119, -4.765267][-575.06741, -57.655244]01
                                  1988148-1.0-1.0[207.1, 151.7][-10.690352, -8.869381][1.207654, -4.245382][-230.013049, 1328.57081]01
                                  1988149-1.0-1.0[207.0, 151.5][-10.692768, -8.874233][1.552735, -2.339263][690.12611, 2021.586565]01
                                  2005363-1.0-1.0[361.0, 415.4][-6.932438, -2.386672][-62.062479, -20.465552][-1099.087619, 1477.17518]01
                                  2005364-1.0-1.0[358.0, 414.5][-7.006376, -2.408998][-61.343786, -18.073031][1834.348384, 2599.156806]01
                                  2005365-1.0-1.0[355.8, 413.8][-7.060582, -2.426362][-53.501231, -14.617634][9396.15507, 4547.960553]01
                                  2005366-1.0-1.0[353.1, 413.2][-7.12709, -2.441245][-41.879965, -10.276475][16194.183852, 5079.286997]01
                                  2005367-1.0-1.0[351.2, 412.9][-7.173881, -2.448686][-27.710881, -6.112645][16598.914618, 4193.246498]01
                                • Events
                                  Events
                                  • DataFrame (9 columns, 264 rows)
                                    shape: (264, 9)
                                    text_idpage_idnameonsetoffsetdurationdispersionamplitudepeak_velocity
                                    i64i64stri64i64i64f64f64f64
                                    01"fixation"198814519883221770.1549580.11007416.24151
                                    01"fixation"198835119885461950.2918330.20639718.88542
                                    01"fixation"198859219887361440.2962970.20954617.690373
                                    01"fixation"198878819890122240.2718540.19271919.130211
                                    01"fixation"198904419891701260.3490.30436218.616167
                                    01"fixation.ivt"200392920040901612.8148512.527788212.117446
                                    01"fixation.ivt"200409120043632722.8190082.518967244.333244
                                    01"fixation.ivt"200436420048835192.7680992.46208194.527643
                                    01"fixation.ivt"200488520051162312.8056742.507902203.067333
                                    01"fixation.ivt"200511720052981812.7444942.578767329.741947
                                  • None
                                    None
                                • list (2 items)
                                  • 'text_id'
                                  • 'page_id'
                                • Experiment
                                  Experiment
                                  • EyeTracker
                                    EyeTracker
                                    • None
                                      None
                                    • None
                                      None
                                    • None
                                      None
                                    • None
                                      None
                                    • 1000
                                      1000
                                    • None
                                      None
                                    • None
                                      None
                                  • 1000
                                    1000
                                  • Screen
                                    Screen
                                    • 68
                                      68
                                    • 30.2
                                      30.2
                                    • 1024
                                      1024
                                    • 'upper left'
                                      'upper left'
                                    • 38
                                      38
                                    • 1280
                                      1280
                                    • 15.599386487782953
                                      15.599386487782953
                                    • -15.599386487782953
                                      -15.599386487782953
                                    • 12.508044410882546
                                      12.508044410882546
                                    • -12.508044410882546
                                      -12.508044410882546
                                Events
                                • DataFrame (9 columns, 264 rows)
                                  shape: (264, 9)
                                  text_idpage_idnameonsetoffsetdurationdispersionamplitudepeak_velocity
                                  i64i64stri64i64i64f64f64f64
                                  01"fixation"198814519883221770.1549580.11007416.24151
                                  01"fixation"198835119885461950.2918330.20639718.88542
                                  01"fixation"198859219887361440.2962970.20954617.690373
                                  01"fixation"198878819890122240.2718540.19271919.130211
                                  01"fixation"198904419891701260.3490.30436218.616167
                                  01"fixation.ivt"200392920040901612.8148512.527788212.117446
                                  01"fixation.ivt"200409120043632722.8190082.518967244.333244
                                  01"fixation.ivt"200436420048835192.7680992.46208194.527643
                                  01"fixation.ivt"200488520051162312.8056742.507902203.067333
                                  01"fixation.ivt"200511720052981812.7444942.578767329.741947
                                • None
                                  None