What’s new

v0.7.dev

Object-oriented hypnogram

This version introduces the new yasa.Hypnogram class, which will gradually become the standard way to store and manipulate hypnograms in YASA. Put simply, YASA now uses an object-oriented approach to hypnograms. That is, hypnograms are now stored as a class (aka object), which comes with several pre-built functions (aka methods) and attributes. See for example below:

from yasa import Hypnogram
# Create an hypnogram object
values = ["W", "W", "W", "S", "S", "S", "S", "S", "W", "S", "S", "S"]
hyp = Hypnogram(values, n_stages=2, start="2022-12-23 22:30:00", scorer="RM")
# Below are some class attributes
hyp.hypno  # Hypnogram values (a pandas.Series of categorical dtype)
hyp.duration  # Total duration of the hypnogram, in minutes
hyp.sampling_frequency  # Sampling frequency of the hypnogram
hyp.mapping  # Mapping from strings to integers
# Below are some class methods
hyp.sleep_statistics()  # Calculate the sleep statistics
hyp.plot_hypnogram()  # Plot the hypnogram
hyp.upsample_to_data()  # Upsample to data

Please see the documentation of yasa.Hypnogram for more details.

Important

The adoption of object-oriented yasa.Hypnogram usage brings along critical changes to several YASA function, for example:

v0.6.4 (January 2024)

Minor release with one bugfix and one improvement to an existing function.

  • Fix MNE plot_topomap deprecation (PR 119)

  • Pass relative_prominence parameters in yasa.rem_detect() (PR 151)

  • Rename yasa.simulate_hypno function to yasa.simulate_hypnogram()

  • Add Hypnogram class in public API (see v0.7.dev).

  • Requires mne>=1.3, numpy>=1.18.1, numba>=0.57.1

v0.6.3 (December 2022)

This is a minor release with one bugfix and one new function. Big shoutout to @remrama for his huge help on this new version!

Bugfix

Solved a bug in the spindles detection which resulted in no spindle being detected on some systems. See issue 107 for detail. PR 115

New functions

Added the yasa.simulate_hypno function to generate a simulated hypnogram, primarily for testing purposes and tutorials. The hypnogram is simulated as a Markov sequence based on sleep stage transition probabilities. Transition probabilities can be user-defined or will default to those published in Metzner et al., 2021, Commun Biol (see Figure 5b).

Improvements

v0.6.2 (August 2022)

ECG analysis - PR 68

  1. Added the yasa.hypno_find_periods() function to find sequences of consecutive values in hypnogram that are longer than a certain duration. This is a flexible function that can be used to detect NREM/REM periods.

  2. Added the yasa.hrv_stage() function, which calculates heart rate (HR) and heart rate variability (HRV) by stage and periods.

  3. Added a new dataset containing 8 hours of ECG data. The dataset is in compressed NumPy format and can be found in notebooks/data_ECG_8hrs_200Hz.npz. The dataset also includes an upsampled hypnogram.

  4. Added a new Jupyter notebook to calculate EEG-HRV overnight coupling based on the yasa.hrv_stage() function. https://github.com/raphaelvallat/yasa/blob/master/notebooks/16_EEG-HRV_coupling.ipynb

Spindles & slow-waves detection - PR 71

  1. Added the yasa.compare_detection() function to determine the correctness of detected events against ground-truth events. It calculates the true positive, false positives and false negatives, and from those, the precision, recall and F1-scores. The input should be the indices of the onset of the event, in samples. It includes a max_distance argument which specifies the tolerance window (in number of samples) for two events to be considered the same.

  2. Added the yasa.SpindlesResults.compare_detection() and yasa.SWResults.compare_detection() method. This is a powerful and flexible function that allows to calculate the performance of the current detection against a) another detection or b) ground-truth annotations. For example, we can compare the output of the spindles detection with different thresholds.

  3. Added the yasa.SpindlesResults.compare_channels() and yasa.SWResults.compare_channels() methods to compare the overlap of the detected events between channels. Agreement is calculated using the F1-score (default), precision or recall.

  4. Add vmin and vmax parameters to yasa.plot_spectrogram(). PR 75

  5. Better handling of flat data in yasa.spindles_detect(). The function previously returned a division by zero error if part of the data was flat. See issue 85

  6. When using an MNE.Raw object, conversion of the data from Volts to micro-Volts is now performed within MNE. PR 70

  7. Use black code formatting.

Others

  1. When using an MNE.Raw object, conversion of the data from Volts to micro-Volts is now performed within MNE. PR 70

  2. Added SleepECG to the dependencies. SleepECG is used for the heartbeats detection in yasa.hrv_stage().

  3. YASA now requires MNE>0.23

v0.6.1 (March 2022)

This release fixes a CRITICAL BUG with the spindles detection. Specifically, the yasa.spindles_detect() could return different results depending on the sampling rate of the data. For example, downsampling the data from 256 Hz to 128 Hz may have significantly reduced the number of detected spindles. As explained in issue 54, this bug was caused by a floating-point error in numpy.convolve() when calculating the soft spindle threshold. Tests seem to indicate that only certain sampling frequencies were impacted, such as 200 Hz, 256 Hz or 400 Hz. Other sampling frequencies such as 100 Hz and 500 Hz were seemingly not affected by this bug. Please double-check any results obtained with yasa.spindles_detect()!

Warning

We recommend all users to upgrade to this new version ASAP and check any results obtained with the yasa.spindles_detect() function!

v0.6.0 (February 2022)

This is a MAJOR release with several API-breaking changes, new functions, bugfixes and a new section in the documentation.

Documentation

  • Added a Quickstart section to illustrate the main functions of YASA. Make sure to check it out!

Plotting

Slow oscillations—sigma coupling

IMPORTANT — The default behavior of coupling=True in yasa.sw_detect() has been changed:

  • YASA now uses a ± 1 second window around the negative peak of the slow-waves (2 sec total) to calculate the coupling, instead of a ± 2 sec window. Overall, this tends to increase the ndPAC values because of the higher temporal specificity. To keep a 4-sec window, use coupling_params['time'] = 2.

  • We’ve enabled the statistical thresholding in the ndPAC calculation. Practically, this means that events with a weak/unreliable coupling are assigned an ndPAC value of zero. Statistical thresholding can be disabled with coupling_params['p'] = None.

Warning

Because of these changes, the coupling values are therefore not comparable with previous versions of YASA. Please make sure to re-run your analyses with the new default parameters.

Events detection

Others

v0.5.1 (August 2021)

This is a bugfix release. The latest pre-trained classifiers for yasa.SleepStaging were accidentally missing from the previous release. They have now been included in this release.

v0.5.0 (August 2021)

This is a major release with an important bugfix for the slow-waves detection as well as API-breaking changes in the automatic sleep staging module. We recommend all users to upgrade to this version with pip install –upgrade yasa.

Slow-waves detection

We have fixed a critical bug in yasa.sw_detect() in which the detection could keep slow-waves with invalid duration (e.g. several tens of seconds). We have now added extra safety checks to make sure that the total duration of the slow-waves does not exceed the maximum duration allowed by the dur_neg and dur_pos parameters (default = 2.5 seconds).

Warning

Please make sure to double-check any results obtained with yasa.sw_detect().

Sleep staging

Recently, we have published a preprint article describing YASA’s sleep staging algorithm and its validation across hundreds of polysomnography recordings. In July 2021, we have received comments from three reviewers, which have led us to implement several changes to the sleep staging algorithm. The most significant change is that the time lengths of the rolling windows have been updated from 5.5 minutes centered / 5 minutes past to 7.5 minutes centered / 2 min past, leading to slight improvements in accuracy. Furthermore, we have also updated the training database and the parameters of the LightGBM classifier. Unfortunately, these changes mean that the new version of the algorithm is no longer compatible with the previous version (0.4.0 or 0.4.1). Therefore, if you’re running a longitudinal study with YASA’s sleep staging, we either recommend to keep the previous version of YASA, or to update to the new version and reprocess all your nights with the new algorithm for consistency.

Sleep statistics

Artefact and Unscored epochs are now excluded from the calculation of the total sleep time (TST) in yasa.sleep_statistics(). Previously, YASA calculated TST as SPT - WASO, thus including Art and Uns. TST is now calculated as the sum of all REM and NREM sleep in SPT.

New FAQ

The online documentation now has a brand new FAQ section! Make sure to check it out at https://raphaelvallat.com/yasa/build/html/faq.html

New function: coincidence matrix

We have added the yasa.SpindlesResults.get_coincidence_matrix() and yasa.SWResults.get_coincidence_matrix() methods to calculate the (scaled) coincidence matrix. The coincidence matrix gives, for each pair of channel, the number of samples that were marked as an event (spindles or slow-waves) in both channels. In other words, it gives an indication of whether events (spindles or slow-waves) are co-occuring for any pair of channel. The scaled version of the coincidence matrix can then be used to define functional networks or quickly find outlier channels.

Minor enhancements

  1. Minor speed improvements in yasa.SleepStaging.

  2. Updated dependency to pyRiemann>=0.2.7, which solves the version conflict with scikit-learn (see issue 33).

  3. flake8 requirements for max line length has been changed from 80 to 100 characters.

v0.4.1 (March 2021)

New functions

  1. Added yasa.topoplot(), a wrapper around mne.viz.plot_topomap(). See 15_topoplot.ipynb

Enhancements

  1. The default frequency range for slow-waves in yasa.sw_detect() is now 0.3-1.5 Hz instead of 0.3-2 Hz. Indeed, most slow-waves have a frequency below 1Hz. This may result in slightly different coupling values when coupling=True so make sure to homogenize your slow-waves detection pipeline across all nights in your dataset.

  2. yasa.trimbothstd() now handles missing values in input array.

  3. yasa.bandpower_from_psd() and yasa.bandpower_from_psd_ndarray() now print a warning if the PSD contains negative values. See issue 29.

  4. Upon loading, YASA will now use the outdated package to check and warn the user if a newer stable version is available.

  5. YASA now uses the antropy package to calculate non-linear features in the automatic sleep staging module. Previously, YASA was using EntroPy, which could not be installed using pip.

v0.4.0 (November 2020)

This is a major release with several new functions, the biggest of which is the addition of an automatic sleep staging module (yasa.SleepStaging). This means that YASA can now automatically score the sleep stages of your raw EEG data. The classifier was trained and validated on more than 3000 nights from the National Sleep Research Resource (NSRR) website.

Briefly, the algorithm works by calculating a set of features for each 30-sec epochs from a central EEG channel (required), as well as an EOG channel (optional) and an EMG channel (optional). For best performance, users can also specify the age and the sex of the participants. Pre-trained classifiers are already included in YASA. The automatic sleep staging algorithm requires the LightGBM and antropy package.

Other changes

  1. yasa.SpindlesResults() and yasa.SWResults() now have a plot_detection method which allows to interactively display the raw data with an overlay of the detected spindles. For now, this only works with Jupyter and it requires the ipywidgets package.

  2. Added hue input parameter to yasa.SpindlesResults.plot_average(), yasa.SWResults.plot_average() to allow plotting by stage.

  3. The get_sync_events() method now also returns the sleep stage when available.

  4. The yasa.sw_detect() now also returns the timestamp of the sigma peak in the SW-through-locked 4-seconds epochs. The timestamp is expressed in seconds from the beginning of the recording and can be found in the SigmaPeak column.

Dependencies

  1. Switch to latest version of TensorPAC.

  2. Added ipywidgets, LightGBM and entropy to dependencies.

v0.3.0 (May 2020)

This is a major release with several API-breaking changes in the spindles, slow-waves and REMs detection.

First, the yasa.spindles_detect_multi() and yasa.sw_detect_multi() have been removed. Instead, the yasa.spindles_detect() and yasa.sw_detect() functions can now handle both single and multi-channel data.

Second, I was getting some feedback that it was difficult to get summary statistics from the detection dataframe. For instance, how can you get the average duration of the detected spindles, per channel and/or per stage? Similarly, how can you get the slow-waves count and density per stage and channel? To address these issues, I’ve now modified the output of the yasa.spindles_detect(), yasa.sw_detect() and yasa.rem_detect() functions, which is now a class (= object) and not a simple Pandas DataFrame. The advantage is that the new output allows you to quickly get the raw data or summary statistics grouped by channel and/or sleep stage using the .summary() method.

>>> sp = yasa.spindles_detect(...)
>>> sp.summary()  # Returns the full detection dataframe
>>> sp.summary(grp_chan=True, grp_stage=True, aggfunc='mean')

Similarly, the yasa.get_bool_vector() and yasa.get_sync_events() functions are now directly implemented into the output, i.e.

>>> sw = yasa.sw_detect(...)
>>> sw.summary()
>>> sw.get_mask()
>>> sw.get_sync_events(center='NegPeak', time_before=0.4, time_after=0.8)

One can also quickly plot an average “template” of all the detected events:

>>> sw.plot_average(center="NegPeak", time_before=0.4, time_after=0.8)

For more details, please refer to the documentation of yasa.SpindlesResults(), yasa.SWResults() and yasa.REMResults().

Important

This is an experimental feature, and it’s likely that these functions will be modified, renamed, or even deprecated in future releases based on feedbacks from users. Please make sure to let me know what you think about the new output of the detection functions!

Other changes

  1. The coupling argument has been removed from the yasa.spindles_detect() function. Instead, slow-oscillations / sigma coupling can only be calculated from the slow-waves detection, which is 1) the most standard way, 2) better because PAC assumptions require a strong oscillatory component in the lower frequency range (slow-oscillations). This also avoids unecessary confusion between spindles-derived coupling and slow-waves-derived coupling. For more details, refer to the Jupyter notebooks.

  2. Downsampling of data in detection functions has been removed. In other words, YASA will no longer downsample the data to 100 / 128 Hz before applying the events detection. If the detection is too slow, we recommend that you manually downsample your data before applying the detection. See for example mne.filter.resample().

  3. yasa.trimbothstd() can now work with multi-dimensional arrays. The trimmed standard deviation will always be calculated on the last axis of the array.

  4. Filtering and Hilbert transform are now applied at once on all channels (instead of looping across individual channels) in the yasa.spindles_detect() and yasa.sw_detect() functions. This should lead to some improvements in computation time.

Older versions

v0.2.0 (April 2020)

This is a major release with several new functions, bugfixes and miscellaneous enhancements in existing functions.

Bugfixes

  1. Sleep efficiency in the yasa.sleep_statistics() is now calculated using time in bed (TIB) as the denominator instead of sleep period time (SPT), in agreement with the AASM guidelines. The old way of computing the efficiency (TST / SPT) has now been renamed Sleep Maintenance Efficiency (SME).

  2. The yasa.sliding_window() now always return an array of shape (n_epochs, …, n_samples), i.e. the epochs are now always the first dimension of the epoched array. This is consistent with MNE default shape of mne.Epochs objects.

New functions

  1. Added yasa.art_detect() to automatically detect artefacts on single or multi-channel EEG data.

  2. Added yasa.bandpower_from_psd_ndarray() to calculate band power from a multi-dimensional PSD. This is a NumPy-only implementation and this function will return a np.array and not a pandas DataFrame. This function is useful if you need to calculate the bandpower from a 3-D PSD array, e.g. of shape (n_epochs, n_chan, n_freqs).

  3. Added yasa.get_centered_indices() to extract indices in data centered around specific events or peaks.

  4. Added yasa.load_profusion_hypno() to load a Compumedics Profusion hypnogram (.xml), as found on the National Sleep Research Resource (NSRR) website.

Enhancements

  1. yasa.sleep_statistics() now also returns the sleep onset latency, i.e. the latency to the first epoch of any sleep.

  2. Added the bandpass argument to yasa.bandpower() to apply a FIR bandpass filter using the lowest and highest frequencies defined in bands. This is useful if you work with absolute power and want to remove contributions from frequency bands of non-interests.

  3. The yasa.bandpower_from_psd() now always return the total absolute physical power (TotalAbsPow) of the signal, in units of uV^2 / Hz. This allows to quickly calculate the absolute bandpower from the relative bandpower.

  4. Added sigma (12-16Hz) to the default frequency bands (bands) in yasa.bandpower() and yasa.bandpower_from_psd().

  5. Added the coupling and freq_sp keyword-arguments to the yasa.sw_detect() function. If coupling=True, the function will return the phase of the slow-waves (in radians) at the most prominent peak of sigma-filtered band (PhaseAtSigmaPeak), as well as the normalized mean vector length (ndPAC).

  6. Added an section in the 06_sw_detection.ipynb notebooks on how to use relative amplitude thresholds (e.g. z-scores or percentiles) instead of absolute thresholds in slow-waves detection.

  7. The upper frequency band for yasa.sw_detect() has been changed from freq_sw=(0.3, 3.5) to freq_sw=(0.3, 2) Hz to comply with AASM guidelines.

  8. Stage is no longer taken into account when finding outliers with sklearn.ensemble.IsolationForest in yasa.spindles_detect().

  9. To be consistent with yasa.spindles_detect(), automatic outlier removal now requires at least 50 (instead of 100) detected events in yasa.sw_detect() and yasa.rem_detect().

  10. Added the verbose parameter to all detection functions.

  11. Added -2 to the default hypnogram format to denote unscored data.

Dependencies

  1. Removed deprecated behavior argument to avoid warning when calling sklearn.ensemble.IsolationForest.

  2. Added TensorPAC and pyRiemann to dependencies.

  3. Updated dependencies version for MNE and scikit-learn.

v0.1.9 (February 2020)

New functions

  1. Added yasa.transition_matrix() to calculate the state-transition matrix of an hypnogram.

  2. Added yasa.sleep_statistics() to extract the sleep statistics from an hypnogram.

  3. Added the coupling and freq_so keyword-arguments to the yasa.spindles_detect() function. If coupling=True, the function will also returns the phase of the slow-waves (in radians) at the most prominent peak of the spindles. This can be used to perform spindles-SO coupling, as explained in the new Jupyter notebooks on PAC and spindles-SO coupling.

Enhancements

  1. It is now possible to disable one or two out of the three thresholds in the yasa.spindles_detect(). This allows the users to run a simpler detection (for example focusing exclusively on the moving root mean square signal).

  2. The yasa.spindles_detect() now returns the timing (in seconds) of the most prominent peak of each spindles (Peak).

  3. The yasa.get_sync_sw has been renamed to yasa.get_sync_events() and is now compatible with spindles detection. This can be used for instance to plot the peak-locked grand averaged spindle.

Code testing

  1. Removed Travis and AppVeyor testing for Python 3.5.

v0.1.8 (October 2019)

  1. Added yasa.plot_spectrogram() function.

  2. Added lspopt in the dependencies.

  3. YASA now requires MNE>0.19.

  4. Added a notebook on non-linear features.

v0.1.7 (August 2019)

  1. Added yasa.sliding_window() function.

  2. Added yasa.irasa() function.

  3. Reorganized code into several sub-files for readability (internal changes with no effect on user experience).

v0.1.6 (August 2019)

  1. Added bandpower function

  2. One can now directly pass a raw MNE object in several multi-channel functions of YASA, instead of manually passing data, sf, and ch_names. YASA will automatically convert MNE data from Volts to uV, and extract the sampling frequency and channel names. Examples of this can be found in the Jupyter notebooks examples.

v0.1.5 (August 2019)

  1. Added REM detection (rem_detect) on LOC and ROC EOG channels + example notebook

  2. Added yasa/hypno.py file, with several functions to load and upsample sleep stage vector (hypnogram).

  3. Added yasa/spectral.py file, which includes the bandpower_from_psd function to calculate the single or multi-channel spectral power in specified bands from a pre-computed PSD (see example notebook at notebooks/10_bandpower.ipynb)

v0.1.4 (May 2019)

  1. Added get_sync_sw function to get the synchronized timings of landmarks timepoints in slow-wave sleep. This can be used in combination with seaborn.lineplot to plot an average template of the detected slow-wave, per channel.

v0.1.3 (March 2019)

  1. Added slow-waves detection for single and multi channel

  2. Added include argument to select which values of hypno should be used as a mask.

  3. New examples notebooks + changes in README

  4. Minor improvements in performance (e.g. faster detrending)

  5. Added html API (/html)

  6. Travis and AppVeyor test for Python 3.5, 3.6 and 3.7

v0.1.2 (February 2019)

  1. Added support for multi-channel detection via spindles_detect_multi function.

  2. Added support for hypnogram mask

  3. Added several notebook examples

  4. Changed some default parameters to optimize behavior

v0.1.1 (January 2019)

  1. Added post-processing Isolation Forest

  2. Updated Readme and added support with Visbrain

  3. Added Cz full night in notebooks/

v0.1 (December 2018)

Initial release of YASA: basic spindles detection.