What’s new#

v0.7.dev#

Object-oriented hypnogram

This version introduces the new yasa.Hypnogram class, which will gradually become the standard way to store and manipulate hypnograms in YASA. Put simply, YASA now uses an object-oriented approach to hypnograms. That is, hypnograms are now stored as a class (aka object), which comes with several pre-built functions (aka methods) and attributes. See for example below:

from yasa import Hypnogram
# Create an hypnogram object
values = ["W", "W", "W", "S", "S", "S", "S", "S", "W", "S", "S", "S"]
hyp = Hypnogram(values, n_stages=2, start="2022-12-23 22:30:00", scorer="RM")
# Below are some class attributes
hyp.hypno  # Hypnogram values (a pandas.Series of categorical dtype)
hyp.duration  # Total duration of the hypnogram, in minutes
hyp.sampling_frequency  # Sampling frequency of the hypnogram
hyp.mapping  # Mapping from strings to integers
hyp.proba  # Probability of each sleep stage, if specified
# Below are some class methods
hyp.sleep_statistics()  # Calculate the sleep statistics
hyp.plot_hypnogram()  # Plot the hypnogram
hyp.upsample_to_data()  # Upsample to data

This brings along critical changes to several YASA function, for example:

yasa.SleepStaging now returns a yasa.Hypnogram instead of a numpy.ndarray. The probability of each sleep stage for each epoch can now be accessed with yasa.Hypnogram.proba.
yasa.simulate_hypnogram now returns a yasa.Hypnogram instead of a numpy.ndarray.
The suggested approach to plotting hypnograms is through the yasa.Hypnogram.plot_hypnogram method. The old function yasa.plot_hypnogram still exists, but now requires a yasa.Hypnogram instance as input.

Other improvements

Added helpful string representation (__repr__) to yasa.SleepStaging.
yasa.simulate_hypnogram now returns a yasa.Hypnogram instead of a numpy.ndarray.
The suggested approach to plotting hypnograms is through the yasa.Hypnogram.plot_hypnogram method. The old function yasa.plot_hypnogram still exists, but now requires a yasa.Hypnogram instance as input.

v0.6.5 (July 2024)#

Minor release with a brand new Evaluation module and several minor bugfixes

PR 130: Add new yasa.EpochByEpochAgreement and yasa.SleepStatsAgreement modules for evaluating the agreement between two scorers (e.g. human vs YASA), either at the epoch-by-epoch level or at the level of summary sleep statistics. This is an experimental feature. Remaining tasks to complete the Evaluation pipeline are described in: raphaelvallat/yasa#166
PR 171: Fix deprecated scipy.integrate.simps`.

v0.6.4 (January 2024)#

Minor release with one bugfix and one improvement to an existing function.

Fix MNE plot_topomap deprecation (PR 119)
Pass relative_prominence parameters in yasa.rem_detect (PR 151)
Rename yasa.simulate_hypno function to yasa.simulate_hypnogram
Add Hypnogram class in public API (see v0.7.dev).
Requires mne>=1.3, numpy>=1.18.1, numba>=0.57.1

v0.6.3 (December 2022)#

This is a minor release with one bugfix and one new function. Big shoutout to @remrama for his huge help on this new version!

Bugfix

Solved a bug in the spindles detection which resulted in no spindle being detected on some systems. See issue 107 for detail. PR 115

New functions

Added the yasa.simulate_hypno function to generate a simulated hypnogram, primarily for testing purposes and tutorials. The hypnogram is simulated as a Markov sequence based on sleep stage transition probabilities. Transition probabilities can be user-defined or will default to those published in Metzner et al., 2021, Commun Biol (see Figure 5b).

Improvements

Added the ax keyword-argument to yasa.plot_hypnogram and removed figsize. Now select figure aesthetics (e.g., size, dpi) by opening a matplotlib.axes.Axes instance and passing to ax. PR 108
yasa.plot_hypnogram now draws the hypnogram with matplotlib.pyplot.stairs instead of matplotlib.pyplot.step. See issue 106 for detail. PR 108

v0.6.2 (August 2022)#

ECG analysis - PR 68

Added the yasa.hypno_find_periods function to find sequences of consecutive values in hypnogram that are longer than a certain duration. This is a flexible function that can be used to detect NREM/REM periods.
Added the yasa.hrv_stage function, which calculates heart rate (HR) and heart rate variability (HRV) by stage and periods.
Added a new dataset containing 8 hours of ECG data. The dataset is in compressed NumPy format and can be found in notebooks/data_ECG_8hrs_200Hz.npz. The dataset also includes an upsampled hypnogram.
Added a new Jupyter notebook to calculate EEG-HRV overnight coupling based on the yasa.hrv_stage function. raphaelvallat/yasa

Spindles & slow-waves detection - PR 71

Added the yasa.compare_detection function to determine the correctness of detected events against ground-truth events. It calculates the true positive, false positives and false negatives, and from those, the precision, recall and F1-scores. The input should be the indices of the onset of the event, in samples. It includes a max_distance argument which specifies the tolerance window (in number of samples) for two events to be considered the same.
Added the yasa.SpindlesResults.compare_detection and yasa.SWResults.compare_detection method. This is a powerful and flexible function that allows to calculate the performance of the current detection against a) another detection or b) ground-truth annotations. For example, we can compare the output of the spindles detection with different thresholds.
Added the yasa.SpindlesResults.compare_channels and yasa.SWResults.compare_channels methods to compare the overlap of the detected events between channels. Agreement is calculated using the F1-score (default), precision or recall.
Add vmin and vmax parameters to yasa.plot_spectrogram. PR 75
Better handling of flat data in yasa.spindles_detect. The function previously returned a division by zero error if part of the data was flat. See issue 85
When using an MNE.Raw object, conversion of the data from Volts to micro-Volts is now performed within MNE. PR 70
Use black code formatting.

Others

When using an MNE.Raw object, conversion of the data from Volts to micro-Volts is now performed within MNE. PR 70
Added SleepECG to the dependencies. SleepECG is used for the heartbeats detection in yasa.hrv_stage.
YASA now requires MNE>0.23

v0.6.1 (March 2022)#

This release fixes a CRITICAL BUG with the spindles detection. Specifically, the yasa.spindles_detect could return different results depending on the sampling rate of the data. For example, downsampling the data from 256 Hz to 128 Hz may have significantly reduced the number of detected spindles. As explained in issue 54, this bug was caused by a floating-point error in numpy.convolve when calculating the soft spindle threshold. Tests seem to indicate that only certain sampling frequencies were impacted, such as 200 Hz, 256 Hz or 400 Hz. Other sampling frequencies such as 100 Hz and 500 Hz were seemingly not affected by this bug. Please double-check any results obtained with yasa.spindles_detect!

Warning

We recommend all users to upgrade to this new version ASAP and check any results obtained with the yasa.spindles_detect function!

v0.6.0 (February 2022)#

This is a MAJOR release with several API-breaking changes, new functions, bugfixes and a new section in the documentation.

Documentation

Added a Quickstart section to illustrate the main functions of YASA. Make sure to check it out!

Plotting

Added the yasa.plot_hypnogram function to plot an hypnogram.

Slow oscillations—sigma coupling

IMPORTANT — The default behavior of coupling=True in yasa.sw_detect has been changed:

YASA now uses a ± 1 second window around the negative peak of the slow-waves (2 sec total) to calculate the coupling, instead of a ± 2 sec window. Overall, this tends to increase the ndPAC values because of the higher temporal specificity. To keep a 4-sec window, use coupling_params['time'] = 2.
We’ve enabled the statistical thresholding in the ndPAC calculation. Practically, this means that events with a weak/unreliable coupling are assigned an ndPAC value of zero. Statistical thresholding can be disabled with coupling_params['p'] = None.

Warning

Because of these changes, the coupling values are therefore not comparable with previous versions of YASA. Please make sure to re-run your analyses with the new default parameters.

Events detection

The yasa.sw_detect function now uses more conservative amplitude thresholds: the max PTP amplitude has been reduced from 500 to 350 uV, the max negative amplitude has been reduced from 300 to 200 uV, and the max positive amplitude has been reduced from 200 to 150 uV.
Added yasa.SWResults.find_cooccurring_spindles to detect whether each slow-wave co-occurr with a sleep spindle.
Added the as_dataframe parameter in yasa.SWResults.get_sync_events and yasa.SpindlesResults.get_sync_events. If set to False, YASA will return the peak-locked data as a list (n_channels) of numpy arrays (n_events, n_times). This facilitates any analyses that requires access to event-locked data (e.g. time-frequency plot, or comodulogram).
Added the mask parameter in yasa.SWResults.summary, yasa.SWResults.get_sync_events, and yasa.SWResults.plot_average. This allows users to only include selected events in the summary or plots (e.g. the slow-waves with the largest peak-to-peak amplitude, or strongest coupling).
Added the mask parameter in yasa.SpindlesResults.summary, yasa.SpindlesResults.get_sync_events, and yasa.SpindlesResults.plot_average. This allows users to only include selected events in the summary or plots (e.g. the spindles with the largest amplitude).
Added the mask parameter in yasa.REMResults.summary, yasa.REMResults.get_sync_events, and yasa.REMResults.plot_average.

Others

yasa.irasa now informs about the maximum resampled fitting range, and raises a warning if parameters/frequencies are ill-specified. See PR42 and associated paper: https://doi.org/10.1101/2021.10.15.464483
Added a verbose parameter to yasa.hypno_upsample_to_data and yasa.irasa.
Remove Travis CI
Remove CI testing for Python 3.6

v0.5.1 (August 2021)#

This is a bugfix release. The latest pre-trained classifiers for yasa.SleepStaging were accidentally missing from the previous release. They have now been included in this release.

v0.5.0 (August 2021)#

This is a major release with an important bugfix for the slow-waves detection as well as API-breaking changes in the automatic sleep staging module. We recommend all users to upgrade to this version with pip install –upgrade yasa.

Slow-waves detection

We have fixed a critical bug in yasa.sw_detect in which the detection could keep slow-waves with invalid duration (e.g. several tens of seconds). We have now added extra safety checks to make sure that the total duration of the slow-waves does not exceed the maximum duration allowed by the dur_neg and dur_pos parameters (default = 2.5 seconds).

Warning

Please make sure to double-check any results obtained with yasa.sw_detect.

Sleep staging

Recently, we have published a preprint article describing YASA’s sleep staging algorithm and its validation across hundreds of polysomnography recordings. In July 2021, we have received comments from three reviewers, which have led us to implement several changes to the sleep staging algorithm. The most significant change is that the time lengths of the rolling windows have been updated from 5.5 minutes centered / 5 minutes past to 7.5 minutes centered / 2 min past, leading to slight improvements in accuracy. Furthermore, we have also updated the training database and the parameters of the LightGBM classifier. Unfortunately, these changes mean that the new version of the algorithm is no longer compatible with the previous version (0.4.0 or 0.4.1). Therefore, if you’re running a longitudinal study with YASA’s sleep staging, we either recommend to keep the previous version of YASA, or to update to the new version and reprocess all your nights with the new algorithm for consistency.

Sleep statistics

Artefact and Unscored epochs are now excluded from the calculation of the total sleep time (TST) in yasa.sleep_statistics. Previously, YASA calculated TST as SPT - WASO, thus including Art and Uns. TST is now calculated as the sum of all REM and NREM sleep in SPT.

New FAQ

The online documentation now has a brand new FAQ section! Make sure to check it out at https://raphaelvallat.com/yasa/build/html/faq.html

New function: coincidence matrix

We have added the yasa.SpindlesResults.get_coincidence_matrix and yasa.SWResults.get_coincidence_matrix methods to calculate the (scaled) coincidence matrix. The coincidence matrix gives, for each pair of channel, the number of samples that were marked as an event (spindles or slow-waves) in both channels. In other words, it gives an indication of whether events (spindles or slow-waves) are co-occuring for any pair of channel. The scaled version of the coincidence matrix can then be used to define functional networks or quickly find outlier channels.

Minor enhancements

Minor speed improvements in yasa.SleepStaging.
Updated dependency to pyRiemann>=0.2.7, which solves the version conflict with scikit-learn (see issue 33).
flake8 requirements for max line length has been changed from 80 to 100 characters.

v0.4.1 (March 2021)#

New functions

Added yasa.topoplot, a wrapper around mne.viz.plot_topomap. See 15_topoplot.ipynb

Enhancements

The default frequency range for slow-waves in yasa.sw_detect is now 0.3-1.5 Hz instead of 0.3-2 Hz. Indeed, most slow-waves have a frequency below 1Hz. This may result in slightly different coupling values when coupling=True so make sure to homogenize your slow-waves detection pipeline across all nights in your dataset.
yasa.trimbothstd now handles missing values in input array.
yasa.bandpower_from_psd and yasa.bandpower_from_psd_ndarray now print a warning if the PSD contains negative values. See issue 29.
Upon loading, YASA will now use the outdated package to check and warn the user if a newer stable version is available.
YASA now uses the antropy package to calculate non-linear features in the automatic sleep staging module. Previously, YASA was using EntroPy, which could not be installed using pip.

v0.4.0 (November 2020)#

This is a major release with several new functions, the biggest of which is the addition of an automatic sleep staging module (yasa.SleepStaging). This means that YASA can now automatically score the sleep stages of your raw EEG data. The classifier was trained and validated on more than 3000 nights from the National Sleep Research Resource (NSRR) website.

Briefly, the algorithm works by calculating a set of features for each 30-sec epochs from a central EEG channel (required), as well as an EOG channel (optional) and an EMG channel (optional). For best performance, users can also specify the age and the sex of the participants. Pre-trained classifiers are already included in YASA. The automatic sleep staging algorithm requires the LightGBM and antropy package.

Other changes

yasa.SpindlesResults and yasa.SWResults now have a plot_detection method which allows to interactively display the raw data with an overlay of the detected spindles. For now, this only works with Jupyter and it requires the ipywidgets package.
Added hue input parameter to yasa.SpindlesResults.plot_average, yasa.SWResults.plot_average to allow plotting by stage.
The get_sync_events() method now also returns the sleep stage when available.
The yasa.sw_detect now also returns the timestamp of the sigma peak in the SW-through-locked 4-seconds epochs. The timestamp is expressed in seconds from the beginning of the recording and can be found in the SigmaPeak column.

Dependencies

Switch to latest version of TensorPAC.
Added ipywidgets, LightGBM and entropy to dependencies.

v0.3.0 (May 2020)#

This is a major release with several API-breaking changes in the spindles, slow-waves and REMs detection.

First, the yasa.spindles_detect_multi and yasa.sw_detect_multi have been removed. Instead, the yasa.spindles_detect and yasa.sw_detect functions can now handle both single and multi-channel data.

Second, I was getting some feedback that it was difficult to get summary statistics from the detection dataframe. For instance, how can you get the average duration of the detected spindles, per channel and/or per stage? Similarly, how can you get the slow-waves count and density per stage and channel? To address these issues, I’ve now modified the output of the yasa.spindles_detect, yasa.sw_detect and yasa.rem_detect functions, which is now a class (= object) and not a simple Pandas DataFrame. The advantage is that the new output allows you to quickly get the raw data or summary statistics grouped by channel and/or sleep stage using the .summary() method.

>>> sp = yasa.spindles_detect(...)
>>> sp.summary()  # Returns the full detection dataframe
>>> sp.summary(grp_chan=True, grp_stage=True, aggfunc='mean')

Similarly, the yasa.get_bool_vector and yasa.get_sync_events functions are now directly implemented into the output, i.e.

>>> sw = yasa.sw_detect(...)
>>> sw.summary()
>>> sw.get_mask()
>>> sw.get_sync_events(center='NegPeak', time_before=0.4, time_after=0.8)

One can also quickly plot an average “template” of all the detected events:

>>> sw.plot_average(center="NegPeak", time_before=0.4, time_after=0.8)

For more details, please refer to the documentation of yasa.SpindlesResults, yasa.SWResults and yasa.REMResults.

Important

This is an experimental feature, and it’s likely that these functions will be modified, renamed, or even deprecated in future releases based on feedbacks from users. Please make sure to let me know what you think about the new output of the detection functions!

Other changes

The coupling argument has been removed from the yasa.spindles_detect function. Instead, slow-oscillations / sigma coupling can only be calculated from the slow-waves detection, which is 1) the most standard way, 2) better because PAC assumptions require a strong oscillatory component in the lower frequency range (slow-oscillations). This also avoids unecessary confusion between spindles-derived coupling and slow-waves-derived coupling. For more details, refer to the Jupyter notebooks.
Downsampling of data in detection functions has been removed. In other words, YASA will no longer downsample the data to 100 / 128 Hz before applying the events detection. If the detection is too slow, we recommend that you manually downsample your data before applying the detection. See for example mne.filter.resample.
yasa.trimbothstd can now work with multi-dimensional arrays. The trimmed standard deviation will always be calculated on the last axis of the array.
Filtering and Hilbert transform are now applied at once on all channels (instead of looping across individual channels) in the yasa.spindles_detect and yasa.sw_detect functions. This should lead to some improvements in computation time.

Older versions#

v0.2.0 (April 2020)

This is a major release with several new functions, bugfixes and miscellaneous enhancements in existing functions.

Bugfixes

Sleep efficiency in the yasa.sleep_statistics is now calculated using time in bed (TIB) as the denominator instead of sleep period time (SPT), in agreement with the AASM guidelines. The old way of computing the efficiency (TST / SPT) has now been renamed Sleep Maintenance Efficiency (SME).
The yasa.sliding_window now always return an array of shape (n_epochs, …, n_samples), i.e. the epochs are now always the first dimension of the epoched array. This is consistent with MNE default shape of mne.Epochs objects.

New functions

Added yasa.art_detect to automatically detect artefacts on single or multi-channel EEG data.
Added yasa.bandpower_from_psd_ndarray to calculate band power from a multi-dimensional PSD. This is a NumPy-only implementation and this function will return a np.array and not a pandas DataFrame. This function is useful if you need to calculate the bandpower from a 3-D PSD array, e.g. of shape (n_epochs, n_chan, n_freqs).
Added yasa.get_centered_indices to extract indices in data centered around specific events or peaks.
Added yasa.load_profusion_hypno to load a Compumedics Profusion hypnogram (.xml), as found on the National Sleep Research Resource (NSRR) website.

Enhancements

yasa.sleep_statistics now also returns the sleep onset latency, i.e. the latency to the first epoch of any sleep.
Added the bandpass argument to yasa.bandpower to apply a FIR bandpass filter using the lowest and highest frequencies defined in bands. This is useful if you work with absolute power and want to remove contributions from frequency bands of non-interests.
The yasa.bandpower_from_psd now always return the total absolute physical power (TotalAbsPow) of the signal, in units of uV^2 / Hz. This allows to quickly calculate the absolute bandpower from the relative bandpower.
Added sigma (12-16Hz) to the default frequency bands (bands) in yasa.bandpower and yasa.bandpower_from_psd.
Added the coupling and freq_sp keyword-arguments to the yasa.sw_detect function. If coupling=True, the function will return the phase of the slow-waves (in radians) at the most prominent peak of sigma-filtered band (PhaseAtSigmaPeak), as well as the normalized mean vector length (ndPAC).
Added an section in the 06_sw_detection.ipynb notebooks on how to use relative amplitude thresholds (e.g. z-scores or percentiles) instead of absolute thresholds in slow-waves detection.
The upper frequency band for yasa.sw_detect has been changed from freq_sw=(0.3, 3.5) to freq_sw=(0.3, 2) Hz to comply with AASM guidelines.
Stage is no longer taken into account when finding outliers with sklearn.ensemble.IsolationForest in yasa.spindles_detect.
To be consistent with yasa.spindles_detect, automatic outlier removal now requires at least 50 (instead of 100) detected events in yasa.sw_detect and yasa.rem_detect.
Added the verbose parameter to all detection functions.
Added -2 to the default hypnogram format to denote unscored data.

Dependencies

Removed deprecated behavior argument to avoid warning when calling sklearn.ensemble.IsolationForest.
Added TensorPAC and pyRiemann to dependencies.
Updated dependencies version for MNE and scikit-learn.