Noise Model

Noise data refers to processing of time ordered data to extract statistics about the noise in the timestream. The core function is to take a power spectrum using the scipy.welch function and then extract the white and low frequency components of the resultant power spectrum. All outputs are returned in units of pA/rtHz which is the sqrt of the psd returned by the welch algorithm and sometimes the amplitude spectral density (ASD in pA/rtHz) is referred to as the power spectrum colloquially (we tried to avoid this confusion in the docs but be aware of this inprecise language). The white component is reported as the amplitude of the flat section of the ASD and is either determined by fitting to a model containing a white and low-f component or by taking the median of the ASD between f_min and f_max which default to 10 and 30 Hz respectively. The low-f component is characterized by the f_knee which is the point where the low-f and white components in the power spectrum (here we really mean PSD) are equal. We find this point by either fitting to a model that contains both components or binning + averaging the ASD below the region used to calculate the white component and finding where it rises to sqrt(2)*(white noise). This doesn’t fully constrain the low-f component because f_knee will scale with the white noise level and the steepness (or spectral index i.e. 1/f^n) is still unknown. To constrain this we additionally report the value of the ASD at 0.01 Hz relative to a baseline required ASD spectrum (215 pA/rtHz at 0.01 Hz corresponding to a white noise = 65 pA/rtHz and the ASD spectral index of the low-f component is 1/f^{1/2}) and in the case of using the fitting option we also report the fitted spectral index n.

Data Taking Example Usage

This example takes 5 minutes of noise data (to get a good measure of 1/f you really want a longer timestream like this but can process shorter data as well) on all channels turned on attached to the smurf blade in slot 2. Then it also generates band summary plots and a channel plot for readout channel 0. Plots are displayed but not saved:

from sodetlib import noise
from sodetlib.det_config import DetConfig

cfg  = DetConfig()
cfg.load_config_files(slot=2)
S = cfg.get_smurf_control()

noise.take_noise(S, cfg, acq_time=300, plot_band_summary=True,
                 show_plot=True, save_plot=False, plotted_rchans=[0])

Analyzing Existing Data

This example assumes that you have your data in an AxisManager loaded using sodetlib.load_session() or sotodlib.io.load_smurf.G3tSmurf:

from sodetlib import noise

#am is the AxisManager with your noise data.
#Can set fit=True but will take longer
outdict = noise.get_noise_params(am,fit=False)
#Make band summary plots
_ = noise.plot_band_noise(am,noisedict=outdict)
#Display but don't save a channel plot for readout channel 0.
rc = 0
fig,ax = noise.plot_channel_noise(am, rc, noisedict=outdict, show_plot=True,
                                  save_plot=False)

API

sodetlib.noise.fit_noise_asd(f, Axx, wl_f_range=(10, 30), p0=None)

Return model fit for a ASD.

Parameters:

  • f (float array) – The frequency information.

  • Axx (float array) – The power spectral data.

  • wl_f_range (float tuple) – tuple contains (f_low, f_high), f_high is used to determine the range of the ASD to fit to the noise model and if fit fails then the white noise is calculated as the median between f_low and f_high.

  • p0 (float array or None, optional, default None) –

    Initial guess for fitting ASDs. If None, sets to p0=[100.0,0.5,0.01] which corresponds to:

    • white-noise level in pA/rtHz

    • exponent of 1/f^n component

    • knee

    • frequency in Hz

Returns:

popt – The fit parameters - [white_noise_level, n, f_knee].

Return type:

float array

sodetlib.noise.get_noise_params(am, wl_f_range=(10, 30), fit=False, nperdecade=10, **asd_args)

Function to calculate the ASD from an axis manager and then calculate the white noise, and fknee (and n-index of 1/f^{n/2} if fit=True). The fit=True option is a lot slower.

Parameters:

  • am (AxisManager) – axis manager loaded using G3tSmurf with timestamps and signal keys.

  • wl_f_range (float tuple) – tuple contains (f_low, f_high), if fit=True see fit_noise_ASD. The white noise is calculated as the median of the ASD (pA/rtHz) between f_low and f_high.

  • fit (bool) – if true will fit the ASD using fit_noise_ASD function

  • nperdecade (int) – number of bins per decade (i.e. between 0.01 to 0.1 or 0.1 to 1) to use to average the ASD over to avoid peaks skewing the search for fknee when not using the fit=True option. If nperdecade = 10 for example then the bins between 0.01 and 1 would be: np.concatenate((np.linspace(0.01,0.1,10),np.linspace(0.1,1,10)[1:]))

Returns:

outdict – dictionary that contains all of the calculated noise parameters by channel, band averaged white noise levels, and f, axx ndarrays from the calculated ASD the keys are:

noise_pars: ndarray

shape is [nchans,3] the 3 items in axis=1 are: 0 = white noise, 1 = n (1/f index nan if fit=False), and 2 = fknee.

band_medians: ndarray

shape is [8,1] median white noise level for each band.

f: ndarray

frequency array from welch periodogram

axx: ndarray

square root of welch output PSD shape is [nchans,len(f)]

bincenters: ndarray

Frequencies where the low frequency spectrum is binned when using Fit = False, if Fit=True then just the array of frequencies that can be used to plot the fit results in lowfn.

lowfn: ndarray

shape = [nchans, len(bincenters)]. Binned noise levels (if Fit=False) or fit noise levels (if Fit=`True`) for the low frequency part of the ASD.

low_f_10mHz: ndarray

shape = [nchans].Ratio of the ASD at 10mHz relative to a reference low-f component that has a white noise component = 65 pA/rtHz and low-f scaling of 1/f^{1/2}.

Return type:

dict

sodetlib.noise.noise_model(freq, wl, n, f_knee)

Crude model for noise modeling

Parameters:

  • freq (float) – independent variable in function (frequency Hz)

  • wl (float) – White-noise level.

  • n (float) – Exponent of 1/f^(n/2) component.

  • f_knee (float) – Frequency at which white noise = 1/f^n component

Returns:

y – dependent variable in function noise in (pA/rtHz)

Return type:

float

sodetlib.noise.plot_band_noise(am, nbins=40, noisedict=None, wl_f_range=(10, 30), fit=False, nperdecade=10, show_plot=True, save_plot=False, save_dir=None, **asd_args)

Makes a summary plot w/ subplots per band of a histogram of the white noise levels and another plot with histograms of the fknees. If an axis AxisManager is passed without a noisedict then get_noise_params will be called to generate a noisedict otherwise those parameters will be skipped.

Parameters:

  • am (AxisManager) – axis manager loaded using G3tSmurf with timestamps and signal keys to be analyzed.

  • nbins (int) – number of bins in the histograms.

  • noisedict (dict) – dictionary returned by get_noise_params see that docstring for details on dictionary keys.

  • wl_f_range (float tuple) – tuple contains (f_low, f_high), if fit=True see fit_noise_asd. The white noise is calculated as the median of the ASD (pA/rtHz) between f_low and f_high.

  • fit (bool) – if true will fit the ASD using fit_noise_asd function

  • nperdecade (int) – used to calculate fknee, see get_noise_params doc string.

  • show_plot (bool) – plot only displayed if true.

  • save_plot (bool) – plot only saved if true. If true then save_dir is required to save properly otherwise will just return without saving.

  • save_dir (str) – directory where plots are saved. Required if save_plot is True.

Returns:

  • fig_wnl (matplotlib.figure.Figure) – matplotlib figure object for white noise plot

  • axes_wnl (matplotlib.axes.Axes) – matplotlib axes object for white noise plot

  • fig_fk (matplotlib.figure.Figure) – matplotlib figure object for fknee plot

  • axes_fk (matplotlib.axes.Axes) – matplotlib axes object for fknee plot

sodetlib.noise.plot_channel_noise(am, rc, save_dir=None, noisedict=None, wl_f_range=(10, 30), fit=False, show_plot=False, save_plot=False, nperdecade=10, plot1overfregion=False, **asd_args)

Function for plotting the tod and psd with white noise and fknee identified for a single channel.

Parameters:

  • am (sotodlib.core.AxisManager) – axis manager loaded using G3tSmurf with timestamps and signal keys to be analyzed.

  • rc (int) – Readout channel (i.e. index of am.signal) to plot.

  • noisedict (dict) – dictionary returned by get_noise_params see that docstring for details on dictionary keys.

  • wl_f_range (float tuple) – tuple contains (f_low, f_high), if fit=True see fit_noise_asd. The white noise is calculated as the median of the ASD (pA/rtHz) between f_low and f_high.

  • fit (bool) – if true will fit the ASD using fit_noise_asd function

  • show_plot (bool) – plot only displayed if true.

  • save_plot (bool) – plot only saved if true.

  • nperdecade (int) – used to calculate fknee, see get_noise_params doc string.

  • plot1overfregion (bool) – if true plots a line and shaded region that represents the SO passing low-f requirement (i.e. fknee set by wl = 65pA/rtHz and slope must be <= 1/f^{1/2} in the ASD)

Returns:

  • fig (matplotlib.figure.Figure) – matplotlib figure object for plot

  • axes (matplotlib.axes.Axes) – matplotlib axes object for plot

sodetlib.noise.plot_noise_all(res, range=(0, 200), text_loc=(0.4, 0.7))

Plots the white noise distribution of all bands together.

Parameters:

  • res (dict) – Result from the take_noise function

  • range (tuple) – Range of the histogram

  • text_loc (tuple) – Location to place the textbox containing median and file info. These coordinates are in the axis tranform frame with (0, 0) being the bottom left and (1, 1) being the top right.

sodetlib.noise.take_noise(S, cfg, acq_time=30, plot_band_summary=True, nbins=40, show_plot=True, save_plot=True, plotted_rchans=None, wl_f_range=(10, 30), fit=False, nperdecade=10, plot1overfregion=False, save_dir=None, g3_tag=None, **asd_args)

Streams data for specified amount of time and then calculated the ASD and calculates the white noise levels and fknees for all channels. Optionally the band medians of the fitted parameters can be plotted and/or individual channel plots of the TOD and ASD with white noise and fknee called out.

Parameters:

  • S (pysmurf.client.base.smurf_control.SmurfControl) – pysmurf control object

  • cfg (sodetlib.det_config.DetConfig) – detconfig object

  • acq_time (float) – acquisition time for the noise timestream.

  • plot_band_summary (bool) – if true will plot band summary of white noise and fknees.

  • show_plot (bool) – if true will display plots.

  • plotted_rchans (int list) – list of readout channels (i.e. index of am.signal) to make channel plots for.

  • wl_f_range (float tuple) – tuple contains (f_low, f_high), if fit=True see fit_noise_asd. The white noise is calculated as the median of the ASD (pA/rtHz) between f_low and f_high.

  • fit (bool) – if true will fit the ASD using fit_noise_asd function

  • nperdecade (int) – used to calculate fknee, see get_noise_params doc string.

  • plot1overfregion (bool) – if true plots a line and shaded region that represents the SO passing low-f requirement (i.e. fknee set by wl = 65pA/rtHz and slope must be <= 1/f^{1/2} in the ASD)

  • g3_tag (string, optional) – Tag to be attached to g3 stream

Returns:

  • am (AxisManager) – AxisManager from the timestream acquired to calculate noise parameters.

  • outdict (dict) – dictionary that contains all calculated noise parameters and figure and axes objects for all plots generated. The keys are:

    ’noisedict’:

    dictionary returned by get_noise_params see that doc string

    ’fig_wnl’:

    matplotlib figure object for white noise band summary plot Only returned if plot_band_summary is True.

    ’axes_wnl’:

    matplotlib axes object for white noise band summary plot Only returned if plot_band_summary is True.

    ’fig_fk’:

    matplotlib figure object for fknee band summary plot Only returned if plot_band_summary is True.

    ’axes_fk’:

    matplotlib axes object for fknee band summary plot Only returned if plot_band_summary is True.

    ’channel_plots’:

    nested dictionary that has a key for each readout channel in the plotted_rchans list and contains a matplotlib figure and axis for each readout channel. Only returned if plot_channel_noise is True.