Configuration¶
This page describes the configuration management scheme used within the Fermipy package and the documents the configuration parameters that can be set in the configuration file.
Class Configuration¶
Classes in the Fermipy package follow a common convention for configuring the runtime behavior of a class instance. Internally every class instance has a dictionary that defines its configuration state. Elements of the configuration dictionary can be scalars (str, int, float) or dictionaries defining nested groups of parameters.
The class configuration dictionary is initialized at the time of object creation by passing a dictionary or a path to YAML configuration file to the class constructor. Keyword arguments can be passed to the constructor to override configuration parameters in the input dictionary. For instance in the following example the config dictionary defines values for the parameters emin and emax. By passing a dictionary for the selection keyword argument, the value of emax in the keyword argument (10000) overrides the value of this parameter in the input dictionary.
config = {
'selection' : { 'emin' : 100,
'emax' : 1000 }
}
gta = GTAnalysis(config,selection={'emax' : 10000})
The first argument can also be the path to a YAML configuration file rather than a dictionary:
gta = GTAnalysis('config.yaml',selection={'emax' : 10000})
Configuration File¶
Fermipy uses YAML files to read and write its configuration in a persistent format. The configuration file has a hierarchical organization that groups parameters into dictionaries that are keyed to a section name (data, binnig, etc.).
data:
evfile : ft1.lst
scfile : ft2.fits
ltfile : ltcube.fits
binning:
roiwidth : 10.0
binsz : 0.1
binsperdec : 8
selection :
emin : 100
emax : 316227.76
zmax : 90
evclass : 128
evtype : 3
tmin : 239557414
tmax : 428903014
filter : null
target : 'mkn421'
gtlike:
edisp : True
irfs : 'P8R2_SOURCE_V6'
edisp_disable : ['isodiff','galdiff']
model:
src_roiwidth : 15.0
galdiff : '$FERMI_DIFFUSE_DIR/gll_iem_v06.fits'
isodiff : 'iso_P8R2_SOURCE_V6_v06.txt'
catalogs : ['3FGL']
The configuration file mirrors the layout of the configuration dictionary. Most of the configuration parameters are optional and if not set explicitly in the configuration file will be set to a default value. The parameters that can be set in each section are described below.
binning¶
Options in the binning section control the spatial and spectral binning of the data.
binning:
# Binning
roiwidth : 10.0
npix : null
binsz : 0.1 # spatial bin size in deg
binsperdec : 8 # nb energy bins per decade
projtype : WCS
Option | Default | Description |
---|---|---|
binsperdec |
8 | Number of energy bins per decade. |
binsz |
0.1 | Spatial bin size in degrees. |
coordsys |
CEL | Coordinate system of the spatial projection (CEL or GAL). |
enumbins |
None | Number of energy bins. If none this will be inferred from energy range and binsperdec parameter. |
hpx_ebin |
True | Include energy binning |
hpx_order |
10 | Order of the map (int between 0 and 12, included) |
hpx_ordering_scheme |
RING | HEALPix Ordering Scheme |
npix |
None | Number of pixels. If none then this will be set from roiwidth and binsz . |
proj |
AIT | Spatial projection for WCS mode. |
projtype |
WCS | Projection mode (WCS or HPX). |
roiwidth |
10.0 | Width of the ROI in degrees. The number of pixels in each spatial dimension will be set from roiwidth / binsz (rounded up). |
components¶
The components section can be used to define analysis configurations for a sequence of independent subselections of the data. Each subselection will have its own binned likelihood instance that will be combined in a global likelihood function for the whole ROI (implemented with the SummedLikelihood class in pyLikelihood). This section is optional and when this section is empty (the default) fermiPy will construct a single likelihood with the parameters of the root analysis configuration.
The component section is defined as a list of dictionaries where each element sets analysis parameters for a different subcomponent of the analysis. Dictionary elements have the same hierarchy of parameters as the root analysis configuration. Parameters not defined in a given element will default to the values set in the root analysis configuration.
The following example illustrates how to define a Front/Back analysis with the a list of dictionaries. Files associated to each component will be given a suffix according to their order in the list (e.g. file_00.fits, file_01.fits, etc.).
# Component section for Front/Back analysis
- { selection : { evtype : 1 } } # Front
- { selection : { evtype : 2 } } # Back
data¶
The data section defines the input data files for the analysis (FT1,
FT2, and livetime cube). evfile
and scfile
can either be
individual files or group of files. The optional ltcube
option can
be used to choose a pre-generated livetime cube. If ltcube
is
null a livetime cube will be generated at runtime with gtltcube
.
data :
evfile : ft1.lst
scfile : ft2.fits
ltcube : null
Option | Default | Description |
---|---|---|
cacheft1 |
True | Cache FT1 files when performing binned analysis. If false then only the counts cube is retained. |
evfile |
None | Path to FT1 file or list of FT1 files. |
ltcube |
None | Path to livetime cube. If none a livetime cube will be generated with gtmktime . |
scfile |
None | Path to FT2 (spacecraft) file. |
extension¶
The options in extension control the default behavior of the
extension
method. For more information
about using this method see the Extension Fitting page.
Option | Default | Description |
---|---|---|
fit_position |
False | Perform a simultaneous fit to the source position and extension. |
free_background |
False | Leave background parameters free when performing the fit. If True then any parameters that are currently free in the model will be fit simultaneously with the source of interest. |
free_radius |
None | Free normalizations of background sources within this angular distance in degrees from the source of interest. If None then no sources will be freed. |
make_plots |
False | Generate diagnostic plots. |
psf_scale_fn |
None | Tuple of two vectors (logE,f) defining an energy-dependent PSF scaling function that will be applied when building spatial models for the source of interest. The tuple (logE,f) defines the fractional corrections f at the sequence of energies logE = log10(E/MeV) where f=0 corresponds to no correction. The correction function f(E) is evaluated by linearly interpolating the fractional correction factors f in log(E). The corrected PSF is given by P’(x;E) = P(x/(1+f(E));E) where x is the angular separation. |
save_model_map |
False | Save model counts cubes for the best-fit model of extension. |
spatial_model |
RadialGaussian | Spatial model that will be used to test the sourceextension. The spatial scale parameter of the model will be set such that the 68% containment radius of the model is equal to the width parameter. |
sqrt_ts_threshold |
None | Threshold on sqrt(TS_ext) that will be applied when update is True. If None then nothreshold is applied. |
update |
False | Update this source with the best-fit model for spatial extension if TS_ext > tsext_threshold . |
width |
None | Sequence of values in degrees for the likelihood scan over spatial extension (68% containment radius). If this argument is None then the scan points will be determined from width_min/width_max/width_nstep. |
width_max |
1.0 | Maximum value in degrees for the likelihood scan over spatial extent. |
width_min |
0.01 | Minimum value in degrees for the likelihood scan over spatial extent. |
width_nstep |
21 | Number of scan points between width_min and width_max. Scan points will be spaced evenly on a logarithmic scale between width_min and width_max . |
write_fits |
True | Write the output to a FITS file. |
write_npy |
True | Write the output dictionary to a numpy file. |
fileio¶
The fileio section collects options related to file bookkeeping.
The outdir
option sets the root directory of the analysis instance
where all output files will be written. If outdir
is null then the
output directory will be automatically set to the directory in which
the configuration file is located. Enabling the usescratch
option
will stage all output data files to a temporary scratch directory
created under scratchdir
.
fileio:
outdir : null
logfile : null
usescratch : False
scratchdir : '/scratch'
Option | Default | Description |
---|---|---|
logfile |
None | Path to log file. If None then log will be written to fermipy.log. |
outdir |
None | Path of the output directory. If none this will default to the directory containing the configuration file. |
outdir_regex |
[‘\.fits$|\.fit$|\.xml$|\.npy$|\.png$|\.pdf$|\.yaml$’] | Stage files to the output directory that match at least one of the regular expressions in this list. This option only takes effect when usescratch is True. |
savefits |
True | Save intermediate FITS files. |
scratchdir |
/scratch | Path to the scratch directory. If usescratch is True then a temporary working directory will be created under this directory. |
usescratch |
False | Run analysis in a temporary working directory under scratchdir . |
workdir |
None | Path to the working directory. |
workdir_regex |
[‘\.fits$|\.fit$|\.xml$|\.npy$’] | Stage files to the working directory that match at least one of the regular expressions in this list. This option only takes effect when usescratch is True. |
gtlike¶
Options in the gtlike section control the setup of the likelihood
analysis include the IRF name (irfs
).
Option | Default | Description |
---|---|---|
bexpmap |
None | |
bexpmap_roi |
None | |
convolve |
True | |
edisp |
True | Enable the correction for energy dispersion. |
edisp_disable |
None | Provide a list of sources for which the edisp correction should be disabled. |
expscale |
None | Exposure correction that is applied to all sources in the analysis component. This correction is superseded by src_expscale if it is defined for a source. |
irfs |
None | Set the IRF string. |
llscan_npts |
20 | Number of evaluation points to use when performing a likelihood scan. |
minbinsz |
0.05 | Set the minimum bin size used for resampling diffuse maps. |
resample |
True | |
rfactor |
2 | |
src_expscale |
None | Dictionary of exposure corrections for individual sources keyed to source name. The exposure for a given source will be scaled by this value. A value of 1.0 corresponds to the nominal exposure. |
srcmap |
None | Set the source maps file. When defined this file will be used instead of the local source maps file. |
use_external_srcmap |
False | Use an external precomputed source map file. |
wmap |
None | Likelihood weights map. |
lightcurve¶
The options in lightcurve control the default behavior of the
lightcurve
method. For more information
about using this method see the Light Curves page.
Option | Default | Description |
---|---|---|
binsz |
86400.0 | Set the lightcurve bin size in seconds. |
free_background |
False | Leave background parameters free when performing the fit. If True then any parameters that are currently free in the model will be fit simultaneously with the source of interest. |
free_radius |
None | Free normalizations of background sources within this angular distance in degrees from the source of interest. If None then no sources will be freed. |
free_sources |
None | List of sources to be freed. These sources will be added to the list of sources satisfying the free_radius selection. |
make_plots |
False | Generate diagnostic plots. |
nbins |
None | Set the number of lightcurve bins. The total time range will be evenly split into this number of time bins. |
time_bins |
None | Set the lightcurve bin edge sequence in MET. This option takes precedence over binsz and nbins. |
use_local_ltcube |
True | |
write_fits |
True | Write the output to a FITS file. |
write_npy |
True | Write the output dictionary to a numpy file. |
model¶
The model section collects options that control the inclusion of
point-source and diffuse components in the model. galdiff
and
isodiff
set the templates for the Galactic IEM and isotropic
diffuse respectively. catalogs
defines a list of catalogs that
will be merged to form a master analysis catalog from which sources
will be drawn. Valid entries in this list can be FITS files or XML
model files. sources
can be used to insert additional
point-source or extended components beyond those defined in the master
catalog. src_radius
and src_roiwidth
set the maximum distance
from the ROI center at which sources in the master catalog will be
included in the ROI model.
model :
# Diffuse components
galdiff : '$FERMI_DIR/refdata/fermi/galdiffuse/gll_iem_v06.fits'
isodiff : '$FERMI_DIR/refdata/fermi/galdiffuse/iso_P8R2_SOURCE_V6_v06.txt'
# List of catalogs to be used in the model.
catalogs :
- '3FGL'
- 'extra_sources.xml'
sources :
- { 'name' : 'SourceA', 'ra' : 60.0, 'dec' : 30.0, 'SpectrumType' : PowerLaw }
- { 'name' : 'SourceB', 'ra' : 58.0, 'dec' : 35.0, 'SpectrumType' : PowerLaw }
# Include catalog sources within this distance from the ROI center
src_radius : null
# Include catalog sources within a box of width roisrc.
src_roiwidth : 15.0
Option | Default | Description |
---|---|---|
assoc_xmatch_columns |
[‘3FGL_Name’] | Choose a set of association columns on which to cross-match catalogs. |
catalogs |
None | |
diffuse |
None | |
diffuse_dir |
None | |
extdir |
None | Set a directory that will be searched for extended source FITS templates. Template files in this directory will take precendence over catalog source templates with the same name. |
extract_diffuse |
False | Extract a copy of all mapcube components centered on the ROI. |
galdiff |
None | Set the path to one or more galactic IEM mapcubes. A separate component will be generated for each item in this list. |
isodiff |
None | Set the path to one or more isotropic templates. A separate component will be generated for each item in this list. |
limbdiff |
None | |
merge_sources |
True | Merge properties of sources that appear in multiple source catalogs. If merge_sources=false then subsequent sources with the same name will be ignored. |
sources |
None | |
src_radius |
None | Radius of circular region in degrees centered on the ROI that selects sources for inclusion in the model. If this parameter is none then no selection is applied. This selection is ORed with the src_roiwidth selection. |
src_radius_roi |
None | Half-width of src_roiwidth selection. This parameter can be used in lieu of src_roiwidth . |
src_roiwidth |
None | Width of square region in degrees centered on the ROI that selects sources for inclusion in the model. If this parameter is none then no selection is applied. This selection will be ORed with the src_radius selection. |
optimizer¶
Option | Default | Description |
---|---|---|
init_lambda |
0.0001 | Initial value of damping parameter for step size calculation when using the NEWTON fitter. A value of zero disables damping. |
max_iter |
100 | Maximum number of iterations for the Newtons method fitter. |
min_fit_quality |
2 | Set the minimum fit quality. |
optimizer |
MINUIT | Set the optimization algorithm to use when maximizing the likelihood function. |
retries |
3 | Set the number of times to retry the fit when the fit quality is less than min_fit_quality . |
tol |
0.001 | Set the optimizer tolerance. |
verbosity |
0 |
plotting¶
Option | Default | Description |
---|---|---|
catalogs |
None | |
cmap |
magma | Set the colormap for 2D plots. |
cmap_resid |
RdBu_r | Set the colormap for 2D residual plots. |
format |
png | |
graticule_radii |
None | Define a list of radii at which circular graticules will be drawn. |
label_ts_threshold |
0.0 | TS threshold for labeling sources in sky maps. If None then no sources will be labeled. |
loge_bounds |
None |
residmap¶
The options in residmap control the default behavior of the
residmap
method. For more
information about using this method see the Residual Map page.
Option | Default | Description |
---|---|---|
exclude |
None | List of sources that will be removed from the model when computing the residual map. |
loge_bounds |
None | Restrict the analysis to an energy range (emin,emax) in log10(E/MeV) that is a subset of the analysis energy range. By default the full analysis energy range will be used. If either emin/emax are None then only an upper/lower bound on the energy range wil be applied. |
make_plots |
False | Generate diagnostic plots. |
model |
None | Dictionary defining the properties of the test source. |
write_fits |
True | Write the output to a FITS file. |
write_npy |
True | Write the output dictionary to a numpy file. |
roiopt¶
The options in roiopt control the default behavior of the
optimize
method. For more
information about using this method see the ROI Optimization and Fitting page.
Option | Default | Description |
---|---|---|
max_free_sources |
5 | Maximum number of sources that will be fit simultaneously in the first optimization step. |
npred_frac |
0.95 | |
npred_threshold |
1.0 | |
shape_ts_threshold |
25.0 | Threshold on source TS used for determining the sources that will be fit in the third optimization step. |
skip |
None | List of str source names to skip while optimizing. |
sed¶
The options in sed control the default behavior of the
sed
method. For more information
about using this method see the SED Analysis page.
Option | Default | Description |
---|---|---|
bin_index |
2.0 | Spectral index that will be use when fitting the energy distribution within an energy bin. |
cov_scale |
3.0 | Scale factor that sets the strength of the prior on nuisance parameters that are free. Setting this to None disables the prior. |
free_background |
False | Leave background parameters free when performing the fit. If True then any parameters that are currently free in the model will be fit simultaneously with the source of interest. |
free_radius |
None | Free normalizations of background sources within this angular distance in degrees from the source of interest. If None then no sources will be freed. |
make_plots |
False | Generate diagnostic plots. |
ul_confidence |
0.95 | Confidence level for flux upper limit. |
use_local_index |
False | Use a power-law approximation to the shape of the global spectrum in each bin. If this is false then a constant index set to bin_index will be used. |
write_fits |
True | Write the output to a FITS file. |
write_npy |
True | Write the output dictionary to a numpy file. |
selection¶
The selection section collects parameters related to the data selection and target definition. The majority of the parameters in this section are arguments to gtselect and gtmktime. The ROI center can be set with the target parameter by providing the name of a source defined in one of the input catalogs (defined in the model section). Alternatively the ROI center can be defined by giving explicit sky coordinates with ra and dec or glon and glat.
selection:
# gtselect parameters
emin : 100
emax : 100000
zmax : 90
evclass : 128
evtype : 3
tmin : 239557414
tmax : 428903014
# gtmktime parameters
filter : 'DATA_QUAL>0 && LAT_CONFIG==1'
roicut : 'no'
# Set the ROI center to the coordinates of this source
target : 'mkn421'
Option | Default | Description |
---|---|---|
convtype |
None | Conversion type selection. |
dec |
None | |
emax |
None | Maximum Energy (MeV) |
emin |
None | Minimum Energy (MeV) |
evclass |
None | Event class selection. |
evtype |
None | Event type selection. |
filter |
None | Filter string for gtmktime selection. |
glat |
None | |
glon |
None | |
logemax |
None | Maximum Energy (log10(MeV)) |
logemin |
None | Minimum Energy (log10(MeV)) |
phasemax |
None | Maximum pulsar phase |
phasemin |
None | Minimum pulsar phase |
ra |
None | |
radius |
None | Radius of data selection. If none this will be automatically set from the ROI size. |
roicut |
no | |
target |
None | Choose an object on which to center the ROI. This option takes precendence over ra/dec or glon/glat. |
tmax |
None | Maximum time (MET). |
tmin |
None | Minimum time (MET). |
zmax |
None | Maximum zenith angle. |
sourcefind¶
The options in sourcefind control the default behavior of the
find_sources
method. For more information
about using this method see the Source Finding page.
Option | Default | Description |
---|---|---|
max_iter |
3 | Set the number of search iterations. |
min_separation |
1.0 | Set the minimum separation in deg for sources added in each iteration. |
model |
None | Set the source model dictionary. By default the test source will be a PointSource with an Index 2 power-law specturm. |
sources_per_iter |
3 | |
sqrt_ts_threshold |
5.0 | Set the threshold on sqrt(TS). |
tsmap_fitter |
tsmap | Set the method for generating the TS map. |
tsmap¶
The options in tsmap control the default behavior of the
tsmap
method. For more information
about using this method see the TS Map page.
Option | Default | Description |
---|---|---|
exclude |
None | List of sources that will be removed from the model when computing the TS map. |
loge_bounds |
None | Restrict the analysis to an energy range (emin,emax) in log10(E/MeV) that is a subset of the analysis energy range. By default the full analysis energy range will be used. If either emin/emax are None then only an upper/lower bound on the energy range wil be applied. |
make_plots |
False | Generate diagnostic plots. |
max_kernel_radius |
3.0 | Set the maximum radius of the test source kernel. Using a smaller value will speed up the TS calculation at the loss of accuracy. |
model |
None | Dictionary defining the properties of the test source. |
multithread |
False | Split the TS map calculation across multiple cores. |
write_fits |
True | Write the output to a FITS file. |
write_npy |
True | Write the output dictionary to a numpy file. |
tscube¶
The options in tscube control the default behavior of the
tscube
method. For more information
about using this method see the TS Cube page.
Option | Default | Description |
---|---|---|
cov_scale |
-1.0 | Scale factor to apply to broadband fitting cov. matrix in bin-by-bin fits ( < 0 -> fixed ) |
cov_scale_bb |
-1.0 | Scale factor to apply to global fitting cov. matrix in broadband fits. ( < 0 -> no prior ) |
do_sed |
True | Compute the energy bin-by-bin fits |
init_lambda |
0 | Initial value of damping parameter for newton step size calculation. A value of zero disables damping. |
max_iter |
30 | Maximum number of iterations for the Newtons method fitter. |
model |
None | Dictionary defining the properties of the test source. By default the test source will be a PointSource with an Index 2 power-law specturm. |
nnorm |
10 | Number of points in the likelihood v. normalization scan |
norm_sigma |
5.0 | Number of sigma to use for the scan range |
remake_test_source |
False | If true, recomputes the test source image (otherwise just shifts it) |
st_scan_level |
0 | Level to which to do ST-based fitting (for testing) |
tol |
0.001 | Critetia for fit convergence (estimated vertical distance to min < tol ) |
tol_type |
0 | Absoulte (0) or relative (1) criteria for convergence. |