Installing the Fermi Science Tools¶

The Fermi STs are a prerequisite for fermipy. To install the STs we recommend using one of the non-ROOT binary distributions available from the FSSC. The following example illustrates how to install the binary distribution on a Linux machine running Ubuntu Trusty:

$ curl -OL http://fermi.gsfc.nasa.gov/ssc/data/analysis/software/tar/ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10-without-rootA.tar.gz
$ tar xzf ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10-without-rootA.tar.gz
$ export FERMI_DIR=ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.19-10-without-rootA/x86_64-unknown-linux-gnu-libc2.19-10
$ source $FERMI_DIR/fermi-init.sh

More information about installing the STs as well as the complete list of the available binary distributions is available on the FSSC software page.

Installing with pip¶

These instructions cover installation with the pip package management tool. This method will install fermipy and its dependencies into the python distribution that comes with the Fermi Science Tools. First verify that you’re running the python from the Science Tools

$ which python

If this doesn’t point to the python in your Science Tools install (i.e. it returns /usr/bin/python or /usr/local/bin/python) then the Science Tools are not properly setup.

Before starting the installation process, you will need to determine whether you have setuptools and pip installed in your local python environment. You may need to install these packages if you are running with the binary version of the Fermi Science Tools distributed by the FSSC. The following command will install both packages in your local environment:

$ curl https://bootstrap.pypa.io/get-pip.py | python -

Check if pip is correctly installed:

$ which pip

Once again, if this isn’t the pip in the Science Tools, something went wrong. Now install fermipy by running

$ pip install fermipy

To run the ipython notebook examples you will also need to install jupyter notebook:

$ pip install jupyter

Finally, check that fermipy imports:

$ python
Python 2.7.8 (default, Aug 20 2015, 11:36:15)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from fermipy.gtanalysis import GTAnalysis
>>> help(GTAnalysis)

Installing with Anaconda Python¶

These instructions cover how to use fermipy with a new or existing conda python installation. These instructions assume that you have already downloaded and installed the Fermi STs from the FSSC and you have set the FERMI_DIR environment variable to point to the location of this installation.

The condainstall.sh script can be used to install fermipy into an existing conda python installation or to create a minimal conda installation from scratch. In either case download and run the condainstall.sh installation script from the fermipy repository:

$ curl -OL https://raw.githubusercontent.com/fermiPy/fermipy/master/condainstall.sh
$ bash condainstall.sh

If you do not already have anaconda python installed on your system this script will create a new installation under $HOME/miniconda. If you already have conda installed and the conda command is in your path the script will use your existing installation. The script will create a separate environment for your fermipy installation called fermi-env.

Once fermipy is installed you can initialize the fermi environment by running condasetup.sh:

$ curl -OL https://raw.githubusercontent.com/fermiPy/fermipy/master/condasetup.sh
$ source condasetup.sh

This will both activate the fermi-env environment and set up your shell environment to run the Fermi Science Tools. The fermi-env python environment can be exited by running:

$ source deactivate

Running at SLAC¶

This section provides specific installation instructions for running in the SLAC computing environment. First download and source the slacsetup.sh script:

$ wget https://raw.githubusercontent.com/fermiPy/fermipy/master/slacsetup.sh -O slacsetup.sh
$ source slacsetup.sh

To initialize the ST environment run the slacsetup function:

$ slacsetup

This will setup your GLAST_EXT path and source the setup script for one of the pre-built ST installations (the current default is 10-01-01). To manually override the ST version you can optionally provide the release tag as an argument to slacsetup:

$ slacsetup XX-XX-XX

Because users don’t have write access to the ST python installation all pip commands that install or uninstall packages must be executed with the --user flag. After initializing the STs environment, install fermipy with pip:

$ pip install fermipy --user

This will install fermipy in $HOME/.local. You can verify that the installation has succeeded by importing GTAnalysis:

$ python
Python 2.7.8 |Anaconda 2.1.0 (64-bit)| (default, Aug 21 2014, 18:22:21)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> from fermipy.gtanalysis import GTAnalysis

Upgrading¶

By default installing fermipy with pip will get the latest tagged released available on the PyPi package respository. You can check your currently installed version of fermipy with pip show:

$ pip show fermipy
---
Metadata-Version: 2.0
Name: fermipy
Version: 0.6.7
Summary: A Python package for analysis of Fermi-LAT data
Home-page: https://github.com/fermiPy/fermipy
Author: The Fermipy developers
Author-email: fermipy.developers@gmail.com
License: BSD
Location: /home/vagrant/miniconda/envs/fermi-env/lib/python2.7/site-packages
Requires: wcsaxes, astropy, matplotlib, healpy, scipy, numpy, pyyaml

To upgrade your fermipy installation to the latest version run the pip installation command with --upgrade --no-deps (remember to also include the --user option if you’re running at SLAC):

$ pip install fermipy --upgrade --no-deps
Collecting fermipy
Installing collected packages: fermipy
  Found existing installation: fermipy 0.6.6
    Uninstalling fermipy-0.6.6:
      Successfully uninstalled fermipy-0.6.6
Successfully installed fermipy-0.6.7

Building from Source¶

These instructions describe how to install fermipy from its git source code repository using setup.py. Installing from source is necessary if you want to do local development or test features in an untagged release. Note that for non-expert users it is recommended to install fermipy with pip following the instructions above. First clone the fermipy repository:

$ git clone https://github.com/fermiPy/fermipy.git
$ cd fermipy

To install the head of the master branch run setup.py install from the root of the source tree:

# Install the latest version
$ git checkout master
$ python setup.py install --user

A useful option if you are doing active code development is to install your working copy as the local installation. This can be done by running setup.py develop:

# Install a link to your source code installation
$ python setup.py develop --user

You can later remove the link to your working copy by running the same command with the --uninstall flag:

# Install a link to your source code installation
$ python setup.py develop --user --uninstall

You also have the option of installing a previous release tag. To see the list of release tags use git tag:

$ git tag
0.4.0
0.5.0
0.5.1
0.5.2
0.5.3
0.5.4
0.6.0
0.6.1

To install a specific release tag, run git checkout with the tag name followed by setup.py install:

# Checkout a specific release tag
$ git checkout X.X.X
$ python setup.py install --user

Issues¶

If you get an error about importing matplotlib (specifically something about the macosx backend) you might change your default backend to get it working. The customizing matplotlib page details the instructions to modify your default matplotlibrc file (you can pick GTK or WX as an alternative). Specifically the TkAgg and macosx backends currently do not work on OSX if you upgrade matplotlib to the version required by fermipy. To get around this issue you can enable the Agg backend at runtime:

>>> import matplotlib
>>> matplotlib.use('Agg')

However this backend does not support interactive plotting.

In some cases the setup.py script will fail to properly install the fermipy package dependecies. If installation fails you can try running a forced upgrade of these packages with pip install --upgrade:

$ pip install --upgrade --user numpy matplotlib scipy astropy pyyaml healpy wcsaxes ipython jupyter

Creating a Configuration File¶

The first step is to compose a configuration file that defines the data selection and analysis parameters. Complete documentation on the configuration file and available options is given in the Configuration page. fermiPy uses the YAML format for its configuration files. The configuration file has a hierarchical organization that groups related parameters into separate dictionaries. In this example we will compose a configuration file for a SOURCE-class analysis of Markarian 421 with FRONT+BACK event types (evtype=3):

data:
  evfile : ft1.lst
  scfile : ft2.fits
  ltcube : ltcube.fits

binning:
  roiwidth   : 10.0
  binsz      : 0.1
  binsperdec : 8

selection :
  emin : 100
  emax : 316227.76
  zmax    : 90
  evclass : 128
  evtype  : 3
  tmin    : 239557414
  tmax    : 428903014
  filter  : null
  target : 'mkn421'

gtlike:
  edisp : True
  irfs : 'P8R2_SOURCE_V6'
  edisp_disable : ['isodiff','galdiff']

model:
  src_roiwidth : 15.0
  galdiff  : '$FERMI_DIFFUSE_DIR/gll_iem_v06.fits'
  isodiff  : 'iso_P8R2_SOURCE_V6_v06.txt'
  catalogs : ['3FGL']

The data section defines the input data set and spacecraft file for the analysis. Here evfile points to a list of FT1 files that encompass the chosen ROI, energy range, and time selection. The parameters in the binning section define the dimensions of the ROI and the spatial and energy bin size. The selection section defines parameters related to the data selection (energy range, zmax cut, and event class/type). The target parameter in this section defines the ROI center to have the same coordinates as the given source. The model section defines parameters related to the ROI model definition (diffuse templates, point sources).

Fermipy gives the user the option to combine multiple data selections into a joint likelihood with the components section. The components section contains a list of dictionaries with the same hierarchy as the root analysis configuration. Each element of the list defines the analysis parameters for an independent sub-selection of the data. Any parameters not defined within the component dictionary default to the value defined in the root configuration. The following example shows the components section that could be appended to the previous configuration to define a joint analysis with four PSF event types:

components:
  - { selection : { evtype : 4  } } # PSF0
  - { selection : { evtype : 8  } } # PSF1
  - { selection : { evtype : 16 } } # PSF2
  - { selection : { evtype : 32 } } # PSF3

Any configuration parameter can be changed with this mechanism. The following example is a configuration in which a different zmax selection and isotropic template is used for each of the four PSF event types:

components:
  - model: {isodiff: isotropic_source_psf0_4years_P8V3.txt}
    selection: {evtype: 4, zmax: 70}
  - model: {isodiff: isotropic_source_psf1_4years_P8V3.txt}
    selection: {evtype: 8, zmax: 75}
  - model: {isodiff: isotropic_source_psf2_4years_P8V3.txt}
    selection: {evtype: 16, zmax: 85}
  - model: {isodiff: isotropic_source_psf3_4years_P8V3.txt}
    selection: {evtype: 32, zmax: 90}

Creating an Analysis Script¶

Once the configuration file has been composed, the analysis is executed by creating an instance of GTAnalysis with the configuration file as its argument and calling its analysis methods. GTAnalysis serves as a wrapper over the underlying pyLikelihood classes and provides methods to fix/free parameters, add/remove sources from the model, and perform a fit to the ROI. For a complete documentation of the available methods you can refer to the fermipy package page.

In the following python examples we show how to initialize and run a basic analysis of a source. First we instantiate a GTAnalysis object with the path to the configuration file and run setup().

from fermipy.gtanalysis import GTAnalysis

gta = GTAnalysis('config.yaml',logging={'verbosity' : 3})
gta.setup()

The setup() method performs the data preparation and response calculations needed for the analysis (selecting the data, creating counts and exposure maps, etc.). Depending on the data selection and binning of the analysis this will often be the slowest step in the analysis sequence. The output of setup() is cached in the analysis working directory so subsequent calls to setup() will run much faster.

Before running any other analysis methods it is recommended to first run optimize():

gta.optimize()

This will loop over all model components in the ROI and fit their normalization and spectral shape parameters. This method also computes the TS of all sources which can be useful for identifying weak sources that could be fixed or removed from the model. We can check the results of the optimization step by calling print_roi():

gta.print_roi()

By default all models parameters are initially fixed. The free_source() and free_sources() methods can be use to free or fix parameters of the model. In the following example we free the normalization of catalog sources within 3 deg of the ROI center and free the galactic and isotropic components by name.

# Free Normalization of all Sources within 3 deg of ROI center
gta.free_sources(distance=3.0,pars='norm')

# Free all parameters of isotropic and galactic diffuse components
gta.free_source('galdiff')
gta.free_source('isodiff')

The minmax_ts and minmax_npred arguments to free_sources() can be used to free or fixed sources on the basis of their current TS or Npred values:

# Free sources with TS > 10
gta.free_sources(minmax_ts=[10,None],pars='norm')

# Fix sources with TS < 10
gta.free_sources(minmax_ts=[None,10],free=False,pars='norm')

# Fix sources with 10 < Npred < 100
gta.free_sources(minmax_npred=[10,100],free=False,pars='norm')

When passing a source name argument both case and whitespace are ignored. When using a FITS catalog file a source can also be referred to by any of its associations. When using the 3FGL catalog, the following calls are equivalent ways of freeing the parameters of Mkn 421:

# These calls are equivalent
gta.free_source('mkn421')
gta.free_source('Mkn 421')
gta.free_source('3FGL J1104.4+3812')
gta.free_source('3fglj1104.4+3812')

After freeing parameters of the model we can execute a fit by calling fit(). The will maximize the likelihood with respect to the model parameters that are currently free.

gta.fit()

After the fitting is complete we can write the current state of the model with write_roi:

gta.write_roi('fit_model')

This will write several output files including an XML model file and an ROI dictionary file. The names of all output files will be prepended with the prefix argument to write_roi().

Once we have optimized our model for the ROI we can use the residmap() and tsmap() methods to assess the fit quality and look for new sources.

# Dictionary defining the spatial/spectral parameters of the test source
model = {'SpatialModel' : 'PointSource', 'Index' : 2.0,
         'SpectrumType' : 'PowerLaw'}

# Both methods return a dictionary with the maps
m0 = gta.residmap('fit_model',model=model)
m1 = gta.tsmap('fit_model',model=model)

More documentation about these methods is available in the Source Detection page.

By default, calls to fit() will execute a global spectral fit over the entire energy range of the analysis. To extract a bin-by-bin flux spectrum (i.e. a SED) you can call sed() method with the name of the source:

gta.sed('mkn421')

More information about sed() method can be found in the SED Analysis page.

Extracting Analysis Results¶

Results of the analysis can be extracted from the dictionary file written by write_roi(). This method writes information about the current state of the analysis to a python dictionary. More documentation on the contents of the output file are available in the Output File page.

By default the output dictionary is written to a file in the numpy format and can be loaded from a python session after your analysis is complete. The following demonstrates how to load the analysis dictionary that was written to fit_model.npy in the Mkn421 analysis example:

>>> # Load analysis dictionary from a npy file
>>> import np
>>> c = np.load('fit_model.npy').flat[0]
>>> print(c.keys())
['roi', 'config', 'sources', 'version']

The output dictionary contains the following top-level elements:

Each source dictionary collects the properties of the given source (TS, NPred, best-fit parameters, etc.) computed up to that point in the analysis.

>>> print c['sources'].keys()
['3FGL J1032.7+3735',
'3FGL J1033.2+4116',
...
'3FGL J1145.8+4425',
'galdiff',
'isodiff']
>>> print c['sources']['3FGL J1104.4+3812']['ts']
87455.9709683
>>> print c['sources']['3FGL J1104.4+3812']['npred']
31583.7166495

Information about individual sources in the ROI is also saved to a catalog FITS file with the same string prefix as the dictionary file. This file can be loaded with the astropy.io.fits or astropy.table.Table interface:

>>> # Load the source catalog file
>>> from astropy.table import Table
>>> tab = Table.read('fit_model.fits')
>>> print(tab[['name','class','ts','npred','flux']])
    name       class       ts           npred                    flux [2]
                                                               1 / (cm2 s)
----------------- ----- -------------- ------------- --------------------------------------
3FGL J1104.4+3812   BLL  87455.9709683 31583.7166495 2.20746290445e-07 .. 1.67062058528e-09
3FGL J1109.6+3734   bll    42.34511826 93.7971922425  5.90635786943e-10 .. 3.6620894143e-10
...
3FGL J1136.4+3405  fsrq  4.78089819776 261.427034151 1.86805869704e-08 .. 8.62638727067e-09
3FGL J1145.8+4425  fsrq  3.78006883967 237.525501441 7.25611442299e-08 .. 3.77056557247e-08

The FITS file contains columns for all scalar and vector elements of the source dictionary. Spectral fit parameters are contained in the param_names, param_values, and param_errors columns:

>>> print(tab[['param_names','param_values','param_errors']][0])
<Row 0 of table
 values=(['Prefactor', 'Index', 'Scale', '', '', ''],
         [2.1301351784512767e-11, -1.7716399431228638, 1187.1300048828125, nan, nan, nan],
         [1.6126233510314277e-13, nan, nan, nan, nan, nan])
 dtype=[('param_names', 'S32', (6,)),
        ('param_values', '>f8', (6,)),
        ('param_errors', '>f8', (6,))]>

Reloading from a Previous State¶

One can reload an analysis instance that was saved with write_roi() by calling either the create() or load_roi() methods. The create() method can be used to construct an entirely new instance of GTAnalysis from a previously saved results file:

from fermipy.gtanalysis import GTAnalysis
gta = GTAnalysis.create('fit_model.npy')

# Continue running analysis starting from the previously saved
# state
gta.fit()

where the argument is the path to an output file produced with write_roi(). This function will instantiate a new analysis object, run the setup() method, and load the state of the model parameters at the time that write_roi() was called.

The load_roi() method can be used to reload a previous state of the analysis to an existing instance of GTAnalysis.

from fermipy.gtanalysis import GTAnalysis

gta = GTAnalysis('config.yaml')
gta.setup()

gta.write_roi('prefit_model')

# Fit a source
gta.free_source('mkn421')
gta.fit()

# Restore the analysis to its prior state before the fit of mkn421
# was executed
gta.load_roi('prefit_model')

Using load_roi() is generally faster than create() when an analysis instance already exists.

IPython Notebook Tutorials¶

Additional tutorials with more detailed examples are available as IPython notebooks in the notebooks directory of the fermipy-extra respository. These notebooks can be browsed as static web pages or run interactively by downloading the fermipy-extra repository and running jupyter notebook in the notebooks directory:

$ git clone https://github.com/fermiPy/fermipy-extra.git
$ cd fermipy-extra/notebooks
$ jupyter notebook index.ipynb

Note that this will require you to have both ipython and jupyter installed in your python environment. These can be installed in a conda- or pip-based installation as follows:

# Install with conda
$ conda install ipython jupyter

# Install with pip
$ pip install ipython jupyter

Class Configuration¶

Classes in the fermiPy package follow a common convention for configuring the runtime behavior of a class instance. Internally every class instance has a dictionary that defines its configuration state. Elements of the configuration dictionary can be scalars (str, int ,float) or dictionaries defining nested blocks of the configuration.

The class configuration dictionary is initialized at the time of object creation by passing a dictionary or a path to YAML configuration file to the class constructor. Keyword arguments can be optionally passed to the constructor to override configuration parameters in the input dictionary. For instance in the following example the config dictionary defines values for the parameters emin and emax. By passing a dictionary for the selection keyword argument, the value of emax in the keyword argument (10000) overrides the value of this parameter in the input dictionary.

config = {
'selection' : { 'emin' : 100,
                'emax' : 1000 }
}

gta = GTAnalysis(config,selection={'emax' : 10000})

The first argument can also be the path to a YAML configuration file rather than a dictionary:

gta = GTAnalysis('config.yaml',selection={'emax' : 10000})

Configuration File¶

fermiPy uses YAML files to read and write its configuration in a persistent format. The configuration file has a hierarchical organization that groups parameters into dictionaries that are keyed to a section name (data, binnig, etc.).

data:
  evfile : ft1.lst
  scfile : ft2.fits
  ltfile : ltcube.fits

binning:
  roiwidth   : 10.0
  binsz      : 0.1
  binsperdec : 8

selection :
  emin : 100
  emax : 316227.76
  zmax    : 90
  evclass : 128
  evtype  : 3
  tmin    : 239557414
  tmax    : 428903014
  filter  : null
  target : 'mkn421'

gtlike:
  edisp : True
  irfs : 'P8R2_SOURCE_V6'
  edisp_disable : ['isodiff','galdiff']

model:
  src_roiwidth : 15.0
  galdiff  : '$FERMI_DIFFUSE_DIR/gll_iem_v06.fits'
  isodiff  : 'iso_P8R2_SOURCE_V6_v06.txt'
  catalogs : ['3FGL']

The configuration file mirrors the layout of the configuration dictionary. Most of the available configuration parameters are optional and if not set explicitly in the configuration file will be set to a default value. The parameters that can be set in each section are described below.

binning:

  # Binning
  roiwidth   : 10.0
  npix       : null
  binsz      : 0.1 # spatial bin size in deg
  binsperdec : 8   # nb energy bins per decade
  projtype   : WCS

# Component section for Front/Back analysis with list style
components:
  - { selection : { evtype : 1 } } # Front
  - { selection : { evtype : 2 } } # Back

# Component section for Front/Back analysis with dictionary style
components:
  front : { selection : { evtype : 1 } } # Front
  back  : { selection : { evtype : 2 } } # Back

data :
  evfile : ft1.lst
  scfile : ft2.fits
  ltcube : null

fileio:
   outdir : null
   logfile : null
   usescratch : False
   scratchdir  : '/scratch'

model :

  # Diffuse components
  galdiff  : '$FERMI_DIR/refdata/fermi/galdiffuse/gll_iem_v06.fits'
  isodiff  : '$FERMI_DIR/refdata/fermi/galdiffuse/iso_P8R2_SOURCE_V6_v06.txt'

  # List of catalogs to be used in the model.
  catalogs :
    - '3FGL'
    - 'extra_sources.xml'

  sources :
    - { 'name' : 'SourceA', 'ra' : 60.0, 'dec' : 30.0, 'SpectrumType' : PowerLaw }
    - { 'name' : 'SourceB', 'ra' : 58.0, 'dec' : 35.0, 'SpectrumType' : PowerLaw }

  # Include catalog sources within this distance from the ROI center
  src_radius  : null

  # Include catalog sources within a box of width roisrc.
  src_roiwidth : 15.0

selection:

  # gtselect parameters
  emin    : 100
  emax    : 100000
  zmax    : 90
  evclass : 128
  evtype  : 3
  tmin    : 239557414
  tmax    : 428903014

  # gtmktime parameters
  filter : 'DATA_QUAL>0 && LAT_CONFIG==1'
  roicut : 'no'

  # Set the ROI center to the coordinates of this source
  target : 'mkn421'

ROI Dictionary¶

Source Dictionary¶

The sources dictionary contains one element per source keyed to the source name. The following table lists the elements of the source dictionary and their descriptions.

Fitting¶

fit is a wrapper on the pyLikelihood fit method and performs a likelihood fit of all free parameters of the model. This method can be used to manually optimize of the model by calling it after freeing one or more source parameters. The following example demonstrates the commands that would be used to fit the normalizations of all sources within 3 deg of the ROI center:

>>> gta.free_sources(distance=2.0,pars='norm')
>>> gta.print_params(True)
 idx parname                  value     error       min       max     scale free
--------------------------------------------------------------------------------
3FGL J1104.4+3812
  18 Prefactor                 1.77         0     1e-05       100     1e-11    *
3FGL J1109.6+3734
  24 Prefactor                 0.33         0     1e-05       100     1e-14    *
galdiff
  52 Prefactor                    1         0       0.1        10         1    *
isodiff
  55 Normalization                1         0     0.001     1e+03         1    *
>>> o = gta.fit()
2016-04-19 14:07:55 INFO     GTAnalysis.fit(): Starting fit.
2016-04-19 14:08:56 INFO     GTAnalysis.fit(): Fit returned successfully.
2016-04-19 14:08:56 INFO     GTAnalysis.fit(): Fit Quality: 3 LogLike:   -77279.869 DeltaLogLike:      501.128
>>> gta.print_params(True)
2016-04-19 14:10:02 INFO     GTAnalysis.print_params():
 idx parname                  value     error       min       max     scale free
--------------------------------------------------------------------------------
3FGL J1104.4+3812
  18 Prefactor                 2.13    0.0161     1e-05       100     1e-11    *
3FGL J1109.6+3734
  24 Prefactor                0.342    0.0904     1e-05       100     1e-14    *
galdiff
  52 Prefactor                0.897    0.0231       0.1        10         1    *
isodiff
  55 Normalization             1.15     0.016     0.001     1e+03         1    *

By default fit will repeat the fit until a fit quality of 3 is obtained. After the fit returns all sources with free parameters will have their properties (flux, TS, NPred, etc.) updated in the ROIModel instance. The return value of the method is a dictionary containing the following diagnostic information about the fit:

The fit also accepts keyword arguments which can be used to configure its behavior at runtime:

>>> o = gta.fit(min_fit_quality=2,optimizer='NEWMINUIT',reoptimize=True)

ROI Optimization¶

The optimize method performs an automatic optimization of the ROI by fitting all sources with an iterative strategy.

>>> o = gta.optimize()

It is generally good practice to run this method once at the start of your analysis to ensure that all parameters are close to their global likelihood maxima.

Configuring Diffuse Components¶

The simplest configuration uses a single file for the galactic and isotropic diffuse components. By default the galactic diffuse and isotropic components will be named galdiff and isodiff respectively. An alias for each component will also be created with the name of the mapcube or file spectrum. For instance the galactic diffuse can be referred to as galdiff or gll_iem_v06 in the following example.

model:
  src_roiwidth : 10.0
  galdiff  : '$FERMI_DIFFUSE_DIR/gll_iem_v06.fits'
  isodiff  : '$FERMI_DIFFUSE_DIR/isotropic_source_4years_P8V3.txt'
  catalogs : ['gll_psc_v14.fit']

To define two or more galactic diffuse components you can optionally define the galdiff and isodiff parameters as lists. A separate component will be generated for each element in the list with the name galdiffXX or isodiffXX where XX is an integer position in the list.

model:
  galdiff  :
    - '$FERMI_DIFFUSE_DIR/diffuse_component0.fits'
    - '$FERMI_DIFFUSE_DIR/diffuse_component1.fits'

To explicitly set the name of a component you can define any element as a dictionary containing name and file fields:

model:
  galdiff  :
    - { 'name' : 'component0' : 'file' : '$FERMI_DIFFUSE_DIR/diffuse_component0.fits' }
    - { 'name' : 'component1' : 'file' : '$FERMI_DIFFUSE_DIR/diffuse_component1.fits' }

Configuring Source Components¶

The list of sources for inclusion in the ROI model is set by defining a list of catalogs with the catalogs parameter. Catalog files can be in either XML or FITS format. Sources from the catalogs in this list that satisfy either the src_roiwidth or src_radius selections are added to the ROI model. If a source is defined in multiple catalogs the source definition from the last file in the catalogs list takes precedence.

model:

  src_radius: 5.0
  src_roiwidth: 10.0
  catalogs :
    - 'gll_psc_v16.fit'
    - 'extra_sources.xml'

Individual sources can also be defined within the configuration file with the sources parameter. This parameter contains a list of dictionaries that defines the spatial and spectral parameters of each source. The keys of the source dictionary map to the spectral and spatial source properties as they would be defined in the XML model file.

model:
  sources  :
    - { name: 'SourceA', glon : 120.0, glat : -3.0,
     SpectrumType : 'PowerLaw', Index : 2.0, Scale : 1000, Prefactor : !!float 1e-11,
     SpatialModel: 'PointSource' }
    - { name: 'SourceB', glon : 122.0, glat : -3.0,
     SpectrumType : 'LogParabola', norm : !!float 1E-11, Scale : 1000, beta : 0.0,
     SpatialModel: 'PointSource' }

For parameters defined as scalars, the scale and value properties will be assigned automatically from the input value. To set these manually a parameter can also be initialized with a dictionary that explicitly sets the value and scale properties:

model:
  sources  :
    - { name: 'SourceA', glon : 120.0, glat : -3.0,
        SpectrumType : 'PowerLaw', Index : 2.0, Scale : 1000,
        Prefactor : { value : 1.0, scale : !!float 1e-11, free : '0' },
        SpatialModel: 'PointSource' }

Spatial Models¶

Fermipy supports three types of pre-defined spatial models which can be defined by setting the SpatialModel property: PointSource (the default), RadialDisk, and RadialGaussian. The spatial extension of RadialDisk and RadialGaussian can be controlled with the SpatialWidth parameter which sets the 68% containment radius in degrees. Note for ST releases prior to 11-01-01, RadialDisk and RadialGaussian sources will be represented with the SpatialMap type.

model:
  sources  :
    - { name: 'DiskSource', glon : 120.0, glat : 0.0,
     SpectrumType : 'PowerLaw', Index : 2.0, Scale : 1000, Prefactor : !!float 1e-11,
     SpatialModel: 'RadialDisk', SpatialWidth: 1.0 }
    - { name: 'GaussSource', glon : 120.0, glat : 0.0,
     SpectrumType : 'PowerLaw', Index : 2.0, Scale : 1000, Prefactor : !!float 1e-11,
     SpatialModel: 'RadialGaussian', SpatialWidth: 1.0 }

Editing the Model at Runtime¶

The model can be manually editing at runtime with the add_source() and delete_source() methods. Sources can be added either before or after calling setup() as shown in the following example.

from fermipy.gtanalysis import GTAnalysis

gta = GTAnalysis('config.yaml',logging={'verbosity' : 3})

# Remove isodiff from the model
gta.delete_source('isodiff')

# Add SourceA to the model
gta.add_source('SourceA',{ 'glon' : 120.0, 'glat' : -3.0,
                'SpectrumType' : 'PowerLaw', 'Index' : 2.0,
                'Scale' : 1000, 'Prefactor' : 1e-11,
                'SpatialModel' : 'PointSource' })

gta.setup()

# Add SourceB to the model
gta.add_source('SourceB',{ 'glon' : 121.0, 'glat' : -2.0,
                 'SpectrumType' : 'PowerLaw', 'Index' : 2.0,
                 'Scale' : 1000, 'Prefactor' : 1e-11,
                 'SpatialModel' : 'PointSource' })

Sources added before calling setup() will be appended to the XML model definition. Sources added after calling setup() will be created dynamically through the pyLikelihood object creation mechanism.

Submodules¶

fermipy.config module¶

class fermipy.config.ConfigManager[source]¶

Bases: object

static create(configfile)[source]¶

Create a configuration dictionary from a yaml config file. This function will first populate the dictionary with defaults taken from pre-defined configuration files. The configuration dictionary is then updated with the user-defined configuration file. Any settings defined by the user will take precedence over the default settings.

static load(path)[source]¶

class fermipy.config.Configurable(config, **kwargs)[source]¶

Bases: object

The base class provides common facilities like loading and saving configuration state.

config¶

Return the configuration dictionary of this class.

configdir¶

configure(config, **kwargs)[source]¶

classmethod get_config()[source]¶

Return a default configuration dictionary for this class.

print_config(logger, loglevel=None)[source]¶

write_config(outfile)[source]¶

Write the configuration dictionary to an output file.

fermipy.config.cast_config(config, defaults)[source]¶

fermipy.config.create_default_config(defaults)[source]¶

Create a configuration dictionary from a defaults dictionary. The defaults dictionary defines valid configuration keys with default values and docstrings. Each dictionary element should be a tuple or list containing (default value,docstring,type).

fermipy.config.validate_config(config, defaults, section=None)[source]¶

fermipy.defaults module¶

fermipy.defaults.make_default_dict(d)[source]¶

fermipy.gtanalysis module¶

class fermipy.gtanalysis.GTAnalysis(config, **kwargs)[source]¶

Bases: fermipy.config.Configurable, fermipy.sed.SEDGenerator, fermipy.residmap.ResidMapGenerator, fermipy.tsmap.TSMapGenerator, fermipy.tsmap.TSCubeGenerator, fermipy.sourcefind.SourceFinder

High-level analysis interface that manages a set of analysis component objects. Most of the functionality of the Fermipy package is provided through the methods of this class. The class constructor accepts a dictionary that defines the configuration for the analysis. Keyword arguments to the constructor can be used to override parameters in the configuration dictionary.

__delattr__¶

x.__delattr__(‘name’) <==> del x.name

__format__()¶

default object formatter

__getattribute__¶

x.__getattribute__(‘name’) <==> x.name

__hash__¶

__reduce__()¶

helper for pickle

__reduce_ex__()¶

helper for pickle

__repr__¶

__setattr__¶

x.__setattr__(‘name’, value) <==> x.name = value

__sizeof__() → int¶

size of object in memory, in bytes

__str__¶

add_gauss_prior(name, parName, mean, sigma)[source]¶

add_source(name, src_dict, free=False, init_source=True, save_source_maps=True, **kwargs)[source]¶

Add a source to the ROI model. This function may be called either before or after setup.

Parameters:	name (str) – Source name. src_dict (dict or Source object) – Dictionary or source object defining the source properties (coordinates, spectral parameters, etc.). free (bool) – Initialize the source with a free normalization parameter.

add_sources_from_roi(names, roi, free=False, **kwargs)[source]¶

Add multiple sources to the current ROI model copied from another ROI model.

Parameters:	names (list) – List of str source names to add. roi (ROIModel object) – The roi model from which to add sources. free (bool) – Initialize the source with a free normalization paramter.

bowtie(name, fd=None, loge=None)[source]¶

Generate a spectral uncertainty band (bowtie) for the given source. This will create an uncertainty band on the differential flux as a function of energy by propagating the errors on the global fit parameters. Note that this band only reflects the uncertainty for parameters that are currently free in the model.

Parameters:	name (str) – Source name. fd (FluxDensity) – Flux density object. If this parameter is None then one will be created. loge (array-like) – Sequence of energies in log10(E/MeV) at which the flux band will be evaluated.

cleanup()[source]¶

components¶

Return the list of analysis components.

config¶

Return the configuration dictionary of this class.

configdir¶

configure(config, **kwargs)¶

constrain_norms(srcNames, cov_scale=1.0)[source]¶

Constrain the normalizations of one or more sources by adding gaussian priors with sigma equal to the parameter error times a scaling factor.

counts_map()[source]¶

Return a Map representation of the counts map.

Returns:	map
Return type:	Map

static create(infile, config=None)[source]¶

Create a new instance of GTAnalysis from an analysis output file generated with write_roi. By default the new instance will inherit the configuration of the saved analysis instance. The configuration may be overriden by passing a configuration file path with the config argument.

Parameters:	infile (str) – Path to the ROI results file. config (str) – Path to a configuration file. This will override the configuration in the ROI results file.

defaults = {u'sourcefind': {u'max_iter': (3, u'Set the number of search iterations.', <type 'int'>), u'min_separation': (1.0, u'Set the minimum separation in deg for sources added in each iteration.', <type 'float'>), u'tsmap_fitter': (u'tsmap', u'Set the method for generating the TS map.', <type 'str'>), u'sqrt_ts_threshold': (5.0, u'Set the threshold on sqrt(TS).', <type 'float'>), u'model': (None, u'Set the source model dictionary. By default the test source will be a PointSource with an Index 2 power-law specturm.', <type 'dict'>), u'sources_per_iter': (3, u'', <type 'int'>)}, u'roiopt': {u'npred_frac': (0.95, u'', <type 'float'>), u'shape_ts_threshold': (25.0, u'Threshold on source TS used for determining the sources that will be fit in the third optimization step.', <type 'float'>), u'npred_threshold': (1.0, u'', <type 'float'>), u'skip': (None, u'List of str source names to skip while optimizing.', <type 'list'>), u'max_free_sources': (5, u'Maximum number of sources that will be fit simultaneously in the first optimization step.', <type 'int'>)}, u'selection': {u'radius': (None, u'Radius of data selection. If none this will be automatically set from the ROI size.', <type 'float'>), u'tmin': (None, u'Minimum time (MET).', <type 'int'>), u'target': (None, u'Choose an object on which to center the ROI. This option takes precendence over ra/dec or glon/glat.', <type 'str'>), u'glon': (None, u'', <type 'float'>), u'emin': (None, u'Minimum Energy (MeV)', <type 'float'>), u'emax': (None, u'Maximum Energy (MeV)', <type 'float'>), u'tmax': (None, u'Maximum time (MET).', <type 'int'>), u'glat': (None, u'', <type 'float'>), u'filter': (None, u'Filter string for ``gtmktime`` selection.', <type 'str'>), u'logemax': (None, u'Maximum Energy (log10(MeV))', <type 'float'>), u'ra': (None, u'', <type 'float'>), u'evtype': (None, u'Event type selection.', <type 'int'>), u'evclass': (None, u'Event class selection.', <type 'int'>), u'zmax': (None, u'Maximum zenith angle.', <type 'float'>), u'logemin': (None, u'Minimum Energy (log10(MeV))', <type 'float'>), u'dec': (None, u'', <type 'float'>), u'roicut': (u'no', u'', <type 'str'>), u'convtype': (None, u'Conversion type selection.', <type 'int'>)}, u'logging': {u'verbosity': (3, u'', <type 'int'>), u'chatter': (3, u'Set the chatter parameter of the STs.', <type 'int'>)}, u'tsmap': {u'multithread': (False, u'', <type 'bool'>), u'model': (None, u'Dictionary defining the properties of the test source.', <type 'dict'>), u'loge_bounds': (None, u'Lower and upper energy bounds in log10(E/MeV). By default the calculation will be performed over the full analysis energy range.', <type 'list'>), u'max_kernel_radius': (3.0, u'', <type 'float'>)}, u'mc': {u'seed': (None, u'', <type 'int'>)}, u'components': (None, u'', <type 'list'>), u'localize': {u'dtheta_max': (0.3, u'Half-width of the search region in degrees used for the first pass of the localization search.', <type 'float'>), u'nstep': (5, u'Number of steps along each spatial dimension in the refined likelihood scan.', <type 'int'>), u'fix_background': (True, u'Fix background parameters when fitting the source flux in each energy bin.', <type 'bool'>), u'update': (False, u'Update the source model with the best-fit position.', <type 'bool'>)}, u'binning': {u'projtype': (u'WCS', u'Projection mode (WCS or HPX).', <type 'str'>), u'binsperdec': (8, u'Number of energy bins per decade.', <type 'float'>), u'enumbins': (None, u'Number of energy bins. If none this will be inferred from energy range and ``binsperdec`` parameter.', <type 'int'>), u'roiwidth': (10.0, u'Width of the ROI in degrees. The number of pixels in each spatial dimension will be set from ``roiwidth`` / ``binsz`` (rounded up).', <type 'float'>), u'hpx_ebin': (True, u'Include energy binning', <type 'bool'>), u'binsz': (0.1, u'Spatial bin size in degrees.', <type 'float'>), u'npix': (None, u'Number of pixels. If none then this will be set from ``roiwidth`` and ``binsz``.', <type 'int'>), u'hpx_order': (10, u'Order of the map (int between 0 and 12, included)', <type 'int'>), u'proj': (u'AIT', u'Spatial projection for WCS mode.', <type 'str'>), u'coordsys': (u'CEL', u'Coordinate system of the spatial projection (CEL or GAL).', <type 'str'>), u'hpx_ordering_scheme': (u'RING', u'HEALPix Ordering Scheme', <type 'str'>)}, u'extension': {u'width': (None, u'Parameter vector for scan over spatial extent. If none then the parameter vector will be set from ``width_min``, ``width_max``, and ``width_nstep``.', <type 'str'>), u'fix_background': (False, u'Fix any background parameters that are currently free in the model when performing the likelihood scan over extension.', <type 'bool'>), u'width_max': (1.0, u'Maximum value in degrees for the likelihood scan over spatial extent.', <type 'float'>), u'sqrt_ts_threshold': (None, u'Threshold on sqrt(TS_ext) that will be applied when ``update`` is True. If None then nothreshold is applied.', <type 'float'>), u'width_min': (0.01, u'Minimum value in degrees for the likelihood scan over spatial extent.', <type 'float'>), u'spatial_model': (u'RadialGaussian', u'Spatial model use for extension test.', <type 'str'>), u'update': (False, u'Update the source model with the best-fit spatial extension.', <type 'bool'>), u'width_nstep': (21, u'Number of steps for the spatial likelihood scan.', <type 'int'>)}, u'sed': {u'use_local_index': (False, u'Use a power-law approximation to the shape of the global spectrum in each bin. If this is false then a constant index set to `bin_index` will be used.', <type 'bool'>), u'bin_index': (2.0, u'Spectral index that will be use when fitting the energy distribution within an energy bin.', <type 'float'>), u'cov_scale': (3.0, u'Scale factor that sets the strength of the prior on nuisance parameters when ``fix_background``=True. Setting this to None disables the prior.', <type 'float'>), u'fix_background': (True, u'Fix background normalization parameters when fitting the source flux in each energy bin. If True background normalizations will be profiled with a prior on their value with strength set by ``cov_scale``.', <type 'bool'>), u'ul_confidence': (0.95, u'Confidence level for upper limit calculation.', <type 'float'>)}, u'fileio': {u'usescratch': (False, u'Run analysis in a temporary working directory under ``scratchdir``.', <type 'bool'>), u'scratchdir': (u'/scratch', u'Path to the scratch directory. If ``usescratch`` is True then a temporary working directory will be created under this directory.', <type 'str'>), u'savefits': (True, u'Save intermediate FITS files.', <type 'bool'>), u'workdir': (None, u'Path to the working directory.', <type 'str'>), u'outdir_regex': ([u'\\.fits$|\\.fit$|\\.xml$|\\.npy$|\\.png$|\\.pdf$|\\.yaml$'], u'Stage files to the output directory that match at least one of the regular expressions in this list. This option only takes effect when ``usescratch`` is True.', <type 'list'>), u'workdir_regex': ([u'\\.fits$|\\.fit$|\\.xml$|\\.npy$'], u'Stage files to the working directory that match at least one of the regular expressions in this list. This option only takes effect when ``usescratch`` is True.', <type 'list'>), u'logfile': (None, u'Path to log file. If None then log will be written to fermipy.log.', <type 'str'>), u'outdir': (None, u'Path of the output directory. If none this will default to the directory containing the configuration file.', <type 'str'>)}, u'gtlike': {u'irfs': (None, u'Set the IRF string.', <type 'str'>), u'minbinsz': (0.05, u'Set the minimum bin size used for resampling diffuse maps.', <type 'float'>), u'bexpmap': (None, u'', <type 'str'>), u'edisp': (True, u'Enable the correction for energy dispersion.', <type 'bool'>), u'srcmap': (None, u'', <type 'str'>), u'resample': (True, u'', <type 'bool'>), u'llscan_npts': (20, u'Number of evaluation points to use when performing a likelihood scan.', <type 'int'>), u'convolve': (True, u'', <type 'bool'>), u'rfactor': (2, u'', <type 'int'>), u'edisp_disable': (None, u'Provide a list of sources for which the edisp correction should be disabled.', <type 'list'>)}, u'residmap': {u'model': (None, u'Dictionary defining the properties of the test source. By default the test source will be a PointSource with an Index 2 power-law specturm.', <type 'dict'>), u'loge_bounds': (None, u'Lower and upper energy bounds in log10(E/MeV). By default the calculation will be performed over the full analysis energy range.', <type 'list'>)}, u'optimizer': {u'retries': (3, u'Set the number of times to retry the fit when the fit quality is less than ``min_fit_quality``.', <type 'int'>), u'optimizer': (u'MINUIT', u'Set the optimization algorithm to use when maximizing the likelihood function.', <type 'str'>), u'verbosity': (0, u'', <type 'int'>), u'max_iter': (100, u'Maximum number of iterations for the Newtons method fitter.', <type 'int'>), u'min_fit_quality': (2, u'Set the minimum fit quality.', <type 'int'>), u'tol': (0.001, u'Set the optimizer tolerance.', <type 'float'>), u'init_lambda': (0.0001, u'Initial value of damping parameter for step size calculation when using the NEWTON fitter. A value of zero disables damping.', <type 'float'>)}, u'model': {u'catalogs': (None, u'', <type 'list'>), u'limbdiff': (None, u'', <type 'list'>), u'src_radius_roi': (None, u'Half-width of ``src_roiwidth`` selection. This parameter can be used in lieu of ``src_roiwidth``.', <type 'float'>), u'extdir': (None, u'Set a directory that will be searched for extended source FITS templates. Template files in this directory will take precendence over catalog source templates with the same name.', <type 'str'>), u'sources': (None, u'', <type 'list'>), u'assoc_xmatch_columns': ([u'3FGL_Name'], u'Choose a set of association columns on which to cross-match catalogs.', <type 'list'>), u'diffuse': (None, u'', <type 'list'>), u'src_roiwidth': (None, u'Width of square selection cut for inclusion of catalog sources in the model. Includes sources within a square region with side ``src_roiwidth`` centered on the ROI. If this parameter is none then no selection is applied. This selection will be ORed with the ``src_radius`` selection.', <type 'float'>), u'isodiff': (None, u'Set the isotropic template.', <type 'list'>), u'merge_sources': (True, u'Merge properties of sources that appear in multiple source catalogs. If merge_sources=false then subsequent sources with the same name will be ignored.', <type 'bool'>), u'extract_diffuse': (False, u'Extract a copy of all mapcube components centered on the ROI.', <type 'bool'>), u'src_radius': (None, u'Radius of circular selection cut for inclusion of catalog sources in the model. Includes sources within a circle of this radius centered on the ROI. If this parameter is none then no selection is applied. This selection will be ORed with the ``src_roiwidth`` selection.', <type 'float'>), u'galdiff': (None, u'Set the galactic IEM mapcube.', <type 'list'>)}, u'data': {u'evfile': (None, u'Path to FT1 file or list of FT1 files.', <type 'str'>), u'cacheft1': (True, u'Cache FT1 files when performing binned analysis. If false then only the counts cube is retained.', <type 'bool'>), u'scfile': (None, u'Path to FT2 (spacecraft) file.', <type 'str'>), u'ltcube': (None, u'Path to livetime cube. If none a livetime cube will be generated with ``gtmktime``.', <type 'str'>)}, u'plotting': {u'catalogs': (None, u'', <type 'list'>), u'format': (u'png', u'', <type 'str'>), u'loge_bounds': (None, u'', <type 'list'>), u'graticule_radii': (None, u'Define a list of radii at which circular graticules will be drawn.', <type 'list'>), u'cmap': (u'ds9_b', u'Set the colormap for 2D plots.', <type 'str'>), u'label_ts_threshold': (0.0, u'TS threshold for labeling sources in sky maps. If None then no sources will be labeled.', <type 'float'>)}, u'tscube': {u'do_sed': (True, u'Compute the energy bin-by-bin fits', <type 'bool'>), u'remake_test_source': (False, u'If true, recomputes the test source image (otherwise just shifts it)', <type 'bool'>), u'st_scan_level': (0, u'Level to which to do ST-based fitting (for testing)', <type 'int'>), u'cov_scale': (-1.0, u'Scale factor to apply to broadband fitting cov. matrix in bin-by-bin fits ( < 0 -> fixed ) ', <type 'float'>), u'max_iter': (30, u'Maximum number of iterations for the Newtons method fitter.', <type 'int'>), u'nnorm': (10, u'Number of points in the likelihood v. normalization scan', <type 'int'>), u'norm_sigma': (5.0, u'Number of sigma to use for the scan range ', <type 'float'>), u'tol_type': (0, u'Absoulte (0) or relative (1) criteria for convergence.', <type 'int'>), u'cov_scale_bb': (-1.0, u'Scale factor to apply to global fitting cov. matrix in broadband fits. ( < 0 -> no prior ) ', <type 'float'>), u'tol': (0.001, u'Critetia for fit convergence (estimated vertical distance to min < tol )', <type 'float'>), u'model': (None, u'Dictionary defining the properties of the test source. By default the test source will be a PointSource with an Index 2 power-law specturm.', <type 'dict'>), u'init_lambda': (0, u'Initial value of damping parameter for newton step size calculation.', <type 'float'>)}}¶

delete_source(name, save_template=True, delete_source_map=False, build_fixed_wts=True, **kwargs)[source]¶

Delete a source from the ROI model.

Parameters:	name (str) – Source name. save_template (bool) – Delete the SpatialMap FITS template associated with this source. delete_source_map (bool) – Delete the source map associated with this source from the source maps file.
Returns:	src – The deleted source object.
Return type:	Model

delete_sources(cuts=None, distance=None, skydir=None, minmax_ts=None, minmax_npred=None, square=False, exclude_diffuse=True)[source]¶

Delete sources in the ROI model satisfying the given selection criteria.

cuts : dict

Dictionary of [min,max] selections on source properties.

distance : float

Cut on angular distance from skydir. If None then no selection will be applied.

skydir : SkyCoord

Reference sky coordinate for distance selection. If None then the distance selection will be applied with respect to the ROI center.

minmax_ts : list

Free sources that have TS in the range [min,max]. If either min or max are None then only a lower (upper) bound will be applied. If this parameter is none no selection will be applied.

minmax_npred : list

Free sources that have npred in the range [min,max]. If either min or max are None then only a lower (upper) bound will be applied. If this parameter is none no selection will be applied.

square : bool

Switch between applying a circular or square (ROI-like) selection on the maximum projected distance from the ROI center.

Returns:	srcs – A list of Model objects.
Return type:	list

energies¶

Return the energy bin edges in MeV.

enumbins¶

Return the number of energy bins.

extension(name, **kwargs)[source]¶

Test this source for spatial extension with the likelihood ratio method (TS_ext). This method will substitute an extended spatial model for the given source and perform a one-dimensional scan of the spatial extension parameter over the range specified with the width parameters. The 1-D profile likelihood is then used to compute the best-fit value, upper limit, and TS for extension. Any background parameters that are free will also be simultaneously profiled in the likelihood scan.

Parameters:	name (str) – Source name. spatial_model (str) – Spatial model that will be used to test the source extension. The spatial scale parameter of the respective model will be set such that the 68% containment radius of the model is equal to the width parameter. The following spatial models are supported: RadialDisk : Azimuthally symmetric 2D disk. RadialGaussian : Azimuthally symmetric 2D gaussian. width_min (float) – Minimum value in degrees for the spatial extension scan. width_max (float) – Maximum value in degrees for the spatial extension scan. width_nstep (int) – Number of scan points between width_min and width_max. Scan points will be spaced evenly on a logarithmic scale between log(width_min) and log(width_max). width (array-like) – Sequence of values in degrees for the spatial extension scan. If this argument is None then the scan points will be determined from width_min/width_max/width_nstep. fix_background (bool) – Fix all background sources when performing the extension fit. update (bool) – Update this source with the best-fit model for spatial extension if TS_ext > tsext_threshold. sqrt_ts_threshold (float) – Threshold on sqrt(TS_ext) that will be applied when update is true. If None then no threshold will be applied. optimizer (dict) – Dictionary that overrides the default optimizer settings.
Returns:	extension – Dictionary containing results of the extension analysis. The same dictionary is also saved to the dictionary of this source under ‘extension’.
Return type:	dict

find_sources(prefix=u'', **kwargs)¶

An iterative source-finding algorithm.

Parameters:

model (dict) – Dictionary defining the properties of the test source. This is the model that will be used for generating TS maps. sqrt_ts_threshold (float) – Source threshold in sqrt(TS). Only peaks with sqrt(TS) exceeding this threshold will be used as seeds for new sources. min_separation (float) – Minimum separation in degrees of sources detected in each iteration. The source finder will look for the maximum peak in the TS map within a circular region of this radius. max_iter (int) – Maximum number of source finding iterations. The source finder will continue adding sources until no additional peaks are found or the number of iterations exceeds this number. sources_per_iter (int) – Maximum number of sources that will be added in each iteration. If the number of detected peaks in a given iteration is larger than this number, only the N peaks with the largest TS will be used as seeds for the current iteration. tsmap_fitter (str) – Set the method used internally for generating TS maps. Valid options: tsmap tscube tsmap (dict) – Keyword arguments dictionary for tsmap method. tscube (dict) – Keyword arguments dictionary for tscube method.

Returns:

peaks (list) – List of peak objects. sources (list) – List of source objects.

fit(update=True, **kwargs)[source]¶

Run the likelihood optimization. This will execute a fit of all parameters that are currently free in the model and update the charateristics of the corresponding model components (TS, npred, etc.). The fit will be repeated N times (set with the retries parameter) until a fit quality greater than or equal to min_fit_quality and a fit status code of 0 is obtained. If the fit does not succeed after N retries then all parameter values will be reverted to their state prior to the execution of the fit.

Parameters:	update (bool) – Update the model dictionary for all sources with free parameters. tol (float) – Set the optimizer tolerance. verbosity (int) – Set the optimizer output level. optimizer (str) – Set the likelihood optimizer (e.g. MINUIT or NEWMINUIT). retries (int) – Set the number of times to rerun the fit when the fit quality is < 3. min_fit_quality (int) – Set the minimum fit quality. If the fit quality is smaller than this value then all model parameters will be restored to their values prior to the fit. reoptimize (bool) – Refit background sources when updating source properties (TS and likelihood profiles).
Returns:	fit – Dictionary containing diagnostic information from the fit (fit quality, parameter covariances, etc.).
Return type:	dict

fit_correlation()[source]¶

free_index(name, free=True, **kwargs)[source]¶

Free/Fix index of a source.

Parameters:	name (str) – Source name. free (bool) – Choose whether to free (free=True) or fix (free=False).

free_norm(name, free=True, **kwargs)[source]¶

Free/Fix normalization of a source.

Parameters:	name (str) – Source name. free (bool) – Choose whether to free (free=True) or fix (free=False).

free_parameter(name, par, free=True)[source]¶

free_shape(name, free=True, **kwargs)[source]¶

Free/Fix shape parameters of a source.

Parameters:	name (str) – Source name. free (bool) – Choose whether to free (free=True) or fix (free=False).

free_source(name, free=True, pars=None, **kwargs)[source]¶

Free/Fix parameters of a source.

Parameters:	name (str) – Source name. free (bool) – Choose whether to free (free=True) or fix (free=False) source parameters. pars (list) – Set a list of parameters to be freed/fixed for this source. If none then all source parameters will be freed/fixed with the exception of those defined in the skip_pars list.

free_sources(free=True, pars=None, cuts=None, distance=None, skydir=None, minmax_ts=None, minmax_npred=None, square=False, exclude_diffuse=False, **kwargs)[source]¶

Free or fix sources in the ROI model satisfying the given selection. When multiple selections are defined, the selected sources will be those satisfying the logical AND of all selections (e.g. distance < X && minmax_ts[0] < ts < minmax_ts[1] && ...).

Parameters:	free (bool) – Choose whether to free (free=True) or fix (free=False) source parameters. pars (list) – Set a list of parameters to be freed/fixed for each source. If none then all source parameters will be freed/fixed. If pars=’norm’ then only normalization parameters will be freed. cuts (dict) – Dictionary of [min,max] selections on source properties. distance (float) – Cut on angular distance from skydir. If None then no selection will be applied. skydir (SkyCoord) – Reference sky coordinate for distance selection. If None then the distance selection will be applied with respect to the ROI center. minmax_ts (list) – Free sources that have TS in the range [min,max]. If either min or max are None then only a lower (upper) bound will be applied. If this parameter is none no selection will be applied. minmax_npred (list) – Free sources that have npred in the range [min,max]. If either min or max are None then only a lower (upper) bound will be applied. If this parameter is none no selection will be applied. square (bool) – Switch between applying a circular or square (ROI-like) selection on the maximum projected distance from the ROI center. exclude_diffuse (bool) – Exclude diffuse sources.
Returns:	srcs – A list of Model objects.
Return type:	list

generate_model(model_name=None)[source]¶

Generate model maps for all components. model_name should be a unique identifier for the model. If model_name is None then the model maps will be generated using the current parameters of the ROI.

get_config()¶

Return a default configuration dictionary for this class.

get_free_param_vector()[source]¶

get_free_source_params(name)[source]¶

get_norm(name)[source]¶

get_params(freeonly=False)[source]¶

get_source_dfde(name)[source]¶

Return differential flux distribution of a source. For sources with FileFunction spectral type this returns the internal differential flux array.

Returns:	loge (ndarray) – Array of energies at which the differential flux is evaluated (log10(E/MeV)). dfde (ndarray) – Array of differential flux values (cm^{-2} s^{-1} MeV^{-1}) evaluated at energies in loge.

get_source_name(name)[source]¶

Return the name of a source as it is defined in the pyLikelihood model object.

get_sources(cuts=None, distance=None, skydir=None, minmax_ts=None, minmax_npred=None, square=False)[source]¶

Retrieve list of sources in the ROI satisfying the given selections.

Returns:	srcs – A list of Model objects.
Return type:	list

get_src_model(name, paramsonly=False, reoptimize=False, npts=None, **kwargs)[source]¶

Compose a dictionary for a source with the current best-fit parameters.

Parameters:	name (str) – paramsonly (bool) – Skip computing TS and likelihood profile. reoptimize (bool) – Re-fit background parameters in likelihood scan. npts (int) – Number of points for likelihood scan.
Returns:	src_dict
Return type:	dict

like¶

Return the global likelihood object.

load_roi(infile, reload_sources=False)[source]¶

This function reloads the analysis state from a previously saved instance generated with write_roi.

Parameters:	infile (str) – reload_sources (bool) – Regenerate source maps for non-diffuse sources.

load_xml(xmlfile)[source]¶

Load model definition from XML.

Parameters:	xmlfile (str) – Name of the input XML file.

localize(name, **kwargs)¶

Find the best-fit position of a source. Localization is performed in two steps. First a TS map is computed centered on the source with half-width set by dtheta_max. A fit is then performed to the maximum TS peak in this map. The source position is then further refined by scanning the likelihood in the vicinity of the peak found in the first step. The size of the scan region is set to encompass the 99% positional uncertainty contour as determined from the peak fit.

Parameters:	name (str) – Source name. dtheta_max (float) – Maximum offset in RA/DEC in deg from the nominal source position that will be used to define the boundaries of the TS map search region. nstep (int) – Number of steps in longitude/latitude that will be taken when refining the source position. The bounds of the scan range are set to the 99% positional uncertainty as determined from the TS map peak fit. The total number of sampling points will be nstep**2. fix_background (bool) – Fix background parameters when fitting the source position. update (bool) – Update the model for this source with the best-fit position. If newname=None this will overwrite the existing source map of this source with one corresponding to its new location. newname (str) – Name that will be assigned to the relocalized source when update=True. If newname is None then the existing source name will be used. optimizer (dict) – Dictionary that overrides the default optimizer settings.
Returns:	localize – Dictionary containing results of the localization analysis. This dictionary is also saved to the dictionary of this source in ‘localize’.
Return type:	dict

log_energies¶

Return the energy bin edges in log10(E/MeV).

loge_bounds¶

Current analysis energy bounds in log10(E/MeV).

make_plots(prefix, mcube_map=None, **kwargs)[source]¶

Make diagnostic plots using the current ROI model.

model_counts_map(name=None, exclude=None)[source]¶

Return the model counts map for a single source, a list of sources, or for the sum of all sources in the ROI. The exclude parameter can be used to exclude one or more components when generating the model map.

Parameters:	name (str or list of str) – Parameter controlling the set of sources for which the model counts map will be calculated. If name=None the model map will be generated for all sources in the ROI. exclude (str or list of str) – List of sources that will be excluded when calculating the model map.
Returns:	map
Return type:	Map

model_counts_spectrum(name, logemin=None, logemax=None, summed=False)[source]¶

Return the predicted number of model counts versus energy for a given source and energy range. If summed=True return the counts spectrum summed over all components otherwise return a list of model spectra.

npix¶

Return the number of energy bins.

optimize(**kwargs)[source]¶

Iteratively optimize the ROI model. The optimization is performed in three sequential steps:

• Free the normalization of the N largest components (as determined from NPred) that contain a fraction npred_frac of the total predicted counts in the model and perform a simultaneous fit of the normalization parameters of these components.
• Individually fit the normalizations of all sources that were not included in the first step in order of their npred values. Skip any sources that have NPred < npred_threshold.
• Individually fit the shape and normalization parameters of all sources with TS > shape_ts_threshold where TS is determined from the first two steps of the ROI optimization.

To ensure that the model is fully optimized this method can be run multiple times.

Parameters:

npred_frac (float) – Threshold on the fractional number of counts in the N largest components in the ROI. This parameter determines the set of sources that are fit in the first optimization step. npred_threshold (float) – Threshold on the minimum number of counts of individual sources. This parameter determines the sources that are fit in the second optimization step. shape_ts_threshold (float) – Threshold on source TS used for determining the sources that will be fit in the third optimization step. max_free_sources (int) – Maximum number of sources that will be fit simultaneously in the first optimization step. skip (list) – List of str source names to skip while optimizing. optimizer (dict) – Dictionary that overrides the default optimizer settings.

outdir¶

Return the analysis output directory.

print_config(logger, loglevel=None)¶

print_model(loglevel=20)[source]¶

print_params(allpars=False, loglevel=20)[source]¶

Print information about the model parameters (values, errors, bounds, scale).

print_roi(loglevel=20)[source]¶

Print information about the spectral and spatial properties of the ROI (sources, diffuse components).

profile(name, parName, logemin=None, logemax=None, reoptimize=False, xvals=None, npts=None, savestate=True, **kwargs)[source]¶

Profile the likelihood for the given source and parameter.

Parameters:	name (str) – Source name. parName (str) – Parameter name. reoptimize (bool) – Re-fit nuisance parameters at each step in the scan. Note that enabling this option will only re-fit parameters that were free when the method was executed.
Returns:	lnlprofile – Dictionary containing results of likelihood scan.
Return type:	dict

profile_norm(name, logemin=None, logemax=None, reoptimize=False, xvals=None, npts=None, fix_shape=True, savestate=True, **kwargs)[source]¶

Profile the normalization of a source.

Parameters:	name (str) – Source name. reoptimize (bool) – Re-optimize free parameters in the model at each point in the profile likelihood scan.

projtype¶

Return the type of projection to use

reload_source(name, init_source=True)[source]¶

Delete and reload a source in the model. This will refresh the spatial model of this source to the one defined in the XML model.

reload_sources(names, init_source=True)[source]¶

remove_prior(name, parName)[source]¶

remove_priors()[source]¶

Clear all priors.

residmap(prefix=u'', **kwargs)¶

Generate 2-D spatial residual maps using the current ROI model and the convolution kernel defined with the model argument.

Parameters:	prefix (str) – String that will be prefixed to the output residual map files. model (dict) – Dictionary defining the properties of the convolution kernel. exclude (str or list of str) – Source or sources that will be removed from the model when computing the residual map. loge_bounds (list) – Restrict the analysis to an energy range (emin,emax) in log10(E/MeV) that is a subset of the analysis energy range. By default the full analysis energy range will be used. If either emin/emax are None then only an upper/lower bound on the energy range wil be applied. make_plots (bool) – Write image files. write_fits (bool) – Write FITS files.
Returns:	maps – A dictionary containing the Map objects for the residual significance and amplitude.
Return type:	dict

roi¶

Return the ROI object.

scale_parameter(name, par, scale)[source]¶

sed(name, **kwargs)¶

Generate a spectral energy distribution (SED) for a source. This function will fit the normalization of the source in each energy bin. By default the SED will be generated with the analysis energy bins but a custom binning can be defined with the loge_bins parameter.

Parameters:	name (str) – Source name. prefix (str) – Optional string that will be prepended to all output files (FITS and rendered images). loge_bins (ndarray) – Sequence of energies in log10(E/MeV) defining the edges of the energy bins. If this argument is None then the analysis energy bins will be used. The energies in this sequence must align with the bin edges of the underyling analysis instance. bin_index (float) – Spectral index that will be use when fitting the energy distribution within an energy bin. use_local_index (bool) – Use a power-law approximation to the shape of the global spectrum in each bin. If this is false then a constant index set to bin_index will be used. fix_background (bool) – Fix background components when fitting the flux normalization in each energy bin. If fix_background=False then all background parameters that are currently free in the fit will be profiled. By default fix_background=True. ul_confidence (float) – Set the confidence level that will be used for the calculation of flux upper limits in each energy bin. cov_scale (float) – Scaling factor that will be applied when setting the gaussian prior on the normalization of free background sources. If this parameter is None then no gaussian prior will be applied. write_fits (bool) – Write a FITS file containing the SED analysis results. write_npy (bool) – Write a numpy file with the contents of the output dictionary. optimizer (dict) – Dictionary that overrides the default optimizer settings.
Returns:	sed – Dictionary containing output of the SED analysis. This dictionary is also saved to the ‘sed’ dictionary of the Source instance.
Return type:	dict

set_edisp_flag(name, flag=True)[source]¶

Enable or disable the energy dispersion correction for the given source.

set_energy_range(logemin, logemax)[source]¶

Set the energy bounds of the analysis. This restricts the evaluation of the likelihood to the data that falls in this range. Input values will be rounded to the closest bin edge value. If either argument is None then the lower or upper bound of the analysis instance will be used.

Parameters:	logemin (float) – Lower energy bound in log10(E/MeV). logemax (float) – Upper energy bound in log10(E/MeV).
Returns:	eminmax – Minimum and maximum energy in log10(E/MeV).
Return type:	array

set_free_param_vector(free)[source]¶

set_log_level(level)[source]¶

set_norm(name, value, update_source=True)[source]¶

set_norm_bounds(name, bounds)[source]¶

set_norm_scale(name, value)[source]¶

set_parameter(name, par, value, true_value=True, scale=None, bounds=None, update_source=True)[source]¶

Update the value of a parameter. Parameter bounds will automatically be adjusted to encompass the new parameter value.

Parameters:	name (str) – Source name. par (str) – Parameter name. value (float) – Parameter value. By default this argument should be the unscaled (True) parameter value. scale (float) – Parameter scale (optional). Value argument is interpreted with respect to the scale parameter if it is provided. update_source (bool) – Update the source dictionary for the object.

set_parameter_bounds(name, par, bounds)[source]¶

Set the bounds of a parameter.

Parameters:	name (str) – Source name. par (str) – Parameter name. bounds (list) – Upper and lower bound.

set_parameter_scale(name, par, scale)[source]¶

Update the scale of a parameter while keeping its value constant.

set_source_dfde(name, dfde, update_source=True)[source]¶

Set the differential flux distribution of a source with the FileFunction spectral type.

Parameters:	name (str) – Source name. dfde (ndarray) – Array of differential flux values (cm^{-2} s^{-1} MeV^{-1}).

set_source_spectrum(name, spectrum_type=u'PowerLaw', spectrum_pars=None, update_source=True)[source]¶

Set the spectral model of a source. This function can be used to change the spectral type of a source or modify its spectral parameters. If called with spectrum_type=’FileFunction’ and spectrum_pars=None, the source spectrum will be replaced with a FileFunction with the same differential flux distribution as the original spectrum.

Parameters:	name (str) – Source name. spectrum_type (str) – Spectrum type (PowerLaw, etc.). spectrum_pars (dict) – Dictionary of spectral parameters (optional). update_source (bool) – Recompute all source characteristics (flux, TS, NPred) using the new spectral model of the source.

setup(init_sources=True, overwrite=False)[source]¶

Run pre-processing for each analysis component and construct a joint likelihood object. This function performs the following tasks: data selection (gtselect, gtmktime), data binning (gtbin), and model generation (gtexpcube2,gtsrcmaps).

Parameters:	init_sources (bool) – Choose whether to compute properties (flux, TS, etc.) for individual sources. overwrite (bool) – Run all pre-processing steps even if the output file of that step is present in the working directory. By default this function will skip any steps for which the output file already exists.

simulate_roi(name=None, randomize=True, restore=False)[source]¶

Generate a simulation of the ROI using the current best-fit model and replace the data counts cube with this simulation. The simulation is created by generating an array of Poisson random numbers with expectation values drawn from the model cube of the binned analysis instance. This function will update the counts cube both in memory and in the source map file. The counts cube can be restored to its original state by calling this method with restore = True.

Parameters:	name (str) – Name of the model component to be simulated. If None then the whole ROI will be simulated. restore (bool) – Restore the data counts cube to its original state.

simulate_source(src_dict=None)[source]¶

Inject simulated source counts into the data.

Parameters:	src_dict (dict) – Dictionary defining the spatial and spectral properties of the source that will be injected.

stage_input()[source]¶

Copy input files to working directory.

stage_output()[source]¶

Copy data products to final output directory.

tscube(prefix=u'', **kwargs)¶

Generate a spatial TS map for a source component with properties defined by the model argument. This method uses the gttscube ST application for source fitting and will simultaneously fit the test source normalization as well as the normalizations of any background components that are currently free. The output of this method is a dictionary containing Map objects with the TS and amplitude of the best-fit test source. By default this method will also save maps to FITS files and render them as image files.

Parameters:	prefix (str) – Optional string that will be prepended to all output files (FITS and rendered images). model (dict) – Dictionary defining the properties of the test source. do_sed (bool) – Compute the energy bin-by-bin fits. nnorm (int) – Number of points in the likelihood v. normalization scan. norm_sigma (float) – Number of sigma to use for the scan range. tol (float) – Critetia for fit convergence (estimated vertical distance to min < tol ). tol_type (int) – Absoulte (0) or relative (1) criteria for convergence. max_iter (int) – Maximum number of iterations for the Newton’s method fitter remake_test_source (bool) – If true, recomputes the test source image (otherwise just shifts it) st_scan_level (int) – make_plots (bool) – Write image files. write_fits (bool) – Write a FITS file with the results of the analysis.
Returns:	maps – A dictionary containing the Map objects for TS and source amplitude.
Return type:	dict

tsmap(prefix=u'', **kwargs)¶

Generate a spatial TS map for a source component with properties defined by the model argument. The TS map will have the same geometry as the ROI. The output of this method is a dictionary containing Map objects with the TS and amplitude of the best-fit test source. By default this method will also save maps to FITS files and render them as image files.

This method uses a simplified likelihood fitting implementation that only fits for the normalization of the test source. Before running this method it is recommended to first optimize the ROI model (e.g. by running optimize()).

Parameters:	prefix (str) – Optional string that will be prepended to all output files (FITS and rendered images). model (dict) – Dictionary defining the properties of the test source. exclude (str or list of str) – Source or sources that will be removed from the model when computing the TS map. loge_bounds (list) – Restrict the analysis to an energy range (emin,emax) in log10(E/MeV) that is a subset of the analysis energy range. By default the full analysis energy range will be used. If either emin/emax are None then only an upper/lower bound on the energy range wil be applied. max_kernel_radius (float) – Set the maximum radius of the test source kernel. Using a smaller value will speed up the TS calculation at the loss of accuracy. The default value is 3 degrees. make_plots (bool) – Write image files. write_fits (bool) – Write a FITS file. write_npy (bool) – Write a numpy file.
Returns:	maps – A dictionary containing the Map objects for TS and source amplitude.
Return type:	dict

unzero_source(name)[source]¶

update_source(name, paramsonly=False, reoptimize=False, **kwargs)[source]¶

Update the dictionary for this source.

Parameters:	name (str) – paramsonly (bool) – reoptimize (bool) – Re-fit background parameters in likelihood scan.

workdir¶

Return the analysis working directory.

write_config(outfile)¶

Write the configuration dictionary to an output file.

write_model_map(model_name, name=None)[source]¶

Save the counts model map to a FITS file.

Parameters:	model_name (str) – String that will be append to the name of the output file. name (str) – Name of the component.

write_roi(outfile=None, save_model_map=False, fmt=u'npy', **kwargs)[source]¶

Write current state of the analysis to a file. This method writes an XML model definition, a ROI dictionary, and a FITS source catalog file. A previously saved analysis state can be reloaded from the ROI dictionary file with the load_roi method.

Parameters:	outfile (str) – String prefix of the output files. The extension of this string will be stripped when generating the XML, YAML and npy filenames. make_plots (bool) – Generate diagnostic plots. save_model_map (bool) – Save the current counts model to a FITS file. fmt (str) – Set the output file format (yaml or npy).

write_xml(xmlfile)[source]¶

Save current model definition as XML file.

Parameters:	xmlfile (str) – Name of the output XML file.

zero_source(name)[source]¶