Agilepy’s documentation

Agilepy is an open-source Python package developed at INAF/OAS Bologna to analyse AGILE/GRID data built on top of the command-line version of the AGILE/GRID Science Tools.

The main purpose of the package is to provide an easy to use high-level Python interface to analyse AGILE/GRID data by simplifying the configuration of the tasks and ensuring straightforward access to the data. The current features are the generation and display of sky maps and light curves, the access to gamma-ray sources catalogues, the analysis to perform spectral model and position fitting including the background evaluation, the aperture photometry analysis, and the wavelet analysis. In addition, Agilepy provides an engineering interface to analyse the time evolution of the AGILE off-axis viewing angle for a chosen sky region, comparing them with Fermi/LAT off-axis evolution.

Agilepy

Agilepy

Agilepy is an open-source Python package developed at INAF/OAS Bologna to analyse AGILE/GRID data built on top of the command-line version of the AGILE/GRID Science Tools.

The main purpose of the package is to provide an easy to use high-level Python interface to analyse AGILE/GRID data by simplifying the configuration of the tasks and ensuring straightforward access to the data. The current features are the generation and display of sky maps and light curves, the access to gamma-ray sources catalogues, the analysis to perform spectral model and position fitting including the background evaluation, the aperture photometry analysis, and the wavelet analysis. In addition, Agilepy provides an engineering interface to analyse the time evolution of the AGILE off-axis viewing angle for a chosen sky region, comparing them with Fermi/LAT off-axis evolution.

Agilepy is similar to Fermipy (https://fermipy.readthedocs.io/) and gammapy (https://docs.gammapy.org/) tools, providing a common way to analyse gamma-ray data.

Agilepy provides the last version of the available Science Tools (BUILD25), the H0025 instrument response functions (IRFs), and the latest version of the diffuse Galactic emission model.

Agilepy (and its dependencies) can be easily installed using Anaconda (https://www.anaconda.com/).

AGILE

AGILE (Astrorivelatore Gamma ad Immagini LEggero) is an astrophysics mission of the Italian Space Agency (ASI) operating since 2007 April devoted to gamma-ray and X-ray astrophysics. It carries two instruments observing at hard X-rays between 18 and 60 keV (Super-AGILE) and in the gamma-ray band between 30 MeV and 50 GeV (Silicon Tracker). The payload is completed by a calorimeter (MCAL) sensitive in the 0.4–100 MeV range and an anticoincidence (AC) system. The combination of the Silicon Tracker, MCAL, and AC forms the Gamma-Ray Imaging Detector (GRID).

A set of different on-board triggers enables the discrimination of background events (mainly cosmic rays in the AGILE Low Earth Orbit) from gamma-ray events. AGILE raw data are down-linked every about 100 min to the ASI Malindi ground station in Kenya, and transmitted first to the Telespazio Mission Control Center at Fucino, and then to the AGILE Data Center (ADC), which is part of the ASI Space Science Data Center (SSDC, previously known as ASDC) and then to the INAF/OAS Bologna for the real-time analysis of data.

Main AGILE websites:

AGILE Apps:

AGILE Science Tools

The AGILE/GRID Science Tools developed by the AGILE Team are used to analyse gamma-ray data starting from spacecraft files (called LOG), and the acquired events (EVT files or event list). They provide a way to generate gamma-ray counts, exposure and diffuse emission maps that are used as input for the binned maximum likelihood estimator (MLE). The analysis depends on the isotropic and Galactic diffuse emission, the gamma-ray photon statistics, and on the instrument response functions (IRFs). IRFs are matrices that characterise the effective area (Aeff), the point spread function (PSF), and the energy dispersion probability (EDP), that depend on the direction of the incoming gamma-ray in instrument coordinates, its energy and on the on-ground event filter.

The result of the MLE is an evaluation of the presence of one or more point-like or extended sources in the sky maps: this is the essential step for the scientific results of AGILE.

A full description and characterisation of the last release of the Science Tools is available in https://arxiv.org/abs/1903.06957. Science Tools, IRFs and Galactic emission model are publicly available from the AGILE website at SSDC: https://agile.ssdc.asi.it.

Agilepy analysis

The AGILE-GRID data analysis can be performed with Agilepy with different techniques:

  • using the maximum likelihood estimator analysis

  • wavelet techniques

  • Lomb-Scargle periodogram analysis (coming soon)

The likelihood analysis reach better sensitivity, more accurate flux measurement, better evaluation of the backgrounds and can work with a detailed source models where more sources can be considered at the same time.

AGILE-GRID light curves can be created in two different ways:

  • using the maximum likelihood estimator analysis

  • using aperture photometry.

Aperture photometry provides a raw measure of the flux of a sigle source and is less computing demanding.

Maximium Likelihood Estimator caveats

During the fitting process some values are fixed and others are variable, depending on the values of the flags. The execution time strongly depends on the number of the variable parameters. It is not possible to predict how long the fitting process will last or how it depends on the number of parameters, but the dependence is not linear. If all the diffuse coefficients are variable and all spectral parameters are free, for M maps and S sources the number of variable parameters will be 2M+4S. In the case of many maps and many sources, this may lead to a very long execution time.

The fitting process takes place in two steps, according to the method of Maximum Likelihood. During each step all the sources are considered one by one, and several fitting attempts are performed by invoking the function TH1D::Fit() provided by the ROOT library, developed by CERN and will find the related documentation on the CERN web site.

Installation

Agilepy is available as Anaconda package or into a ready-to-use Docker container (from 1.4.0)

Note

AGILE DATASET DOWNLOAD Now it possible to download all the public AGILE dataset stored on SSDC datacenter through a REST Api. Agilepy automatically handles the data and no actions are required from the user. For more information visit this page. This major release includes many new important features and a general refactoring.

Installation with Anaconda

Agilepy (and its dependencies) can be easily installed using Anaconda. You just need to decide the name of the virtual environment that will be created by anaconda.

conda config --add channels conda-forge
conda config --add channels plotly
conda create -n <virtualenv_name> -c agilescience agilepy

Note

If you want to try agilepy’s new features that are not officially released yet, a develpoment environment called agilepy-environment is available into Anaconda cloud. It contains all the dependencies unless agilepy, which must be installed by hand cloning the repository. Check the installation instructions here

Supported platforms:

  • linux-64

  • osx-64

Note

An experimental package for IBM POWER architecture(ppc64le) is available on Anaconda cloud. Due to some incompability this package does not contain ROOT and AGILE science tools that need to be installed from source. Check the instructions to install AGILE science tools here

Tested on:

  • CentOs 7.6

  • Ubuntu 18.04

  • Ubuntu 19.10

  • Ubuntu 20.04

  • macOs 10.14

  • macOs 10.15

  • macOS 12.0.1

In order to use the software you need to activate the virtual environment first:

conda activate <virtualenv_name>

or

source activate <virtualenv_name>

Running jupyter server:

start_agilepy_notebooks.sh

Installation with Docker

You can pull the image directly from dockerhub using the following command:

docker pull agilescience/agilepy:release-<version>

Note

Check the installation instructions for Docker here

Note

If you want to try agilepy’s new features that are not officially released yet, you need to pull a develop image available using agilepy:develop-latest tag

Using this command you can launch the container and automatically start jupyter notebook.

docker run --rm -it -p 8888:8888 \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
-v $PWD/shared_dir:/shared_dir \
agilescience/agilepy:release-<version> /bin/bash -c \
"source /opt/anaconda3/etc/profile.d/conda.sh && conda activate agilepydev && \
jupyter notebook --ip='*' --port=8888 --no-browser --allow-root --notebook-dir="/Agilepy/agilepy/notebooks" --NotebookApp.token='' --NotebookApp.password=''"

shared_dir must be created before launching the command, it is not necessary, but useful for several cases (exporting analysis outside the container, link another dataset etc.)

Jupyter server is at localhost:8888

Agilepy’s containers can be found at dockerhub page

Supported platforms:

  • linux-64

  • osx-64

  • win-64(see note)

Tested on:

  • CentOs 7.6

  • Ubuntu 18.04

  • Ubuntu 19.10

  • Ubuntu 20.04

  • macOs 10.14

  • macOs 10.15

  • Windows 10 v2004 (May 2020 Update)

Note

It’s possible to run Agilepy’s container in Windows10(still not supported by Anaconda installation), in order to do that, you need to install WSL2 and docker first.

Check the installation instructions for WSL2 here and docker here

Manual Installation

If the isntallation does not work with the instructions above, it is recommended to install agilepy and its dependencies from scratch. The dependencies required by Agilepy are:

Root 6.26 Cfitsio 4.1 Zlib

AGILE’s Science Tools (the correct tag to install is on sciencetools_version.txt in the repository main directory)

Agilepy python dependencies

Uninstalling

Anaconda

conda env remove --name <virtualenv_name>

Docker

docker rmi agilescience/agilepy:release-<version>

Package distribution structure

The virtual environment <virtualenv_name> folder is under the “envs” folder within the root folder of your anaconda installation.

It contains all the dependencies Agilepy requires. Here, there is the “agiletools” directory, containing AGILE’s scientific software.

Quickstart guide

To import the library:

from agilepy.api import AGAnalysis

You can create the (required) yaml configuration file, calling the following static method:

AGAnalysis.getConfiguration(
      "./agconfig.yaml", # the destination path of the configuration file
      "username", # the name of the flare advocate
      "OJ287", # the name of the source
      58930, # tmin
      58936, # tmax
      "MJD", # time type
      206.8121188769472, # glon
      35.8208923457401, # glat
      "$HOME/agilepy_analysis", # the destination path of the output directory
      1, # the verbosity level
      evtfile="/AGILE_PROC3/FM3.119_ASDC2/INDEX/EVT.index", # optional parameter
      logfile="/AGILE_PROC3/DATA_ASDC2/INDEX/LOG.log.index" # optional parameter
)

In order to interact with the library you need to obtain an instance of the AGAnalysis class:

ag = AGAnalysis('agconfig.yaml')

Then you have to load the models of the sources (you can filter them by their distance (degree) from l,b provided within the configuration file):

sources = ag.loadSourcesFromCatalog('2AGL', rangeDist=(0, 10))

Keyword arguments can be passed via setOptions() to override configuration parameters:

ag.setOptions(binsize=0.50, outdir="./output")

To generate sky maps:

maplistfile = ag.generateMaps()

To display and interact with the sky maps:

ag.displayCtsSkyMaps(smooth=True, sigma=3)
ag.displayExpSkyMaps()
ag.displayGasSkyMaps()

To perform an maximum likelyhood estimation analysis:

sourcefiles = ag.mle()

You can query the sources with an arbitrary boolean expression string..

selectedSources = ag.selectSources("flux > 0 AND dist <= 1 OR sqrtTS > 3")

..and fix or free a source’s parameter:

sourcefiles = ag.freeSources('name == "CYGX3"', "flux", True)

You can generate a light curve data file with…

lightCurveData = ag.lightCurveMLE("CYGX3", tmin=58930 , tmax=58936, binsize=10800)

…and display the interactive light curve plot with:

ag.displayLightCurve("mle")

If you want to manually update the value of a source’s spectrum parameter, you can do it with:

sources = ag.selectSources('name == "2AGLJ2021+4029"')
source = sources.pop()
source.spectrum.set("index", 1.8)

Hint

Check out the API documentation and the Jupiter notebooks section!

Jupyter Notebooks

Several Jupyter notebooks are available. You can start a Jupyter server calling:

start_agilepy_notebooks.sh

Tutorial notebooks

There’re several categories of tutorial notebooks:

  • science_api_tutorial: the most important ones. They show the basic usage of Agilepy to perform a scientific analysis for sources detection. The following notebooks are useful example of the use of Agilepy and they can be runned using provided sample data:

    • VELA: analysis of Vela region

    • 3C454.3: analysis of November’s 2010 gamma-ray flare of AGN 3C454.3.

    • AITOFF: how to produce a full sky AITOFF projection image.

  • Wavelet analysis: it shows how to use the Agilepy’s wavelet analysis API.

  • engineering_api_tutorial: they show how to use the Agilepy’s engineering analysis API.

The following notebook is another useful example of the use of Agilepy that is runned downloading AGILE data from SSDC website:

Analysis notebooks

These notebooks have been developed for internal purposes of the AGILE Team. A template notebook is also provided to speed up the development of a new analysis notebook for AGILE Flare Advocate team.

Download AGILE-GRID data

The AGILE-GRID data can be downloaded in two ways: automated by Agilepy or manually.

Automated download using SSDC REST Api

The AGILE-GRID data is download automatically by agilepy.

Version 1.5.0 has implemented the SSDC REST Api in order to get the AGILE dataset from SSDC datacenter. All the required data is downloaded according to tmin and tmax values selected in configuration file. This feature works when generatesmaps method is called.

Pre-requisites:
  • Internet connection (> 200 Mb/s)

SSDC Data policy:
  • EVT files contain 15 days of data (2 files per month)

  • LOG files contain 1 day of data

Eg for getting data from 10/10/2018 to 05/11/2018 it returns:
  • 3 EVT files (30/09-15/10, 15/10-31/10, 31/10-15/11)

  • 26 LOG files, one file for each day

Two query files are created to keep track of the query history and to implement the policy above. Before calling Rest api, Agilepy checks if the dates selected are in query files, if True download is not performed. If False Agilepy downloads the data in /tmp/ folder, it unpacks them into the selected datapath and it automatically calls indexgen tool for generating index files. Finally, it updates query files.

Example

AGAnalysis.getConfiguration(
        confFilePath = confFilePath,
        evtfile=None,
        logfile=None,
        userName = "username",
        sourceName = "PKS1510-089",
        tmin = 54891,
        tmax = 54921,
        timetype = "MJD",
        glon = 351.29,
        glat = 40.13,
        outputDir = "$HOME/agilepy_analysis",
        verboselvl = 0,
        userestapi=True,
        datapath="$HOME/agile_dataset"
)

Advanced Information

Query files

Agilepy uses text files called “qfiles”. These files contain the slots requested by the user, according to SSDC policies. With query file it is possible to avoid multiple downloads for the same dates (useful for slow connections and if there are no data in the selected range days).

Index files

Index file is created by Indexgen tool immediately after the download. No action is required from the user.

Plotting index files vs query files

A function to plot the dates from index and query file can be useful to check differences between asked data and real data. Sometimes could happen that data are not available for several reasons (instruments off etc), data will not be downloaded but agilepy writes in query files in order to not perform a second request.

AGILE Data Coverage

SSDC uploads AGILE dataset once per month, this means that it could not be possible to select a date close to the present day. In this particular case, query files must not be uploaded, because in the future data will be available. Agilepy gets AGILE data coverage from SSDC and writes it into a file called Agilepy_coverage, when AGDataset starts it checks if last coverage is more than 60 days old from the present date and it updates it if positive.

Manual download

The AGILE-GRID data obtained both in pointing and in spinning mode are publicly available and can be download manually from the ASI/SSDC https://www.asdc.asi.it/mmia/index.php?mission=agilemmia

Prepare index files

There’re two types of data files: events list (EVT) and log data (LOG). They both are compressed fits files. Each file refers to a specific time interval.

Example:

agql1511240600_1511240730.LOG.gz
agql1511240730_1511240900.EVT__FM.gz

In order to use Agilepy (or the Agile science tools) a special file, called “index”, is needed. This file is used by Agilepy to know the position of the data files and which file refers to which interval. Two index files are needed: one for the event data and one for the log data.

Those index files have four column:

  • file name

  • time start of the file in Terrestrial Time (TT)

  • time end of the file in Terrestial time (TT)

  • LOG or EVT marker to identify the fole types

Here some examples of LOG and EVT indexes:

head -n 3 /ASDC_PROC3/DATA_ASDCe/INDEX/LOG.log.index
/AGILE_PROC3/DATA_ASDCe/LOG/ag-107092735_STD0P_GO.LOG.gz 107092735. 107179134.9 LOG
/AGILE_PROC3/DATA_ASDCe/LOG/ag-107179135_STD0P_GO.LOG.gz 107179135. 107265534.9 LOG
/AGILE_PROC3/DATA_ASDCe/LOG/ag-107265535_STD0P_GO.LOG.gz 107265535. 107351934.9 LOG


head -n 3 /ASDC_PROC3/FM3.119_ASDCSTDk/INDEX/EVT.index
/ASDC_PROC3/FM3.119_ASDCSTDk/EVT/ag0910311200_0911301200_STD1Kal_FM.EVT.gz 184075134.000000 186667134.000000 EVT
/ASDC_PROC3/FM3.119_ASDCSTDk/EVT/ag0911301200_0912201200_STD1Kal_FM.EVT.gz 186667134.000000 188395134.000000 EVT
/ASDC_PROC3/FM3.119_ASDCSTDk/EVT/ag0912201200_1001151200_STD1Kal_FM.EVT.gz 188395134.000000 190641534.000000 EVT

You can use the AG_indexgen tool to generate the .index file:

AG_indexgen <path to data> <type> <output file>

Where <type> can be EVT or LOG.

Example:

AG_indexgen /AGILE_PROC3/FM3.119_ASDC2/EVT EVT /home/user/data.index

Agilepy test data

The Agilepy conda package gets shipped with two subsets of the AGILE data archive for the purpose of unit testing and to show how to run scientific analysis with the tutorial notebooks.

test_dataset_6.0

A test data to analyse Vela region. The provided period is MJD 58026.50-58031.50.

The index files are the following:

evtfile="$AGILE/agilepy-test-data/test_dataset_6.0/EVT/EVT.index"
logfile="$AGILE/agilepy-test-data/test_dataset_6.0/LOG/LOG.index"

test_dataset_agn

A test data to analyse the November’s 2010 flare of 3C454.3 source. The provided period is MJD 55513.00-55520.00.

The index files are the following:

evtfile="$AGILE/agilepy-test-data/test_dataset_agn/EVT/EVT.index"
logfile="$AGILE/agilepy-test-data/test_dataset_agn/LOG/LOG.index"

Configuration file

General

A yaml configuration file is required in order to run Agilepy.

It is composed by several sections and each section holds several configuration options, some of which are optional (having a default value), some others are required.

It supports environment variables (they can be used to define file system paths).

It can be created easily, calling the following static method and passing the minimal set of (required) configuration parameters.

AGAnalysis.getConfiguration(
      confilepath="./agconfig.yaml", # the destination path of the configuration file
      userName="username", # the name of the flare advocate
      sourceName="OJ287", # the name of the source
      tmin=58930, # tmin
      tmax=58936, # tmax
      timetype="MJD", # time type
      glon=206.8121188769472, # glon
      glat=35.8208923457401, # glat
      outputDir="$HOME/agilepy_analysis", # the destination path of the output directory
      verboselvl=1, # the verbosity level
      evtfile="evt indexfile", # optional parameter
      logfile="log indexfile", # optional parameter
      datapath="datapath",
      userestapi=True,

)

The method above will create the following configuration file:

input:
  evtfile: None
  logfile: None
  userestapi: True
  datapath: datapath
output:
  outdir: $HOME/agilepy_analysis
  filenameprefix: analysis_product
  logfilenameprefix: analysis_log
  sourcename: sourcename
  username: username
  verboselvl: 2

selection:
  emin: 100
  emax: 10000
  tmin: 54935.0
  tmax: 54936.0
  timetype: MJD
  glon: 355.447
  glat: -0.2689
  proj: ARC
  timelist: None
  filtercode: 5
  fovradmin: 0
  fovradmax: 60
  albedorad: 80
  dq: 0
  phasecode: null
  lonpole: 180
  lpointing: null
  bpointing: null
  maplistgen: "None"

maps:
  mapsize: 40
  useEDPmatrixforEXP: false
  expstep: null
  spectralindex: 2.1
  timestep: 160
  projtype: WCS
  proj: ARC
  binsize: 0.25
  energybins:
    - 100, 10000
  fovbinnumber: 1
  offaxisangle: 30

model:
  modelfile: null
  galmode: 1
  isomode: 1
  galcoeff: null
  isocoeff: null
  emin_sources: 100
  emax_sources: 10000
  galmode2: 0
  galmode2fit: 0
  isomode2: 0
  isomode2fit: 0

mle:
  ranal: 10
  ulcl: 2
  loccl: 95
  expratioevaluation: true
  expratio_minthr: 0
  expratio_maxthr: 15
  expratio_size: 10
  minimizertype: Minuit
  minimizeralg: Migrad
  minimizerdefstrategy: 2
  mindefaulttolerance: 0.01
  integratortype: 1
  contourpoints: 40
  edpcorrection: 0.75
  fluxcorrection: 0

ap:
  radius: 3
  timeslot: 3600

plotting:
  twocolumns: False

Updating the configuration options

The user should not directly manipulate the configuration file, because the configuration file is read only once, when the AGBaseAnalysis constructor is called. Hence, the configuration file modification will not affect the internal configuration object. Also, updating the values held by this object will not affect the original values written on disk.

In order to update the internal configuration object, the user can rely on the following methods:

For example:

ag.setOptions(binsize=0.50, energybins=[[100, 300], [500, 1000]])
print(ag.getOption("energybins"))

Configuration options

This section describes the configuration options.

Section: ‘input’

This section defines the input data files. The input data files are indexes: each row holds the file system position of an actual event data/log file, together with the time interval it refers to. If userestapi if True the selection of evtfile and logfile is not required, Agilepy creates its own index files automatically. See more details in this link.

Option

Description

Type

Required

Default

evtfile

Path to index evt file name

str

no

None

logfile

Path to index log file name

str

no

None

userestapi

If true downloads date into datapath

bool

no

True

datapath

the position of AGILE data

str

no

None

Section: ‘output’

The output section collects options related to the output files generation and logging.

The ‘outdir’ option sets the root directory of the analysis results where all output files are written.

Agilepy use two loggers, one logs messages on the console, the other writes messages on disk. The ‘verboselvl’ option sets the verbosity of the Agilepy console logger. The Agilepy file logger verbosity is set to 2 by default. There are 4 kind of messages based on their importance factor:

  • CRITICAL: a message describing a critical problem, something unexpected, preceding a program crash or an Exception raise.

  • WARNING: an indication that something unexpected happened, or indicative of some problem in the near future (e.g. ‘disk space low’). The software is still working as expected.

  • INFO: confirmation that things are working as expected.

  • DEBUG: detailed information, typically of interest only when diagnosing problems.

Option

Description

Type

Required

Default

outdir

Path of the output directory

str

yes

null

filenameprefix

The filename prefix of each output file

str

yes

null

logfilenameprefix

The filename prefix of the log file

str

yes

null

sourcename

The name of the source under analysis

str

yes

null

userName

The name of the user performing the analysis

str

yes

null

verboselvl

0 ⇒ CRITICAL and WARNING messages are logged on the console.
1 ⇒ CRITICAL, WARNING and INFO messages are logged on the console.
2 ⇒ CRITICAL, WARNING, INFO and DEBUG messages are logged on the console

int

no

1

Section: ‘selection’

The temporal, spatial and spectral binning of the data can be customized using the configuration options of this section.

The center of the ROI (region of interest) is defined by explicit Galactic sky coordinates (glon and glat).

Option

Description

Type

Default

Required

emin

Energy min in MeV

int

100

no

emax

Energy max in MeV

int

10000

no

glat

Center of the ROI (’latitude’ or ‘b’)

float

null

yes

glon

Center of the ROI (’longitude’ or ‘l’)

float

null

yes

tmin

Minimum time (in MJD or TT)

float

null

yes

tmax

Maximum time (in MJD or TT)

float

null

yes

timetype

The date format of tmin and tmax.
Possibile values: [‘MJD’, ‘TT’]

str

null

yes

timelist

it’s a path to a file containing a list of time intervals in TT
format to generate maps
integrated within a time window.
If specified, ‘tmin’ and ‘tmax’ are ignored.

str

null

no

filtercode

filtercode = 5 select G filtercode = 0 select G+L+S

int

5

no

fovradmin

fovradmin < fovradmax

int

0

no

fovradmax

fovradmax > fovradmin (dq = 0 is necessary for setting)

int

60

no

albedorad

albedo selection cut (dq = 0 is necessary for setting)

int

80

no

dq

Data quality selection filter.
A combination of fovradmax and albedorad.
Possible values are [0,1,2,3,4,5,6,7,8,9]
dq = 0 -> albedorad and fovradmax are free and they must always be specified in setOption
dq = 1 -> albedorad=80, fovradmax=60
dq = 2 -> albedorad=80, fovradmax=50
dq = 3 -> albedorad=90, fovradmax=60
dq = 4 -> albedorad=90, fovradmax=50
dq = 5 -> albedorad=100, fovradmax=50
dq = 6 -> albedorad=90, fovradmax=40
dq = 7 -> albedorad=100, fovradmax=40
dq = 8 -> albedorad=90, fovradmax=30
dq = 9 -> albedorad=100, fovradmax=30

int

0

no

phasecode

Photon list selection parameter based
on the orbital phase. If ‘None’, the
automated selection is done following
the ‘phasecode’ rule

int

null

no

Phasecode rule
  • phasecode = 2 -> spinning mode, SAA excluded with AC counts method.

  • phasecode = 6 -> spinning mode, SAA excluded according to the magnetic field intensity (old definition of SAA, defined by TPZ)

  • phasecode = 18 -> pointing mode, SAA and recovery exluded.

It is suggested to use phasecode = 2 for data taken in spinning mode.

def setPhaseCode(tmax)
   if @phasecode == -1
      if tmax.to_f >= 182692800.0
         @phasecode = 6 #SPIN
      else
         @phasecode = 18 #POINTING
      end
   end
end
filtercode rule

A set of different on-board triggers enables the discrimination of background events (mainly cosmic rays in the AGILE Low Earth Orbit) from gamma-ray events. The data processing of the GRID events use an additional on-ground filters and provides a classification of each event:

  • P : events classified as a charged particle and rejected

  • G : events classified as gamma-ray photons. This is the most useful class for the analysis

  • S : events classified as single-track: this is a special class of events with no separation between the electron and positron tracks

  • L : limbo events, not clearly classified.

The events provided in the EVT files are of type G, S, and L. The AGILE team recommends to use the G class for scientific analysis. Only for gamma-ray bursts or other short transient events, and for pulsar timing analysis the G, S and L classes should be used together.

Section: ‘maps’

These options control the behaviour of the sky maps generation tools. The ‘energybin’ and ‘fovbinnumber’ options set the number of maps that are generated:

number of maps = number of energy bins * fovbinnumber

The ‘energybin’ option is a list of strings with the following format:

energybins:
  - 100, 1000
  - 1000, 3000

The ‘fovbinnumber’ option sets the number of bins between ‘fovradmin’ and ‘fovradmax’ as:

number of fov bins = (fovradmax-fovradmin)/fovbinnumber

Note

One map is generated for each possible combination between the ‘energybin’ (emin, emax) and the ‘fovbinnumber’ (fovmin, fovmax). The order of map generation is described by the following pseudocode:

For each fovmin..fovmax:
For each emin..emax:
generateMap(fovmin, fovmax, emin, emax)

Option

Description

Type

Default

Required

mapsize

Width of the ROI in degrees

float

40

no

useEDPmatrixforEXP

Use the EDP matrix to generate the exposure map.

boolean

False

no

expstep

Step size of the exposure map, if ‘None’ it depends by
round(1 / binsize, 2) (e.g. 0.3->3, 0.25->4, 0.1->10)

int

None

no

spectralindex

Spectral index of the exposure map

float

2.1

no

timestep

LOG file step size of exposure map (LOG file are at 0.1s)

float

160

no

projtype

Projection mode. Possible values: [’WCS’]

str

WCS

no

proj

Spatial projection for WCS mode.
Possible values: [’ARC’, ‘AIT’]

str

ARC

no

skytype

gasmap:
0) SKY000-1 + SKY000-5,
1) gc_allsky maps + SKY000-5,
2) SKY000-5
3) SKY001 (old galcenter, binsize 0.1, full sky),
4) SKY002 (new galcenter, binsize 0.1, full sky)

int

4

no

binsize

Spatial bin size in degrees

float

0.25

no

energybin

The enegy bins of analysis. A list of value. | To configure: | 1) directly in the yaml configuration file; | 2) Use the method e.g. ag.setOptions(energybins=[[100, 300], [500, 1000]]) | 3) Use the method ag.setOptionEnergybin(value)

List<String>

[100, 10000]

no

fovbinnumber

Number of bins between fovradmin and fovradmax.
Dim = (fovradmax-fovradmin)/fovbinnumber

int

1

no

Section: ‘model’

The ‘galcoeff’ and ‘isocoeff’ options values can take the default value of null or they can be a a list of values separated by a comma. If they are set to null it means they are free to change.

model:
  galcoeff: 0.8, 0.6, 0.5, 0.4
  isocoeff: 8, 10, 12, 14

In this case, you should pay attention on how the sky maps are generated: the following example show which iso/gal coefficients are assigned to which map.

selection:
  fovradmin: 0
  fovradmax: 60

maps:
  energybins:
    - 100, 300
    - 300, 1000
  fovbinnumber: 2

model:
  galcoeff: 0.8, 0.6, 0.5, 0.4
  isocoeff: 8, 10, 12, 14
FOV bins:
(0, 30), (30, 60)
Map #1 has: fovmax:0 fovmax:30 emin:100 emax:300 galcoeff:0.8 isocoeff:8
Map #2 has: fovmax:0 fovmax:30 emin:300 emax:1000 galcoeff:0.6 isocoeff:10
Map #3 has: fovmax:30 fovmax:60 emin:100 emax:300 galcoeff:0.5 isocoeff:12
Map #4 has: fovmax:30 fovmax:60 emin:300 emax:1000 galcoeff:0.4 isocoeff:14

Option

Description

Type

Default

Required

modelfile

A file name that contains point
sources, diffuse and isotropic components

string

null

yes

galmode

int

1

no

isomode

int

1

no

galcoeff

set into .maplist if >= 0

null, float or str

null

no

isocoeff

set into .maplist if >= 0

null, float or str

null

no

emin_sources

energy min of the modelfile

int

100

no

emax_sources

energy max of the modelfile

int

10000

no

galcoeff and isocoeff

galcoeff and isocoeff are the coefficients for the Galactic and isotropic diffuse emission components respectively. The values may be fixed during the fitting process or some or all of them may be optimized by allowing them to vary. Agilepy allows to evaluate these coefficient and fix them or to keep these coefficient free.

Positive values are considered fixed, while negative values are free to vary starting from their absolute values. These coefficients are affected by the galmode and isomode coefficients described in the following section.

galmode and isomode

‘galmode’ and ‘isomode’ are integer values describing how the corresponding coefficients ‘galcoeff’ or ‘isocoeff’ found in all the lines of the maplist will be used:

0: all the coefficients are fixed.
1: all the coefficients are fixed if positive, variable if negative (the absolute value is the initial value). This is the default behaviour.
2: all the coefficients are variable, regardless of their sign.
3: all the coefficients are proportionally variable, that is the relative weight of their absolute value is kept.

Section: ‘mle’

The maximum likelihood estimation analysis is configured by the following options:

Option

Description

Type

Default

Required

ranal

Radius of analysis

float

10

No

ulcl

Upper limit confidence level, expressed as sqrt(TS)

float

2

No

loccl

Source location contour confidence level (default 95 (%)confidence level). Possible values: [ 99, 95, 98, 50]

int

95

No

fluxcorrection

Correction of the flux taking into account the spectral model. Possible values: [0 (no correction), 1 (enable correction)].

int

0

No

Exp-ratio evaluation options

See details in this link.

Option

Type

Default

Required

Description

expratioevaluation

bool

yes

none

expratio_minthr

float

0

none

expratio_maxthr

float

15

none

expratio_size

float

10

none

Section: ‘ap’

This section describes the configuration parameters for the Aperture Photometry analysis.

Option

Description

Type

Required

Default

radius

The radius of analysis

float

no

3

timeslot

The size of the temporal bin

int

no

3600

Section: ‘plot’

This section defines the plotting configuration.

Option

Description

Type

Required

Default

twocolumns

The plot is adjusted to the size of a two column journal publication

boolean

False

no

Sources file

The sources can be defined using one of two different formats: xml document and text file.

The flux parameter estimates are relevant in the fitting process, as the sources are considered one by one, starting with the one with the brightest initial flux value, regardless of the order they are given in the source file.

Spectral models

A full energy band spectral fit of the data is performed with different spectral model. The spectral representations used in the BUILD25 are PL, exponential cut-off PL, super-exponential cut-off PL, and log parabola (LP). More details are reported in https://arxiv.org/abs/1903.06957

The PL spectral model is used for all sources that are not significantly curved and have low exposure,

_images/pl.png

where N0 is the prefactor and alpha is the index explicitly evaluated by the MLE method. Our MLE spectral fitting does not explicitly output the prefactor value, which is internally calculated by the numerical procedure. The majority of the AGILE sources are described by a PL.

The exponential cut-off PL spectral model (PC) is

_images/pc.png

where N0 is the prefactor, α is the index, and Ec is the cut-off energy. The values Ec and α are explicitly provided by the MLE method.

The super exponential cut-off PL spectral model (PS) is

_images/ps.png

where N0 is the prefactor, α is the first index, β the second index, and Ec is the cut-off energy. The parameters α, Ec, and β are explicitly provided by the MLE method.

The LP spectral model is

_images/lp.png

where N0 is the prefactor, Ec is the pivot energy, α is the first index, β the curvature. The parameters α, Ec, and β are explicitly provided by the MLE method.

The selection of curved spectra followed the acceptance criteria described in bulgarelli19. Briefly, a source is considered significantly curved if T Scurved > 16, where T Scurved = 2 × (log L(curved spectrum)−log L(power law), where L is the likelihood function obtained changing only the spectral representation of that source and refitting all free parameters.

Source library format (xml document)

<?xml version="1.0" ?>
<source_library title="source library">

  <!-- Point Sources -->
  <source name="2AGLJ2202+4214" type="PointSource">
    <spectrum type="PowerLaw">
      <parameter name="flux" free="1"  value="7.45398e-08"/>
      <parameter name="index" free="1" scale="-1.0" value="1.96903" min="0.5" max="5"/>
    </spectrum>
    <spatialModel type="PointSource" location_limit="0">
      <parameter name="pos" value="(92.4102, -10.3946)" free="0" />
    </spatialModel>
  </source>

  <source name="2AGLJ0007+7308" type="PointSource">
    <spectrum type="PLExpCutoff">
       <parameter name="flux" free="1"  value="41.6072e-08"/>
       <parameter name="index" free="1" scale="-1.0" value="1.29082" min="0.5" max="5"/>
       <parameter name="cutoffEnergy" free="1" scale="-1.0" value="2003.9" min="20" max="10000"/>
    </spectrum>
    <spatialModel type="PointSource" location_limit="0">
       <parameter name="pos" value="(119.677, 10.544)" free="0" />
    </spatialModel>
  </source>

  <source name="2AGLJ0835-4514" type="PointSource">
    <spectrum type="PLSuperExpCutoff">
      <parameter name="flux" free="1"  value="969.539e-08"/>
      <parameter name="index1" free="1" scale="-1.0" value="1.71345" min="0.5" max="5"/>
      <parameter name="cutoffEnergy" free="1" value="3913.06" min="20" max="10000"/>
      <parameter name="index2" free="1" value="1.3477" min="0"  max="100"/>
    </spectrum>
    <spatialModel type="PointSource" location_limit="0">
      <parameter name="pos" value="(263.585, -2.84083)" free="0" />
    </spatialModel>
  </source>

  <source name="2AGLJ1801-2334" type="PointSource">
    <spectrum type="LogParabola">
      <parameter name="flux" free="1"  value="35.79e-08"/>
      <parameter name="index" free="1" scale="-1.0" value="3.37991" min="1" max="4"/>
      <parameter name="pivotEnergy" free="1" scale="-1.0" value="2935.07" min="500" max="3000"/>
      <parameter name="curvature" free="1" scale="-1.0" value="0.682363" min="0.1" max="3"/>
    </spectrum>
    <spatialModel type="PointSource" location_limit="0">
      <parameter name="pos" value="(6.16978, -0.0676943)" free="1" />
    </spatialModel>
  </source>

</source_library>

Working with sources

The Source abstraction

The main abstraction of Agilepy is the Source class. It is described by several parameters, some of which can be free to vary, and they are changed by the mle() analysis.

The set of the parameters describing the source can vary, depending on the spectrum and spatial model types of the source.

The different types of sources are described here.

How to load or add new sources

In order to perform a scientific analysis with Agilpy, at least one Source model must be loaded. There are several ways to do that.

The loadSourcesFromCatalog(catalogName, rangeDist=0, inf, show=False) allows to load a source catalog, while filtering the sources by their distance (degree) from the l,b position provided within the configuration file.

sources = ag.loadSourcesFromCatalog('2AGL', rangeDist=(0, 10))

The loadSourcesFromFile(sourcesFilepath, rangeDist=0, inf, show=False) loads the sources, reading their model from a file.

The addSource(sourceName, sourceDict) method allows the user to define on the fly a source model with a python dictionary. Check the tutorial notebooks for an example.

How to select Source objects

The sources can be selected via the the selectSources(selection, show=False) method. The “selection” argument supports either lambda functions and boolean expression strings. The user can call selectSources (with show=True) to show the source description

 source = ag.selectSources('name == "2AGLJ2254+1609"', show=False).pop()
 print(source)
 -----------------------------------------------------------
 Source name: 2AGLJ2254+1609 (PointSource)
  * Free parameters: flux
  * Initial source parameters: (PowerLaw)
     - flux(ph/cm2s): 7.50937e-07
     - index: 2.20942
     - Source position: (86.1236, -38.1824) (l,b)
     - Distance from map center: 0.011 deg
-----------------------------------------------------------

Other examples:

sources = ag.selectSources('name == "PKS1510-089"', show=False)
sources = ag.selectSources('flux > 0', show=False)
sources = ag.selectSources(lambda name, sqrtTS: name == "2AGLJ2021+4029" AND sqrtTS> 0, show=False)

How to let the source’s parameters to vary

In order to free or fix a sources’ parameter, the user can rely on the freeSources(selection, parameterName, free, show=False) method. The “selection” argument is used like in selectSources, so you can free a parameter of multiple sources at once.

aganalysis.freeSources(lambda name, dist, flux : Name == "2AGLJ2021+4029" AND dist > 0 AND flux > 0, "flux", True)
ag.freeSources('name == "2AGLJ1513-0905"', "index", True, show=True)

Check the api documentation or the tutorial notebooks for additional examples.

How to check which source’s parameters are free to vary

The user can obtain this information by printing the Source object or calling the getFreeParams() method of the Source object.

print(source.getFreeParams())
['flux']

The “multi” description of a Source object

If the user performs an mle analysis, the Source object will contain also the analysis results.

 print(source)
 -----------------------------------------------------------
 Source name: 2AGLJ2254+1609 (PointSource) => sqrt(ts): 10.2226
  * Free parameters: flux index
  * Initial source parameters: (PowerLaw)
  - flux(ph/cm2s): 7.50937e-07
  - index: 2.20942
  - Source position: (86.1236, -38.1824) (l,b)
  - Distance from map center: 0.011 deg
  * Last MLE analysis:
  - flux(ph/cm2s): 8.68363e-06 +/- 1.62474e-06
  - index: 2.51001 +/- 0.173795
  - upper limit(ph/cm2s): 1.13967e-05
  - ergLog(erg/cm2s): 1.39217e-09 +/- 2.6048e-10
  - galCoeff: [0.7, 0.7, 0.7, 0.7, 0.7]
  - isoCoeff: [5.24757, 3.14662, 0.953512, 1.59944e-10, 0.557554] +/- [0.519283, 0.209362, 0.051478, 0.000108147, 0.000720814]
  - exposure(cm2s): 13672200.0
  - exp-ratio: 0.0
  - L_peak: 86.1236
  - B_peak: -38.1824
  - Distance from start pos: 0.0
  - position:
      - L: -1.0
      - B: -1.0
      - Distance from start pos: -1.0
      - radius of circle: -1.0
      - ellipse:
      - a: -1.0
      - b: -1.0
      - phi: -1.0
-----------------------------------------------------------

The values L_peak and B_peak set to the initial values in the source location is fixed. If it is allowed to vary then they are set to the position for which the TS is maximized. If a confidence contour was found, the parameters of the “ellipse” section describe the best-fit ellipse of the contour, described in detail below. The counts and fluxes are provided, as well as their symmetric, positive, and negative errors if the flux is allowed to vary. For convenience, the exposure of the source, used to calculate the source counts from the flux, is also provided. Finally, the spectral index and its error, or the other spectral parameters, if applicable, are provided.

How to manually inspect source’s attributes

The user can rely on a getter method get(sourceAttribute) method.

print(source.get("cutoffEnergy"))
print(source.get("index"))
print(source.get("pos"))
print(source.get("dist"))
print(source.get("locationLimit"))
print(source.get("multiFlux"))

How to manually change a source’s attributes

The user can rely on a setter method set(sourceAttribute) method.

source.set("index2", 1.34774)

The setAttributes() method allows to change the following attributes: value, free, scale, min, max, locationLimit. Example:

source.spectrum.cutoffEnergy.setAttributes(min=3000, max=5000)

In order to change the position of a source, the user can rely on the updateSourcePosition(sourceName, glon, glat) method.

Products

Sky maps

.cts.gz’ file

Counts maps are generated by the procedure AG_ctsmapgen embedded into Agilpy.

AG_ctsmapgen reads the event files listed in the event file index (see “AGILE data” section), bins the counts between tmin and tmax, and outputs a FITS image file. The image is a two-dimensional array in the ARC or AIT projection. The projection, size and resolution, the center and rotation of the map in Galactic coordinates, tmin, tmax, emin, and emax, along with various integration parameters (fovrad, fovradmin, albrad, phasecode) are managed by Agilepy.

The parameters are described in the “Configuration file” section.

_images/analysis_product_EMIN00100_EMAX10000_01.cts.png

.exp.gz’ file

Exposure maps are generated by the procedure AG_expmapgen embedded into Agilpy.

The task AG_expmapgen reads the log files listed in the LOG index (see “AGILE data” section), integrates the exposure between tmin and tmax, and outputs a FITS exposure image file. The image is a two-dimensional array in either the ARC or AIT projection. The projection, size and resolution, and center and rotation of the map in Galactic coordinates, tmin, tmax, emin, emax, and index file are managed by Agilepy, along with various integration parameters (fovrad, albrad, y tol, roll tol, earth tol, phasecode), and an interpolation step size (binstep). The interpolation procedure is a linear interpolation method in which only one bin each N is calculated (where N is the step size parameter). For a bin size of 0.5 deg or 0.25 deg with a step size of N = 4 it is possible to get a good approximation of the exposure map.

The parameters are described in the “Configuration file” section.

_images/analysis_product_EMIN00100_EMAX10000_01.exp.png

.gas.gz’ file

Diffuse emission maps are generated by the procedure AG_gasmapgen embedded into Agilpy. AG_gasmapgen reads an exposure map produced by AG_expmapgen and the master diffuse emission map and outputs a FITS image file, in the same format as the exposure map, in which each pixel contains the diffuse emission in that pixel. The image is a square array in the ARC projection. The diffuse emission map contain models of the diffuse emission convolved with the energy-dependent point spread function and combined into predefined observed energy ranges according to the appropriate energy dispersion function for G events using the FM3.119 background filter. The diffuse emission map automatically selected by Agilepy based on the energy range of the analysis; e.g. if the analysis is performed between 100 MeV and 50 GeV, Agilepy select the file 100_50000.0.1.SFMG_H0025.conv.sky.gz. The first number in the file name is the minimum energy and the second number is the maximum energy, followed by the resolution of the maps (0.1), the background event rejection filter (FM3.119) and the instrument response functions (IRFs).

_images/analysis_product_EMIN00100_EMAX10000_01.gas.png

.int.gz’ file

Intensity maps are generated by the procedure AG_intmapgen embedded into Agilepy. AG_intmapgen reads an exposure map produced by AG_expmapgen and a counts map produced by AG_ctsmapgen and outputs a FITS image file, in the same format as the counts map, in which each pixel contains the intensity in that pixel. The image is a square array in the ARC projection. The two input maps should have been produced using the same set of parameters. The intensity map is not used in scientific analysis; it is useful solely as a visualization tool.

_images/analysis_product_EMIN00100_EMAX10000_01.int.png

Result of the Maximum Likelihood Estimator

Agilepy shows a high-level view of the results of the maximum likelihood estimator. See this link for more details.

The details of the output of the science tool AG_multi that performs the likelihood procedure is still accessible. This section describe the output of the AG_multi science tool, that performs a Maximum Likelyhood Estimator analysis to find the best position, flux and spectral parameters of a list of sources given set of count maps.

Confidence Contour files

If a confidence contour was found, the parameters on the following line describe the best-fit ellipse of the contour, described in detail below.

If source location was requested for a given source and a source location contour was found, then three additional files are generated for that source. These files are written using galactic coordinates in degrees and can be loaded by applications such as ds9 and overlaid on the maps provided as input to AG_multi to visualize the source location contours. One of the three files, with extension .con, contains the source contour as found by the ROOT functions, expressed as a list of galactic coordinates, one point per line, where the last line is a repetition of the first. It may depict any shape. The other two files describe the ellipse that best fits the contour. One has extension .ellipse.con and represents the ellipse as a contour in a format analogous to that of the .con file. The other has extension .reg and describes same ellipse by its axes and orientation.

Determination of the ellipse. If AG_multi was able to find a source contour, an ellipse is fit to the contour. The source contour is a list of points which defines a polygon by connecting each point sequentially. The value of Radius found in the HTML output is the radius in degrees of a circle with the same area as the polygon. AG_multi determines the ellipse which best fits the contour. This ellipse will have the same area as the polygon, and the distance between each contour point and the intersection between the ellipse and the line connecting that point to the centre will be minimized. The ellipse is completely described by three parameters: the two axes and the rotation (in degrees) of the first axis around the centre, as expected by the ds9 application. If the ellipse is a circle, its axes will both be equal to the Radius found in the HTML output. The ellipse is described by two files that are readable by ds9: one is a .reg file which contains the centre, the axes and the rotation of the ellipse, while the other describes the same ellipse as a list of points in galactic coordinates, thus using the same syntax of a contour file, and has extension .ellipse.con. This is an example of ellipse .reg file:

ExpRatio

Owing to the non-homogeneous sky coverage of the AGILE observations, it is possible that sources lie near the borders of certain pointings. In order to have an unbiased estimate of the coeffcients of the Galactic diffuse emission and isotropic background that could lead to to an incorrect evaluation of the flux and position of the source, exposure uniformity within the region of the analysis is required. We applied a specific check to verify the uniformity of the exposure within the 10-degree radius of the AGILE MLE analysis centred at each source candidate position, over the considered timescale. The fraction of pixels of the exposure map within the region of analysis having a value below a pre-defined threshold was calculated, and if it was more than 10% the region was considered unreliable and the candidate was discarded. The exposure threshold value is evaluated by calculating the mean exposure of the observation over the full FoV area and comparing this exposure with the values of some reference good exposures.

The parameters expratioevaluation, expratio_minthr, expratio_maxthr, expratio_size described here.

Light curves

AGILE-GRID light curves can be created in two different ways:

  • using a maximum likelihood estimator analysis

  • using aperture photometry.

The likelihood analysis reach better sensitivity, more accurate flux measurement, better evaluation of the backgrounds and can work with a detailed source models where more sources can be considered at the same time.

Aperture photometry provides a raw measure of the flux of a sigle source and is less computing demanding.

The likelihood light curve file contains the results of the generation of a light curve. The columns described are the following sections.

Time of the analysis in MJD:

  • time_start_mjd: time start (MJD)

  • time_end_mjd: time end (MJD)

Result of the analysis of the single source:

  • sqrt(ts): the square root of the Test Statistic value of the results of the maximum likelihood estimator (mle)

  • flux (ph/cm2/s/sr)

  • flux_err (ph/cm2/s/sr)

  • flux_ul (ph/cm2/s/sr)

  • gal: the value of the galactic diffuse emission (gal) parameter

  • gal_error: the error of the galactic diffuse emission (gal) parameter

  • iso: the value of the isotropic emission (iso) parameter

  • iso_error: the error of the isotropic emission (iso) parameter

  • (l_peak, b_peak): position in Galactic coordinate (l_peak, b_peak): peak coordinates. If it is allowed to vary then they are set to the position for which the TS is maximized.

  • dist_peak: distance between current l_peak, b_peak and previous position

  • (l, b): position in Galactic coordinate evaluated by mle with the determination of the 95% confidence level elliptical confidence region

  • r: radius of 95% c.l. circular confidence region, deg. Statistical error only

  • ell_dist: the distance between (l,b) and the initial position

  • a: the semimajer axis of the elliptical confidence region

  • b: the semiminor axis of the elliptical confidence region

  • phi: rotation of the elliptical confidence region

  • exposure

  • ExpRatio: see above section

  • counts

  • counts_err

  • Index

  • Index_Err

  • Par2

  • Par2_Err

  • Par3

  • Par3_Err

  • Erglog

  • Erglog_Err

  • Erglog_UL

Time of the analysis in UTC and TT:

  • time_start_utc

  • time_end_utc

  • time_start_tt

  • time_end_tt

The following are the initial parameters of the analysis of the source:

  • Fix: initial fixflag

  • index: initial spectral index

  • ULConfidenceLevel

  • SrcLocConfLevel

  • start_l: initial Galactic coordinate l

  • start_b: initial Galactic coordinate b

  • start_flux: inital flux for the MLKE

  • typefun: type of spectral model

  • par2: initial value of par2

  • par3: initial value of par3

The following are the parameters of the MLE analysis:

  • galmode2

  • galmode2fit

  • isomode2

  • isomode2fit

  • edpcor

  • fluxcor

  • integratortype

  • expratioEval

  • expratio_minthr

  • expratio_maxthr

  • expratio_size

Parameters of the maps:

  • Emin

  • emax

  • fovmin

  • fovmax

  • albedo

  • binsize

  • expstep

  • phasecode

Technical results of the fitting:

  • fit_cts

  • fit_fitstatus0

  • fit_fcn0

  • fit_edm0

  • fit_nvpar0

  • fit_nparx0

  • fit_iter0

  • fit_fitstatus1

  • fit_fcn1

  • fit_edm1

  • fit_nvpar1

  • fit_nparx1

  • fit_iter1

  • fit_Likelihood1

time_start_mjd time_end_mjd sqrt(ts) flux flux_err flux_ul gal gal_error iso iso_error l_peak b_peak dist_peak l b r ell_dist a b phi exposure ExpRatio counts counts_err Index Index_Err Par2 Par2_Err Par3 Par3_Err Erglog Erglog_Err Erglog_UL time_start_utc time_end_utc time_start_tt time_end_tt Fix index ULConfidenceLevel SrcLocConfLevel start_l start_b start_flux typefun par2 par3 galmode2 galmode2fit isomode2 isomode2fit edpcor fluxcor integratortype expratioEval expratio_minthr expratio_maxthr expratio_size Emin emax fovmin fovmax albedo binsize expstep phasecode fit_cts fit_fitstatus0 fit_fcn0 fit_edm0 fit_nvpar0 fit_nparx0 fit_iter0 fit_fitstatus1 fit_fcn1 fit_edm1 fit_nvpar1 fit_nparx1 fit_iter1 fit_Likelihood1
58026.49921296296 58027.49921296296 7.3538 944.812e-08 213.193e-08 1419.5e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3718660.0 1.99923 35.1344 7.92794 1.71345 0 3913.06 0 1.34774 0 1.67967e-09 3.7901e-10 2.52355e-09 2017-09-30T11:58:52.000 2017-10-01T11:58:52.000 433857532.0 433943932.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 92 -1 344.873 0.5 0 52 3 0 317.834 5.85172e-17 1 52 3 404.289
58027.49921296296 58028.49921296296 8.87831 1055.24e-08 211.709e-08 1524.18e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3843970.0 3.11419 40.5633 8.13802 1.71345 0 3913.06 0 1.34774 0 1.87599e-09 3.76372e-10 2.70965e-09 2017-10-01T11:58:52.000 2017-10-02T11:58:52.000 433943932.0 434030332.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 91 -1 341.314 0.5 0 52 3 0 301.902 5.88891e-17 1 52 3 391.515
58028.49921296296 58029.49921296296 7.31495 820.826e-08 198.261e-08 1266.82e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3788250.0 1.99923 31.095 7.51063 1.71345 0 3913.06 0 1.34774 0 1.45925e-09 3.52464e-10 2.25212e-09 2017-10-02T11:58:52.000 2017-10-03T11:58:52.000 434030332.0 434116732.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 96 -1 357.958 0.5 0 52 3 0 331.204 1.01185e-16 1 52 3 423.045
58029.49921296296 58030.49921296296 6.78978 840.67e-08 208.19e-08 1307.27e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3806190.0 3.11419 31.9975 7.92411 1.71345 0 3913.06 0 1.34774 0 1.49452e-09 3.70116e-10 2.32404e-09 2017-10-03T11:58:52.000 2017-10-04T11:58:52.000 434116732.0 434203132.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 110 -1 404.846 0.5 0 52 3 0 381.795 1.35163e-15 1 52 3 486.25
58030.49921296296 58031.49921296296 7.63221 820.4e-08 190.928e-08 1249.81e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3793810.0 2.9604 31.1244 7.24344 1.71345 0 3913.06 0 1.34774 0 1.45849e-09 3.39428e-10 2.22189e-09 2017-10-04T11:58:52.000 2017-10-05T11:58:52.000 434203132.0 434289532.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 97 -1 365.286 0.5 0 52 3 0 336.161 1.10047e-16 1 52 3 426.229

Advanced configuration

Selection

Option

Type

Default

Required

Description

lonpole

int

180

no

lpointing

float

TBD

no

l in Galactic coordinates of the center of the pointing during the ‘pointing period’

bpointing

float

TBD

no

b in Galactic coordinates of the center of the pointing during the ‘pointing period’

maplistgen

string

TBD

no

filename of a file for expmapgen with mapspec.fovradmin >> mapspec.fovradmax >> mapspec.emin >> mapspec.emax >> mapspec.index

Maps

Option

Type

Default

Required

Description

offaxisangle

float

30

No

Off axis pointing for mle analysis. Values are between 0 and 60 deg. Agilepy into .maplist

Model

Option

Type

Default

Required

Description

galmode2

int

0

No

Fix the gal parameters using the internal analysis of the MLE.
0) gal0 and gal1 are kept free
1) set gal0 for L0 and gal1 for L1
2) set gal0 for L0 and L1
3) set gal1 for L0 and L1
4) set gal1 - gal1err for L0 and L1
5) set gal1 + gal1err for L0 and L1

galmode2fit

int

0

No

Fix the gal parameters for different energy bins performing a linear fit of gal values evaluated using the internal analysis of the MLE.
0) do not fit
1) pol0 fit
2) powerlaw fit

isomode2

int

0

No

Fix the iso parameters using the internal analysis of the MLE.
0) none
1) set iso0 for L0 and gal1 for L1
2) set iso0 for L0 and L1
3) set iso1 for L0 and L1
4) set iso1 - iso1err for L0 and L1
5) set iso1 + iso1err for L0 and L1

isomode2fit

int

0

No

Fix the iso parameters for different energy bins performing a linear fit of iso values evaluated using the internal analysis of the MLE.
0) do not fit
1) pol0 fit
2) powerlaw fit

mle

Advanced options for optimizer

Option

Type

Default

Required

Description

minimizertype

string

Minuit

No

Use Minuit if position is free. For other values see below.

minimizeralg

string

Migrad

No

For other values see below.

minimizerdefstrategy

int

2

No

Default 2 for Minuit. For other values see below.

mindefaulttolerance

float

0.01

No

See below.

integratortype

int

1

No

1 gauss
2 gaussht
3 gausslefevre
4 gausslefevreht

contourpoints

int

40

No

Number of points to determine the contour (0-400).

minimizertype = Minuit (library libMinuit). Old version of Minuit, based on the TMinuit class. The list of possible algorithms (minimizeralg) are:
1) Migrad (default one)
2) Simplex
3) Minimize (it is a combination of Migrad and Simplex)
4) MigradImproved
5) Scan
6) Seek
minimizertype = Minuit2 (library libMinuit2). New C++ version of Minuit. The list of the possible algorithms (minimizeralg) :
1) Migrad (default)
2) Simplex
3) Minimize
4) Scan

minimizertype = Fumili . This is the same algorithm of TFumili, but implemented in the Minuit2 library.

minimizertype = GSLMultiMin (library libMathMore). Minimizer based on the Multidimensional Minimization routines of the Gnu Scientific Library (GSL). The list of available algorithms (minimizeralg) is | 1) BFGS2 (default) : second version of the vector Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm; | 2) BFGS : old version of the vector Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm; | 3) ConjugateFR : Fletcher-Reeves conjugate gradient algorithm; | 4) ConjugatePR : Polak-Ribiere conjugate gradient algorithm; | 5) SteepestDescent: steepest descent algorithm;

#*** * GSLMultiFit (library libMathMore). Minimizer based on the Non-Linear Least-Square routines of GSL. This minimizer can be used only for least-square fits.
#*** * GSLSimAn (library libMathMore). Minimizer based on simulated annealing.
#*** * Genetic (library libGenetic). Genetic minimizer based on an algorithm implemented in the TMVA package.

Each minimizer can be configured using the ROOT::Math::MinimizerOptions class. The list of possible option that can be set are:

minimizertype:
Minimizer type (MinimizerOptions::SetMinimizerType(const char * )) .
* Print Level (MinimizerOptions::SetPrintLevel(int )) to set the verbose printing level (default is 0).
mindefaulttolerance:
* Tolerance (MinimizerOptions::SetTolerance(double )) tolerance used to control the iterations.
* Precision (MinimizerOptions::SetTolerance(double )). Precision value in the evaluation of the minimization function. Default is numerical double precision.
  • Maximum number of function calls (MinimizerOptions::SetMaxFunctionCalls(int )).

  • Maximum number of iterations (MinimizerOptions::SetMaxIterations(int )). Note that this is not used by Minuit. FCN Upper value for Error Definition (MinimizerOptions::SetMaxIterations(int )). Value in the minimization function used to compute the parameter errors. The default is to get the uncertainties at the 68% CL is a value of 1 for a chi-squared function minimization and 0.5 for a log-likelihood function.

minimizerdefstrategy:
* Strategy (MinimizerOptions::SetStrategy(int )), minimization strategy used. For each minimization strategy Minuit uses different configuration parameters (e.g. different requirements in computing derivatives, computing full Hessian (strategy = 2) or an approximate version. The default is a value of 1. In this case the full Hessian matrix is computed only after the minimization.

Advanced options for internal corrections

Option

Type

Default

Required

Description

edpcorrection

float

0.75

No

Perform a flux correction based on EDP evaluation for highest energy channels. Default 0.75, otherwise any value between 0 and 1. EDP correction is enabled only for E>1000 MeV and if fluxcorrection=1, and only for point sources. flux = flux * edpcorrection

fluxcorrection

int

0

No

Perform a flux correction of the flux using the source spectral model and considering that the exposure is calculated with a Power Law with spectral index of 2.1.
0) no correction
1) Flux calculation correction for spectral shape in output
2) correction in input and output

Analysis API

AGBaseAnalysis

class core.AGBaseAnalysis.AGBaseAnalysis(configurationFilePath)[source]
__init__(configurationFilePath)[source]
getAnalysisDir()[source]

It returns the path of the output directory

Returns

the path of the output directory

Return type

path (str)

deleteAnalysisDir()[source]

It deletes the output directory where all the products of the analysis are written.

Args:

Returns

True if the directory is succesfully deleted, False otherwise.

setOptions(**kwargs)[source]

It updates configuration options specifying one or more key=value pairs at once.

Parameters

kwargs – key-values pairs, separated by a comma.

Returns

None

Raises
  • ConfigFileOptionTypeError – if the type of the option value is not wrong.

  • ConfigurationsNotValidError – if the values are not coherent with the configuration.

  • CannotSetHiddenOptionError – if the option is hidden.

  • OptionNotFoundInConfigFileError – if the option is not found.

Note

The config attribute is initialized by reading the corresponding yaml configuration file, loading its contents in memory. Updating the values held by this object will not affect the original values written on disk.

Example

>>> aganalysis.setOptions(mapsize=60, binsize=0.5)
True
getOption(optionName)[source]

It reads an option value from the configuration.

Parameters

optionName (str) – the name of the option.

Returns

The option value

Raises

OptionNotFoundInConfigFileError – if the optionName is not found in the configuration.

printOptions(section=None)[source]

It prints the configuration options in the console.

Parameters

section (str) – you can specify a configuration file section to be printed out.

Returns

None

AGAnalysis

class api.AGAnalysis.AGAnalysis(configurationFilePath, sourcesFilePath=None)[source]

This class contains the high-level API to run scientific analysis, data visualization and some utility methods.

This constructor of this class requires a yaml configuration file.

__init__(configurationFilePath, sourcesFilePath=None)[source]

AGAnalysis constructor.

Parameters
  • configurationFilePath (str) – a relative or absolute path to the yaml configuration file.

  • sourcesFilePath (str, optional) – a relative or absolute path to a file containing the description of the sources. Defaults to None. Three different types of formats are supported: AGILE format (.txt), XML format (.xml) and AGILE catalog files (.multi).

Raises
  • AGILENotFoundError – if the AGILE environment variable is not set.

  • PFILESNotFoundError – if the PFILES environment variable is not set.

Example

>>> from agilepy.api import AGAnalysis
>>> aganalysis = AGAnalysis('agconfig.yaml', sourcesFilePath='sources.xml')
destroy()[source]

It clears the list of sources and the current maplist file.

static getConfiguration(confFilePath, userName, sourceName, tmin, tmax, timetype, glon, glat, outputDir, verboselvl, evtfile='/AGILE_PROC3/FM3.119_ASDC2/INDEX/EVT.index', logfile='/AGILE_PROC3/DATA_ASDC2/INDEX/LOG.log.index', userestapi=True, datapath=None)[source]

Utility method to create a configuration file.

Parameters
  • confFilePath (str) – the path and filename of the configuration file that is going to be created.

  • userName (str) – the username of who is running the software.

  • sourceName (str) – the name of the source.

  • tmin (float) – the start time of the analysis.

  • tmax (float) – the stop time of the analysis.

  • timetype (str) – timetype format possible values: “TT” or “MJD”

  • glon (float) – the galactic longitude (L) of the analysis.

  • glat (float) – the galactic latitude (B) of the analysis.

  • outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’

  • verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug

  • evtfile (str, optional [default=/AGILE_PROC3/FM3.119_ASDC2/INDEX/EVT.index]) – the index file to be used for event data if userestapi is set to False. The time range starts from 107092735 TT, 54244.49924768 MJD, 2007-05-24 11:58:55 UTC

  • logfile (str, optional [default=/AGILE_PROC3/DATA_ASDC2/INDEX/LOG.log.index]) – the index file to be used for log data if userestapi is set to False. The time range starts from 107092735 TT, 54244.49924768 MJD, 2007-05-24 11:58:55 UTC

  • userestapi (bool, optional [default=True]) – If True, the SSDC REST API will be used to download missing data.

  • datapath (str, optional [default=None]) – Datapath to download AGILE data if userestapi is set to True. Index files will be generated into this path.

Returns

None

loadSourcesFromCatalog(catalogName, rangeDist=(0, inf), show=False)[source]

It loads the sources from a catalog.

You can also specify a rangeDist argument to filter out the sources which distance from (glon, glat) is not in the rangeDist interval.

If the catalog is 2AGL and if the energy range (emin, emax) specified by the user in the configuration file is different from the catalog’s energy range, the flux of every source will be scaled.

Parameters
  • catalogName (str) – the catalog name (2AGL, 4FGL).

  • rangeDist (tuple, optional) – a interval (min, max) of distances (degree). It defaults to (0, +inf).

Raises
  • FileNotFoundError – if the catalog is not supported.

  • SourceModelFormatNotSupported – if the input file format is not supported.

  • SourcesFileLoadingError – if any error occurs during the parsing of the sources file.

Returns

The List of sources that have been succesfully loaded into the SourcesLibrary.

loadSourcesFromFile(sourcesFilepath, rangeDist=(0, inf), show=False)[source]

It loads the sources, reading them from a file. Three different types of format are supported: AGILE format (.txt), XML format (.xml) and AGILE catalog files (.multi).

You can also specify a rangeDist argument to filter out the sources which distance from (glon, glat) is not in the rangeDist interval.

Parameters

sourcesFilePath (str) – a relative or absolute path to a file containing the description of the sources. rangeDist (tuple): a interval (min, max) of distances (degree)

Raises
  • SourceModelFormatNotSupported – if the input file format is not supported.

  • SourcesFileLoadingError – if any error occurs during the parsing of the sources file.

Returns

The List of sources that have been succesfully loaded into the SourcesLibrary.

selectSources(selection, show=False)[source]

It returns the sources matching the selection criteria from the sourcesLibrary.

The sources can be selected with the selection argument, supporting either lambda functions and boolean expression strings.

The selection criteria can be expressed using the following Source class’s parameters:

  • name: the unique code identifying the source.

  • dist: the distance of the source from the center of the maps.

  • flux: the flux value.

  • sqrtts: the radix square of the ts.

Warning

The sqrtts parameter is available only after the maximum likelihood estimation analysis is performed.

Parameters
  • selection (lambda or str) – a lambda function or a boolean expression string specifying the selection criteria.

  • quiet (boolean) (optional default=False) – if quiet is True, the method will not console log the selected sources.

Returns

List of sources.

freeSources(selection, parameterName, free, show=False)[source]

It can set to True or False a parameter’s free attribute of one or more source.

Example of source model:

<source name="2AGLJ2021+4029" type="PointSource">
    <spectrum type="PLExpCutoff">
      <parameter name="flux" free="1"  value="119.3e-08"/>
      <parameter name="index" free="0" scale="-1.0" value="1.75" min="0.5" max="5"/>
      <parameter name="cutoffEnergy" free="0" value="3307.63" min="20" max="10000"/>
    </spectrum>
    <spatialModel type="PointSource" location_limit="0">
      <parameter name="pos" value="(78.2375, 2.12298)" free="0" />
    </spatialModel>
</source>

The supported parameterName are:

  • flux

  • index

  • index1

  • index2

  • cutoffEnergy

  • pivotEnergy

  • curvature

  • index2

  • pos

Parameters
  • selection (lambda or str) – a lambda function or a boolean expression string specifying the selection criteria.

  • parameterName (str) – the free attribute of this parameter will be updated.

  • free (bool) – the boolean value used.

Returns

The list of the sources matching the selection criteria.

Note

The sourcesLibrary attribute is initialized by reading the corresponding xml descriptor file, loading its contents in memory. Updating the values held by this object will not affect the original values written on disk.

Example

The following calls are equivalent and they set to True the free attribute of the “Flux” parameter for all the sources named “2AGLJ2021+4029” which distance from the map center is greater than zero and which flux value is greater than zero.

>>> aganalysis.freeSources(lambda name, dist, flux : Name == "2AGLJ2021+4029" AND dist > 0 AND flux > 0, "flux", True)
[..]
>>> aganalysis.freeSources('name == "2AGLJ2021+4029" AND dist > 0 AND flux > 0', "flux", True)
[..]
fixSource(source)[source]

Set to False all freeable params of a source.

Parameters

sourceName (Source) – the source object.

Returns

True if at least one parameter of the source changes, False otherwise.

addSource(sourceName, sourceDict)[source]

It adds a new source in the Sources Library. You can add a source, passing a dictionary containing the source’s data. Some keys are required, other are optional. Here’s an example:

newSourceDict = {
    "glon" : 6.16978,
    "glat": -0.0676943,
    "spectrumType" : "LogParabola",
    "flux": 35.79e-08,
    "curvature": 0.682363
}
newSource = aganalysis.addSource("2AGLJ1801-2334", newSourceDict)
Parameters
  • sourceName (str) – the name of the source

  • sourceDict (dict) – a dictionary containing the source’s data.

Raises

SourceParamNotFoundError – if sourceName is None or empty, OR the sourceDict input dictionary does not contain at least one of the following keys: [“glon”, “glat”,”spectrumType”].

Returns

The source object if it is been loaded. None, otherwise.

deleteSources(selection, show=False)[source]

It deletes the sources matching the selection criteria from the sourcesLibrary.

Parameters

selection (lambda or str) – a lambda function or a boolean expression string specifying the selection criteria.

Returns

The list containing the deleted sources.

getSources()[source]

It returns all the sources. :returns: List of sources.

updateSourcePosition(sourceName, glon, glat)[source]

It updates a source (l,b) position parameters. The ‘dist’ value will also be updated.

Parameters
  • sourceName (str) – the name of the source

  • glon (float) – galactic longitude.

  • glat (float) – galactic latitude.

Raises
  • SourceNotFound – if the source has not been loaded into the SourcesLibrary.

  • ValueError – if glon or glat values are out of range.

Returns

True if the position changes, False otherwise.

writeSourcesOnFile(outfileNamePrefix, fileFormat, sources=None)[source]

It writes on file the list of sources loaded into the SourceLibrary. The supported formats (‘txt’ AND ‘xml’) are described here: Sources file.

Parameters
  • outfileNamePrefix (str) – the name of the output file (without the extension).

  • fileFormat (str) – Possible values: [‘txt’, ‘xml’, ‘reg’].

  • sources (List<Source>) – a list of Source objects. If is is None, every loaded source will be written on file.

Raises

SourceModelFormatNotSupported – if the file format is not supported.

Returns

Path to the file

setOptionTimeMJD(tmin, tmax)[source]

It performs a time conversion MJD -> TT and set new values using setOptions functions

Parameters
  • tmin (float) – tmin in MJD

  • tmax (float) – tmax in MJD

setOptionEnergybin(value)[source]

An useful utility method that maps a value in a specific energybin and it calls the setOptions function.

Parameters

Value (int) –

A value between 0 and 5

  • 0: [[100, 10000]]

  • 1: [[100, 50000]]

  • 2: [[100, 300], [300, 1000], [1000, 3000], [3000, 10000]]

  • 3: [[100, 300], [300, 1000], [1000, 3000], [3000, 10000], [10000, 50000]]

  • 4: [[50, 100], [100, 300], [300, 1000], [1000, 3000], [3000, 10000]]

  • 5: [[50, 100], [100, 300], [300, 1000], [1000, 3000], [3000, 10000], [10000, 50000]]

Raises

ValueOutOfRange – if the value is not between 0 and 5.

generateMaps(config=None, maplistObj=None, dqdmOff=False)[source]

It generates (one or more) counts, exposure, gas and int maps and a maplist file.

The method’s behaviour varies according to several configuration options (see docs Configuration file).

Note

It resets the configuration options to their original values before exiting.

Returns

The absolute path to the generated maplist file.

Raises

ScienceToolInputArgMissing – if not all the required configuration options have been set.

Example

>>> aganalysis.generateMaps()
/home/rt/agilepy/output/testcase.maplist4
calcBkg(sourceName, galcoeff=None, pastTimeWindow=14.0, excludeTmaxTmin=False)[source]

It estimates the isotropic and galactic background coefficients.

Note

The current value of the isotropic and galactic bkg coeffs will be set to -1 in order to be free to vary during the fit. It automatically updates the configuration.

Parameters
  • sourceName (str) – the name of the source under analysis.

  • galcoeff (List, optional) – the galactic background coefficients (one for each map).

  • pastTimeWindow (float, optional) – the number of days previous tmin. It defaults to 14. If it’s 0, the background coefficients will be estimated within the [tmin, tmax] interval.

  • excludeTmaxTmin (bool, optional) – if this value is True (defalut=False) and pasttimewindow > 0 the calcbkg method computes coeffs between [tmin-pastTimeWindow, tmin] instead of [tmin-pastTimeWindow, tmax]

Returns

the estimates of the background coefficients and the path to the updated maplist file.

Return type

galactic coefficients, isotropic coefficients, maplist file (List, List, str)

mle(maplistFilePath=None, config=None, updateSourceLibrary=True, position='ellipse')[source]

It performs a maximum likelihood estimation analysis on every source withing the sourceLibrary, producing one output file per source.

The method’s behaviour varies according to several configuration options (see docs Configuration file).

The outputfiles have the .source extension.

The method’s behaviour varies according to several configuration options (see docs here).

Note

It resets the configuration options to their original values before exiting.

Parameters

maplistFilePath (str) – if not provided, the mle() analysis will use the last generated mapfile produced by generateMaps().

Returns

A list of absolute paths to the output .source files.

Raises

MaplistIsNone – is the input argument is None.

Example

>>> maplistFilePath = aganalysis.generateMaps()
>>> aganalysis.mle(maplistFilePath)
[/home/rt/agilepy/output/testcase0001_2AGLJ2021+4029.source /home/rt/agilepy/output/testcase0001_2AGLJ2021+3654.source]
lightCurveMLE(sourceName, tmin=None, tmax=None, timetype=None, binsize=86400, position='ellipse')[source]

It generates a cvs file containing the data for a light curve plot.

Parameters
  • sourceName (str) – the name of the source under analysis.

  • tmin (float, optional) – starting point of the light curve. It defaults to None. If None the ‘tmin’ value of the configuration file will be used.

  • tmax (float, optional) – ending point of the light curve. It defaults to None. If None the ‘tmax’ value of the configuration file will be used.

  • timetype (str, optional) – the time format (‘MJD’ or ‘TT’). It defaults to None. If None the ‘timetype’ value of the configuration file will be used.

  • binsize (int, optional) – temporal bin size. It defaults to 86400.

  • position (str, optional) – the position of the source: {“ellipse”, “peak”, “initial”}

Returns

The absolute path to the light curve data output file.

aperturePhotometry()[source]

It generates a cvs file containing the data for a light curve plot. The method’s behaviour varies according to several configuration options (see docs Configuration file).

Returns

The absolute path to the light curve data output file.

Raises

ScienceToolInputArgMissing – if not all the required configuration options have been set.

displayCtsSkyMaps(maplistFile=None, singleMode=True, smooth=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]

It displays the last generated cts skymaps.

Parameters
  • maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.

  • singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.

  • smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.

  • saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.

  • fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .

  • title (str, optional) – the title of the image. It defaults to None.

  • cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html

  • regFiles (list, optional) – A list containing region file(s) absolute path.

  • regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles.

  • catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.

  • catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.

  • normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.

Returns

It returns the paths to the image files written on disk.

displayExpSkyMaps(maplistFile=None, singleMode=True, smooth=False, sigma=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]

It displays the last generated exp skymaps.

Parameters
  • maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.

  • singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.

  • smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.

  • saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.

  • fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .

  • title (str, optional) – the title of the image. It defaults to None.

  • cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html

  • regFiles (list, optional) – A list containing region file(s) absolute path

  • regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles

  • catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.

  • catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.

  • normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.

Returns

It returns the paths to the image files written on disk.

displayGasSkyMaps(maplistFile=None, singleMode=True, smooth=False, sigma=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]

It displays the last generated gas skymaps.

Parameters
  • maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.

  • singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.

  • smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.

  • saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.

  • fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .

  • title (str, optional) – the title of the image. It defaults to None.

  • cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html

  • regFiles (list, optional) – A list containing region file(s) absolute path

  • regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles

  • catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.

  • catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.

  • normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.

Returns

It returns the paths to the image files written on disk.

displayIntSkyMaps(maplistFile=None, singleMode=True, smooth=False, sigma=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]

It displays the last generated int skymaps.

Parameters
  • maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.

  • singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.

  • smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.

  • saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.

  • fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .

  • title (str, optional) – the title of the image. It defaults to None.

  • cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html

  • regFiles (list, optional) – A list containing region file(s) absolute path

  • regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles

  • catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.

  • catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.

  • normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.

Returns

It returns the paths to the image files written on disk.

displayLightCurve(analysisName, filename=None, lineValue=None, lineError=None, saveImage=False, fermiLC=None)[source]

It displays the light curve plot. You can call this method after lightCurveMLE() or aperturePhotometry(). If you pass a filename containing the light curve data, this file will be used instead of using the data generated by the mentioned methods.

Parameters
  • analysisName (str, required) – the analysis used to generate the light curve data: choose between ‘mle’ or ‘ap’.

  • filename (str, optional) – the path of the Lightcurve text data file. It defaults to None. If None the last generated file will be used.

  • lineValue (int, optional) – mean flux value. It defaults to None.

  • lineError (int, optional) – mean flux error lines. It defaults to None.

  • saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.

  • fermiLC (str, optional) – the path of FERMI lightcurve data

Returns

It returns the lightcurve plot

displayGenericColumns(filename, columns, um=None, saveImage=False)[source]

An utility method for viewing a generic column from the lightcurvedata

Parameters
  • filename (str) – the path of the Lightcurve text data file. It defaults to None. If None the last generated file will be used.

  • columns (array of str) – the name of the columns to display(first display is always LC), possible values: (time_start_mjd | time_end_mjd | sqrt(ts) | flux | flux_err | flux_ul | gal | gal_error | iso | iso_error | l_peak | b_peak | dist_peak | l | b | r | ell_dist | a | b | phi | exposure | ExpRatio | counts | counts_err | Index | Index_Err | Par2 | Par2_Err | Par3 | Par3_Err | Erglog | Erglog_Err | Erglog_UL | time_start_utc | time_end_utc | time_start_tt | time_end_tt | Fix | index | ULConfidenceLevel | SrcLocConfLevel | start_l | start_b | start_flux | typefun | par2 | par3 | galmode2 | galmode2fit | isomode2 | isomode2fit | edpcor | fluxcor | integratortype | expratioEval | expratio_minthr | expratio_maxthr | expratio_size | Emin | emax | fovmin | fovmax | albedo | binsize | expstep | phasecode | fit_cts | fit_fitstatus0 | fit_fcn0 | fit_edm0 | fit_nvpar0 | fit_nparx0 | fit_iter0 | fit_fitstatus1 | fit_fcn1 | fit_edm1 | fit_nvpar1 | fit_nparx1 | fit_iter1 | fit_Likelihood1)

  • um (array of str) – unit of measurement, same length of columns

  • saveImage (bool) – if set to true, saves the image into the output directory. It defaults to False.

Returns

If saveImage is true tha path of the image otherwise the plot

convertCatalogToXml(catalogFilepath)[source]

It takes a catalog file in AGILE (.txt) format as input and converts it in the xml format.

Parameters

catalogFilepath (str) – a relative or absolute path to the catalog file in AGILE (.txt) format.

Raises

SourceModelFormatNotSupported – if the input file format is not supported.

Returns

A string rapresenting the path to the output file.

parseMaplistFile(maplistFilePath=None)[source]

It parses the maplistfile in order to return sky map files paths.

Parameters
  • maplistFilePath (str) – if you don’t specify the maplistfile path, the

  • one (last generated) –

Returns

A matrix where each row contains a cts, ext and gas map.

Raises

MaplistIsNone – if no maplist file is passed and no maplist file have been generated yet.

AGAnalysisWavelet

class api.AGAnalysisWavelet.AGAnalysisWavelet(configurationFilePath, sourcesFilePath=None)[source]

This class contains the high-level API methods you can use to run wavelet analysis.

This class requires you to setup a yaml configuration file to specify the software’s behaviour.

WARNING: This class is deprecated due to the new Python wavelet tool in development

__init__(configurationFilePath, sourcesFilePath=None)[source]

AGAnalysis constructor.

Parameters
  • configurationFilePath (str) – a relative or absolute path to the yaml configuration file.

  • sourcesFilePath (str, optional) – a relative or absolute path to a file containing the description of the sources. Defaults to None. Three different types of format are supported: AGILE format (.txt), XML format (.xml) and AGILE catalog files (.multi).

static getConfiguration(confFilePath, userName, outputDir, verboselvl, ctsmap, scaletype, scalemin, scalemax, scalenum, methistsize, cclsizemin, cclsizemax, cclradmin, cclradmax, cclscalemin, cclscalemax)[source]

Utility method to create a configuration file.

Parameters
  • confFilePath (str) – the path and filename of the configuration file that is going to be created.

  • userName (str) – the username of who is running the software.

  • outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’

  • verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug

  • ctsmap (str) – the path of cts map generated by AGAnalys class

  • scaletype (str) –

  • scalemin (float) –

  • scalemax (float) –

  • scalenum (int) –

  • methistsize (float) –

  • cclsizemin (float) –

  • cclsizemax (float) –

  • cclradmin (float) –

  • cclradmax (float) –

  • cclscalemin (float) –

  • cclscalemax (float) –

Returns

None

waveletAnalysis()[source]

It runs the three science tools required to perform wavelet analysis, and it creates three directories respectively for any tool

waveletDisplay(fitsfile, saveImage=False, multiImage=False)[source]

It displays the wevelets generated by waveletAnalysis.

Parameters
  • fitsfile (str) – fits filepath

  • saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.

  • multiImage (bool) – if set to true plots an image for each wavelet

Returns

It returns the paths to the image files written on disk.

Engineering API

AGEngAgileOffaxisVisibility

class api.AGEngAgileOffaxisVisibility.AGEngAgileOffaxisVisibility(configurationFilePath)[source]

This class contains the high-level API methods you can use to run engineering analysis.

This class requires you to setup a yaml configuration file to specify the software’s behaviour.

Class attributes:

config

it is used to read/update configuration values.

Type

AgilepyConfig

logger

it is used to log messages with different importance levels.

Type

AgilepyLogger

__init__(configurationFilePath)[source]

AGEngAgileOffaxisVisibility constructor.

Parameters

configurationFilePath (str) – the relative or absolute path to the yaml configuration file.

Example

>>> from agilepy.api import AGEngAgileOffaxisVisibility
>>> ageng = AGEngAgileOffaxisVisibility('agconfig.yaml')
static getConfiguration(confFilePath, userName, outputDir, verboselvl)[source]

Utility method to create a configuration file.

Parameters
  • confFilePath (str) – the path and filename of the configuration file that is going to be created.

  • userName (str) – the username of who is running the software.

  • outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’

  • verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug

Returns

None

visibilityPlot(logfilesIndex, tmin, tmax, src_x, src_y, ref, zmax=60, step=1, writeFiles=True, computeHistogram=True, saveImage=True, fileFormat='png', title='Visibility Plot')[source]

It computes the angular separations between the center of the AGILE GRID field of view and the coordinates for a given position in the sky, given by src_ra and src_dec.

Parameters
  • logfilesIndex (str) – the index file for the logs files.

  • tmin (float) – inferior observation time limit to analize.

  • tmax (float) – superior observation time limit to analize.

  • src_x (float) – source position x (unit: degrees)

  • src_y (float) – source position y (unit: degrees)

  • zmax (float) – maximum zenith distance of the source to the center of the detector (unit: degrees)

  • step (integer) – time interval in seconds between 2 consecutive points in the resulting plot. Minimum accepted value: 0.1 s.

  • writeFiles (bool) – if True, two text files with the separions data will be written on file.

  • saveImage (bool) – If True, the image will be saved on disk

  • fileFormat (str) – The output format of the image

  • title (str) – The plot title

Returns

the angular separations ti_tt (List): tf_tt (List): ti_mjd (List): tf_mjd (List): skyCordsFK5.ra.deg skyCordsFK5.dec.deg

Return type

separations (List)

AGEngAgileFermiOffAxisVisibilityComparison

class api.AGEngAgileFermiOffAxisVisibilityComparison.AGEngAgileFermiOffAxisVisibilityComparison(configurationFilePath)[source]

This class contains the high-level API methods to run offaxis and offaxis_ap_comparison tools. It’s a AGEng subclass

__init__(configurationFilePath)[source]

AGEngAgileFermiOffAxisVisibilityComparison constructor.

Parameters

configurationFilePath (str) – the relative or absolute path to the yaml configuration file.

Example

>>> from agilepy.api.AGEngAgileFermiOffAxisVisibilityComparison import AGEngAgileFermiOffAxisVisibilityComparison
>>> ageng = AGEngAgileFermiOffAxisVisibilityComparison('agconfig.yaml')
visibilityPlot(time_windows, ra, dec, fermi_datapath, agile_datapath, run, zmax, mode=<built-in function all>, step=1)[source]

It runs offaxis tools and creates a directory containing the result files

Parameters
  • time_windws (2d float Array) – It contains the tstart-tstop intervals to process the data, the structure has developed as a 2d array(eg [[t1,t2],[t3, t4], …, [tn-1, tn]])

  • ra (float) – ra value

  • dec (float) – dec value

  • fermi_datapath (str) – fermi log filepath

  • agile_datapath (str) – agile log filepath

  • run (integer) – run number

  • zmax (float) – maximum offaxis degrees

  • mode (str) – options “agile” | “fermi” | “all”: Select all to plot both data, otherwise it will plot only agile/fermi data

  • step (float) – step value for plotting

Returns

A new directory containing the results

Return type

dir (str)

apOffaxisComparation(agile_pathAP, fermi_pathAP, tstart_mjd, tstop_mjd, path_offaxis, vertical_boxes_mjd=[], zmax=60, timetype='MJD', data_column_name='cts', time_range=None, trigger_time_tt=None, add_rm=False, rm_files=None, rm_labels=None)[source]

It compares and shows aperture photometry data with offaxis results.

WARNING: This class should be instanced anew any time a new plot is generated, to ensure its correctness.

Parameters
  • agile_pathAP (str) – agile ap filepath

  • fermi_pathAP (str) – fermi ap filepath

  • tstart_mjd (float) – time start in MJD

  • tstop_mjd (float) – time stop in MJD

  • path_offaxis (str) – directory path to offaxis results

  • vertical_boxes_mjd (list) – time in MJD

  • zmax (float) – maximum offaxis degrees

  • timetype (str) – time type to plot [MJD, TT]

  • data_column_name (str) – name of column to plot

  • time_range (list) – time xrange in MJD or TT depending on “timetype”

  • trigger_time_tt (float or None) – trigger time in TT

  • add_rm (bool) – add fourth plot with AGILE ratemeters

  • rm_files (list) – list of absolute paths for all RM to plot

  • rm_lables (list) – list of labels to pair to RM files

Returns

void

static getConfiguration(confFilePath, userName, outputDir, verboselvl)[source]

Utility method to create a configuration file.

Parameters
  • confFilePath (str) – the path and filename of the configuration file that is going to be created.

  • userName (str) – the username of who is running the software.

  • outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’

  • verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug

Returns

None

AstroUtils API

class utils.AstroUtils.AstroUtils[source]
static distance(l1, b1, l2, b2)[source]

Computes the angular distance between two galatic coordinates.

Parameters
  • l1 (float) – longitude of first coordinate

  • b1 (float) – latitude of first coordinate

  • l2 (float) – longitude of second coordinate

  • b2 (float) – latitude of second coordinate

Returns

the angular distance between (l1, b1) and (l2, b2)

static AP_filter(filename, threshold, tstart, tstop, outpath)[source]

This function filters an aperture photometry file using a threshold value for exposure, it discards the rows lower than threshold and returns a new file merging the continous rows.

Parameters
  • filename (str) – path of the aperture photometry file

  • threshold (float) – exposure threshold

  • tstart (float) – time start in UTC

  • tstop (float) – time stop in UTC

Returns

A filtered file result.txt

Source API

Source

class core.source.Source.Source(name)[source]
getName()[source]

It gets the name for the source

Parameters

None

Returns

Source name

Return type

name(str)

getFreeableParams()[source]

It gets parameters to free

Parameters

None

Returns

parameters to free

Return type

List(str)

Point Source

class core.source.Source.PointSource(name)[source]
get(parameterName)[source]

It returns a source’s parameter.

Parameters

paramName (str) – the name of the source’s parameter.

Raises

SourceParameterNotFound – if the source’s parameter is not found.

Returns

A dictionary containing the source’s attributes.

Example

>>> s.get("index")
>>> s.get("pos")
>>> s.get("multiFlux")
getVal(parameterName)[source]

It returns a source’s parameter value.

Parameters

paramName (str) – the name of the source’s parameter.

Raises

SourceParameterNotFound – if the source’s parameter is not found.

Returns

The value of the source’s parameter.

Example

>>> s.getVal("index")
>>> s.getVal("pos")
>>> s.getVal("multiFlux")
set(parameterName, attributeValueDict)[source]

It sets a source’s parameter.

Parameters

parameterName (str) – the name of the source’s parameter.

Returns

None

Example

>>> s.set("index",{"value":1, "min":10})
setVal(parameterName, parameterValue)[source]

It sets a source’s parameter.

Parameters

parameterName (str) – the name of the source’s parameter.

Returns

None

Example

>>> s.set("index",{"value":1, "min":10})
getFreeParams()[source]

It returns the source’s attributes that are free to vary.

Returns

The list of attributes that are free to vary.

Example

>>> s.getFreeParams()
>>> ["flux", "index"]

Science Tools

Name

Short description

Documents

AG_add_diff

AG_checkMapValue

AG_ctsmapgen

Generates a counts map from the Agile satellite events list, given a list of time intervals, an energy range and a field of view interval.

AG_ctsmapgenT

Performs the same selection as AG_ctsmapgen, writing its output to text files for futher automatic usage.

AG_diff_conv

AG_expmapgen

Generates an exposure map from the Agile satellite log files related to a given a list of time intervals.

ExpmapgenDoc.pdf

AG_expmapgenT

Performs the same selection as AG_expmapgen, writing its output to text files for futher automatic usage.

AG_gasmapgen

Generates the Galactic diffuse emission maps

AG_intmapgen

Generates the intensity maps

AG_iterativeGenSrcList

AG_multi

Performs a Maximum Likelyhood Analysis to find the best position, flux and spectral index of a list of sources to explain the given set of count maps.

AG_multiUserManual.html (outdated) CalcoloTS.html PSF_generation.pdf

AG_multiext

It does the same job as AG_multi also considering a set of extended sources.

AG_multiExtUM.html

AG_multisim

Given a list of sources and a set of exposure maps it generates the corresponding counts map by means of a Monte Carlo simulation. It optionally performs on these maps the same analysis of AG_multi. The entire process can be iterated. The generated maps can be saved to disk or just analyzed on the fly.

AG_multisimUM.html

AG_multiterative

This application iteratively performs the same analysis of AG_multi, tentatively adding more sources, one at a time, from a second sources list. It produces the best sources list explaining the data.

AG_pasteMap

This utility program joins two sky maps together adapting their resolution.

AG_spotfinder

AG_thetamapgen

Input files of the MLE

This page provides detailed description of the input files of the science tool AG_multi.

Map list input files

‘.maplist4’ file

The map list is a text file listing containing at least one line of text. Each line of text describes one set of maps and it is possible to include empty lines or comment lines. The comment lines begin with an exclamation mark.

Each line contains a set of maps:

<countsMap> <exposureMap> <gasMap> <offaxisangle> <galcoeff> <isocoeff>

where:

AGILE format (text file)

The source list is a text file listing at least one source. Each line of text describes one source, and it is possible to include empty lines or comment lines. The comment lines begin with an exclamation mark.

Each source is described by a line containing space-separated values, in the following order:

'flux' 'l' 'b' 'index' 'fixflag' 'minSqrt(TS)' 'name' 'locationlimit' 'funtype' 'par2' 'par3' 'index limit min' 'index limit max' 'par2 limit min' 'par2 limit max' 'par3 limit min' 'par3 limit max'

The ‘flux’ parameter is expressed in cm^-2 s^-1, and galactic longitude ‘l ‘and latitude ‘b’ are expressed in degrees.

minSqrt(TS) is the minimum acceptable value for the square root of TS: if the optimized significance of a source lies below this value, the source is considered undetected and will be ignored (set to flux = 0) when considering the other sources.

After the source’s name (which should not contain a space), an optional value for the location limitation (’locationlimit’) in degrees may be provided. If this value is present and not zero, the longitude and latitude of the source will not be allowed to vary by more than this value from its initial position.

According to the fixflag, some or all values will be optimized by being allowed to vary.

The ‘funtype’ specify the spectral model. PL indicates power-law fit to the energy spectrum; PC indicates power-law with exponential cut-off fit to the energy spectrum; PS indicates power-law with super-exponential cut-off fit to the energy spectrum; LP indicates log-parabola fit to the energy spectrum.

The ‘index’ of each source represents the initial estimates of the values for that source (a positive number) and could represent the spectral index of the source (see the following table). The other spectral parameters depend on the spectral shape of the source. ‘index limit min’ and ‘index limit max’ specifies the minimum and maximum range where the ‘index’ is searched.

The ‘par2’ and ‘par3’ parameters represent additional spectral parameters in the following table. ‘par2 limit min’, ‘par2 limit max’, ‘par3 limit min’, and ‘par3 limit max’ specify the minimum and maximum range of the ‘par2’ and ‘par3’ respectively.

funtype

spectral model

index

par2

par3

0

PL

PowerLaw

x^(-[index])

α

1

PC

PLExpCutoff

x^(-[index]) * e^(- x / [par2])

α

Ec

2

PS

PLSuperExpCutoff

x^(-[index]) * e^(- pow(x / [par2], [par3]))

α

Ec

β

3

LP

LogParabola

( x / [par2] ) ^ ( -( [index] + [par3] * log ( x / [par2] ) ) )

α

Ec

β

The match of the parameteres is:

  • index = α: Spectral index for PL, PC, and PS spectral models, first index for LP spectral model; Could be Index or Index1 in the XML format

  • par2 = Ec (MeV): cut-off energy for PC and PS spectral models, pivot energy for LP spectral model;

  • par3 = β: Second index for PS spectral models, curvature for LP spectral model;

The usual energy range used to calculate these parameters is 100 MeV – 10 GeV. The MLE procedure calculates also the 1σ uncertainty of the spectral parameters:

  • ∆α: Statistical 1σ uncertainty of α;

  • ∆Ec (MeV): Statistical 1σ uncertainty of Ec;

  • ∆β: Statistical 1σ uncertainty of β.

According to the ‘fixflag’ some or all of those values will be optimized by being allowed to vary. The fixflag is a bit mask, each bit indicating whether the corresponding value is to be allowed to vary:

fixflag = 0 everything is fixed (free=”0”)
fixflag = 1 indicates the flux (free=”1” in <parameter name=”Flux”>)
fixflag = 2 the position is free (free=”1” in <spatialModel type=”PointSource”>)
fixflag = 4 the Index or Index1 is free (free=”1” in <parameter name=”index”> or <parameter name=”index1”> )
fixflag = 8 the par2 is free (free=”1” in <parameter name=”cutoffEnergy”> or <parameter name=”pivotEnergy”>)
fixflag = 16 the par3 is free (free=”1” in <parameter name=”index2”> or <parameter name=”curvature”>)
fixflag = 32 force position to be variable only in Loop2 (free=”2” in <spatialModel type=”PointSource”>)

The user may combine these values, but the flux will always be allowed to vary if at least one of the other values are.

flux

pos(free=1)

index/index1

par2=cutoffEnergy/pivotEnergy

par3=index2/curvature

pos(free=2)

fixflag

1

2

4

8

16

32

Examples:
fixflag = 0: everything is fixed. This is for known sources which must be included in order to search for other nearby sources.
fixflag = 1: flux variable, position fixed
fixflag = 2: only the position is variable, but MLE will let the flux vary too, so this is equivalent to 3.
fixflag = 3: flux and position variable, index fixed
fixflag = 4: index variable (and flux variable)
fixflag = 5: flux and index variable, position fixed
fixflag = 7: flux, position and index variable and also
fixflag = 28: index, par2 and par3 variable (and flux variable)
fixflag = 30: position, index, par2 and par3 variable (and flux variable)
fixflag = 32: position=2, the rest is fixed

Output files of the MLE

This page provides detailed description of the output files of the science tool AG_multi.

The details of the output of the science tool AG_multi that performs the likelihood procedure is still accessible from agilepy. This section describes the “low level” results of the AG_multi procedure. The results are available in the $HOME/agilepy_analysis/<sourcename>_<username>_<date>-<time>/mle directory, where <sourcename> and <username> are defined in the yaml configuration file, <date> and <time> are defined by the system when the analysis starts.

At the end of the fitting process AG_multi generates two main files, describing the most relevant results for all the sources, and a set of source-specific files containing more detailed data about that source.

One of the two main files is in HTML format, and it includes both the input and output data grouped in tables. Having a look at this file the user should quickly understand the outcome of the fitting process and its main results. The next section describes the HTML output in more detail.

The second of the two main files contains the same data printed in text format. This file is divided in two sections. The first contains one line for each diffuse component and the second one line for each source. The first line of each section begins with an exclamation mark (a comment line for many applications) labeling the values printed beneath. In each line the values are separated by a space. This is an example of the text output of the analysis of the 2AGLJ2254+1609 (3C454.3) with the test dataset provided. For this analysis, only one set of maps and one source is used. The iotropic emission components coefficients are kep free and symmetric errors are provided. The flux and position of the source are allowed to vary, while the spectral index is fixed. The name, significance of the source detection, position, source counts with error, source flux with error, and spectral index with error are provided.

! DiffName, Flux, Err, +Err, -Err
Galactic 0.7 0 0 0
Isotropic 8.79898 0.969867 0.984804 -0.955381
! SrcName, sqrt(TS), L_peak, B_peak, Counts, Err, Flux, Err, Index, Err, Par2, Par2Err, Par3, Par3Err, TypeFun
2AGLJ2254+1609 35.5482 86.0638 -38.1753 719.369 35.2059 2.63371e-05 1.28894e-06 2.20942 0 0 0 0 0 0

index, par2, par3 and related errors depend by the spectral mode used.

The counts and fluxes are provided, as well as their errors if the flux is allowed to vary. Finally, the spectral index and its error, if applicable, are provided.

Note

If a source is outside the Galactic plane, fix the diffuse emission coefficient parameter (gal) to 0.7 with ag.setOptions(galcoeff=[0.7])

.source’ file

The .source file is an internal technical file produced by the maximum likelihood estimator mle() procedure for each source. It contains all the analysis results for each source that is part of the ensemble of models. Agilepy extract from this .source file the most important parameters useful for the final user.

When possible, two additional files describing the source contour (possibile only if position is kept free).

The text file contains some comment-like lines (first character is an exclamation mark) labeling the values printed beneath. This is an example of text output, consistent with the example given above:

! Label Fix index ULConfidenceLevel SrcLocConfLevel start_l start_b start_flux [ lmin,  lmax ] [ bmin, bmax ] typefun par2 par3 galmode2 galmode2fit isomode2 isomode2fit edpcor fluxcor integratortype expratioEval expratio_minthr expratio_maxthr expratio_size [ index_min , index_max ] [ par2_min , par2_max ] [ par3_min , par3_max ] contourpoints minimizertype minimizeralg minimizerdefstrategy minimizerdeftol
! sqrt(TS)
! L_peak B_peak Dist_from_start_position
! L B Dist_from_start_position r a b phi
! Counts Err +Err -Err UL
! Flux(ph/cm2s) [0 , 1e+07] Err +Err -Err UL(ph/cm2s) ULbayes(ph/cm2s) Exp(cm2s) ExpSpectraCorFactor null null null Erglog(erg/cm2s) Erglog_Err Erglog_UL(erg/cm2s) Sensitivity FluxPerChannel(ph/cm2s)
! Index [0.5 , 5] Index_Err Par2 [20 , 10000] Par2_Err Par3 [0 , 100] Par3_Err
! cts fitstatus0 fcn0 edm0 nvpar0 nparx0 iter0 fitstatus1 fcn1 edm1 nvpar1 nparx1 iter1 Likelihood1
! Gal coeffs [0 , 100] and errs
! Gal zero coeffs and errs
! Iso coeffs [0 , 100] and errs
! Iso zero coeffs and errs
! Start_date(UTC) End_date(UTC) Start_date(TT) End_date(TT) Start_date(MJD) End_date(MJD)
! Emin..emax(MeV) fovmin..fovmax(deg) albedo(deg) binsize(deg) expstep phasecode ExpRatio
! Fit status of steps ext1, step1, ext2, step2, contour, index, ul [-1 step skipped, 0 ok, 1 errors]
! Number of counts for each step (to evaluate hypothesis)
! skytypeL.filter_irf skytypeH.filter_irf
2AGLJ2254+1609 1 2.20942 2 5.99147 86.1236 -38.1824 2.64387e-05 [ -1 , -1 ]  [ -1 , -1 ]  0 0 0 0 0 0 0 0.75 0 1 1 0 15 10 [ 0.5 , 5 ] [ 20 , 10000 ] [ 0 , 100 ] 40 Minuit Migrad 2 0.01
47.8468
86.1236 -38.1824 0
-1 -1 -1 -1 -1 -1 -1
718.633 31.0247 31.4119 -30.6392 782.234
2.64387e-05 1.14141e-06 1.15565e-06 -1.12722e-06 2.87787e-05 2.01487e-05 2.71811e+07 1 0 0 0 4.27293e-09 1.8447e-10 4.6511e-09 0.0 2.64387e-05
2.20942 0 0 0 0 0
909 -1 2456.44 0.5 0 8 3 0 1311.78 7.28513e-16 1 8 3 1828.16
0.7 0
0.7 0
8.83231 0
8.83231 0
2010-11-13T00:01:06 2010-11-21T00:01:06 216691200.0000000 217382400.0000000 55513.0000000 55521.0000000
100..10000 0..60 80 0.25 0 6 0
-1 -1 -1 0 -1 -1 0
-1 2124 -1 2124 -1 -1 2124
SKY002.SFMG_H0025 SKY002.SFMG_H0025

The counts and fluxes are provided, as well as their symmetric, positive, and negative errors if the flux is allowed to vary. For convenience, the exposure of the source, used to calculate the source counts from the flux, is also provided. Finally, the spectral index and its error, if applicable, are provided.

.source’ Attributes

Parameter name

UM

Description

default

range

Label

Fix

Value of the fixflag

index

Initial value of the index parameter

ULConfidenceLevel

Upper limit confidence level espressed as sqrt(TS)

2

SrcLocConfLevel

Source location contour confidence level %

95 99 95 68 50

95

start_l

start_b

start_flux

(ph/cm2s)

[ lmin lmax ]

[ bmin bmax ]

typefun

par2

par3

galmode2

galmode2fit

isomode2

isomode2fit

edpcor

fluxcor

integratortype

expratioEval

expratio_minthr

expratio_maxthr

expratio_size

[ index_min index_max ]

[ par2_min par2_max ]

[ par3_min par3_max ]

contourpoints

minimizertype

minimizeralg

minimizerdefstrategy

minimizerdeftol

sqrt(TS)

L_peak

B_peak

Dist_from_start_position

L

B

Dist_from_start_position

r

a

b

phi

Counts

Err

+Err

-Err

UL

Flux

(ph/cm2s)

Err

+Err

-Err

UL

(ph/cm2s)

ULbayes

(ph/cm2s)

Exp

(cm2s)

ExpSpectraCorFactor

Erglog

(erg/cm2s)

Erglog_Err

Erglog_UL

(erg/cm2s)

Sensitivity

FluxPerChannel

(ph/cm2s)

Index

Index_Err

Par2

Par2_Err

Par3

Par3_Err

cts

fitstatus0

fcn0

edm0

nvpar0

nparx0

iter0

fitstatus1

fcn1

edm1

nvpar1

nparx1

iter1

Likelihood1

Gal coeffs

errs

Gal zero coeffs

errs

Iso coeffs

errs

Iso zero coeffs

errs

Start_date(UTC)

End_date(UTC)

Start_date(TT)

End_date(TT)

Start_date(MJD)

End_date(MJD)

Emin..emax

MeV

fovmin..fovmax

deg

albedo

deg

binsize

deg

expstep

phasecode

ExpRatio

Fit status of steps ext1

Fit status of steps step1

Fit status of steps ext2

Fit status of steps step2

Fit status of steps contour

Fit status of steps index

Fit status of steps ul

Number of counts for ext1

Number of counts for step1

Number of counts for ext2

Number of counts for step2

Number of counts for contour

Number of counts for index

Number of counts for ul

skytypeL.filter_irf

skytypeH.filter_irf

.source.con’ file

outfile.source.con: source contour (if found).

.source.reg’ file

outfile.source.reg: ellipse best fitting the source contour (if found).

.log’ file

Log file with a line for each step of the fitting process.

HTML output

AG_multi provides an HTML output of the results. The HTML output file is divided into two sections, input and output. The input section contains three subsections: the command line options, the map list and the source list contents. The command line options are listed in two tables, one with the names of the IRFs (PSD, SAR and EDP) files, the other with the rest of the command line. The maplist subsection also contains two tables. The first lists the mapfile contents and the second contains the data from the map files themselves. This table contains one map per row, and each column contains one value only if it is the same for all the maps. The last table of the input section contains the source list contents. The output section is also divided into three subsections. The first is a table showing the Galactic and isotropic coefficients and their errors. Also in this table some cells may be grouped together when the values are all the same. The second is a table showing the fit results for the sources and their errors. One of the listed values is the contour equivalent radius, explained in the next section. The last table shows the source flux per energy channel, and it is present only when different energy channels are considered. This table has one row for each source and one column for each energy channel.

Need Help

If you have troubles please email to:

  • leonardo.baroncelli[at]inaf.it

  • antonio.addis[at]inaf.it

  • andrea.bulgarelli[at]inaf.it

or open an issue of GitHub: https://github.com/AGILESCIENCE/Agilepy/issues

Known issues

  • The unit test “test_aperture_photometry” fails on macos, therefore the aperturePhotometry() method is not available on this OS.

  • Each notebook should instantiate only one AGAnalysis object, otherwise the logger will be duplicated.

  • In certain situations %matplotlib widget has a weird behaviour. If you have problems with map sizes or interactions, comment the line %matplotlib widget

Development

Install the development environment

If you want to try agilepy’s new features that are not officially released yet, a develpoment environment called agilepy-environment is available into Anaconda cloud. It contains all the dependencies unless agilepy, which must be installed by hand cloning the repository.

Anaconda

conda config --add channels conda-forge
conda config --add channels plotly
conda create -n agilepydev -c agilescience agiletools agilepy-dataset
conda activate agilepydev
git clone https://github.com/AGILESCIENCE/Agilepy.git
cd Agilepy && git checkout develop
conda env update -f environment.yml
python setup.py develop

Docker

docker pull agilescience/agilepy-recipe:latest
mkdir shared_dir && cd shared_dir && git clone https://github.com/AGILESCIENCE/Agilepy.git \
&& cd Agilepy && git checkout develop

docker run --rm -it -p 8888:8888 \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
-v $SHARED_DIR_PATH:/shared_dir \
agilescience/agilepy-recipe:latest

## -- Inside the container --
conda activate agilepydev
cd /shared_dir/Agilepy python setup.py develop

jupyter notebook --port 8889 --ip 0.0.0.0  --allow-root

Now you have the agilepy’s latest version installed in your environment.

Git flow

Branches

Two main branches:

  • master: contains only production releases.

  • develop: contains commits that will be included in the next production release.

Two support branches:

  • feature branch: each new feature (Trello’s card) should be developed in its own feature branch, branching from develop and merged back into it. The feature branch are not pushed into the remote.

  • hotfix branch: if an hotfix is needed it should be develop in its own branch, branching from master and merged back to it.

Git flow

Versioning

The master branch contains only production releases: when the develop branch (or hotfix branch) is merged to master a new release tag must be created. Its name follows the semantic versioning.

x.y.z

Incrementing:

  • x version when you make incompatible API changes,

  • y version when you add functionality in a backwards compatible manner, and

  • z version when you make backwards compatible bug fixes.

Branches names

The master and the develop branch have an infinite lifetime, hence their name is fixed.

The feature branch takes the following format:

feature-#<card-number>-<short-description>

e.g. feature-#61-new-cool-feature

The hotfix branch name takes the following format:

hotfix-#<card-number>-<release-number>

e.g. hotfix-#57-1.0.0

The release number is the one of the production release from which it originates from.

Getting started

Development of a new feature

Create a new feature branch:

git checkout develop
git pull origin develop
git checkout -b feature-#61-new-cool-feature develop

Development and testing of the new feature.

When you have finished, update the CHANGELOG.md and commit your changes.

vim CHANGELOG
git commit -m "feature-#61-new-cool-feature done"

In the meantime it is possible that someone else have pushed his work into the develop branch. In this case you have to merge the changes in your feature branch.

git pull **origin** develop

Finally you can merge your feature branch back to develop branch.

git merge --no-ff feature-#61-new-cool-feature
git branch -d feature-#61-new-cool-feature
git push origin develop
Add configuration parameters

Let’s say we want to add the following configuration section to the AGAnalysis’ configuration file.

ap:
    radius: 0.25
    timeslot: 3600
  • Add the new section to the AGAnalysis.getConfiguration() method.

  • Add the type of the configuration parameters within the AGAnalysisConfig.checkOptionsType() method (in the corresponding lists).

  • If the parameters need some kind of validation (this is not the case), add a new method in ValidationStrategies and call it within the AGAnalysisConfig.validateConfiguration() (check examples).

  • If the parameters need some kind of transformation (this is not the case), add a new method in CompletionStrategies and call it within the AGAnalysisConfig.completeConfiguration() (check examples).

  • Add the new configuration section to all the unit test configuration files.

  • Document the new configuration parameters within the manual/configuration_file.rst file.

Add a new science tool

Let’s say we want to add a new (c++) science tool: AG_ap.

  • Add a new class within the api/ScienceTools.py script. You need to implement some abstract methods.

  • You can use the new class as follows:

apTool = AP("AG_ap", self.logger)
apTool.configureTool(self.config)
if not apTool.allRequiredOptionsSet(self.config):
    raise ScienceToolInputArgMissing("Some options have not been set.")
products = apTool.call()
Release of a new version

Change the version of the software in setup.py. The version increment must be take in account all the commits of the develop branch. You can check the CHANGELOG.md to facilitate this process. Please, add the new tag within the CHANGELOG.md file.

git checkout master
git merge --no-ff develop
git tag -a <new-tag>
git push origin <new-tag>

DevOps

A high level description of agilepy’s devops is in the image below:

Git flow

This scheme workflow produces three images:

  • base_image: It’s an image with all the dependencies except Agilepy python library, it’s used for developing purposes only by developers. Base image is built after a new commit in agilepy-recipe repository.

  • latest code image: It’s the base_image with Agilepy’s develop branch at latest commit, useful for using or testing agilepy’s updates not officially released. This image is not supported nor stable and is built by dockerhub after github’s testing pipelines are successful.

  • released image: The base_image with Agilepy’s release tag. By default the community shall be download this image. It’s built when a new tag is created.