Agilepy’s documentation¶
Agilepy is an open-source Python package developed at INAF/OAS Bologna to analyse AGILE/GRID data built on top of the command-line version of the AGILE/GRID Science Tools.
The main purpose of the package is to provide an easy to use high-level Python interface to analyse AGILE/GRID data by simplifying the configuration of the tasks and ensuring straightforward access to the data. The current features are the generation and display of sky maps and light curves, the access to gamma-ray sources catalogues, the analysis to perform spectral model and position fitting including the background evaluation, the aperture photometry analysis, and the wavelet analysis. In addition, Agilepy provides an engineering interface to analyse the time evolution of the AGILE off-axis viewing angle for a chosen sky region, comparing them with Fermi/LAT off-axis evolution.

Agilepy¶
Agilepy is an open-source Python package developed at INAF/OAS Bologna to analyse AGILE/GRID data built on top of the command-line version of the AGILE/GRID Science Tools.
The main purpose of the package is to provide an easy to use high-level Python interface to analyse AGILE/GRID data by simplifying the configuration of the tasks and ensuring straightforward access to the data. The current features are the generation and display of sky maps and light curves, the access to gamma-ray sources catalogues, the analysis to perform spectral model and position fitting including the background evaluation, the aperture photometry analysis, and the wavelet analysis. In addition, Agilepy provides an engineering interface to analyse the time evolution of the AGILE off-axis viewing angle for a chosen sky region, comparing them with Fermi/LAT off-axis evolution.
Agilepy is similar to Fermipy (https://fermipy.readthedocs.io/) and gammapy (https://docs.gammapy.org/) tools, providing a common way to analyse gamma-ray data.
Agilepy provides the last version of the available Science Tools (BUILD25), the H0025 instrument response functions (IRFs), and the latest version of the diffuse Galactic emission model.
Agilepy (and its dependencies) can be easily installed using Anaconda (https://www.anaconda.com/).
AGILE¶
AGILE (Astrorivelatore Gamma ad Immagini LEggero) is an astrophysics mission of the Italian Space Agency (ASI) operating since 2007 April devoted to gamma-ray and X-ray astrophysics. It carries two instruments observing at hard X-rays between 18 and 60 keV (Super-AGILE) and in the gamma-ray band between 30 MeV and 50 GeV (Silicon Tracker). The payload is completed by a calorimeter (MCAL) sensitive in the 0.4–100 MeV range and an anticoincidence (AC) system. The combination of the Silicon Tracker, MCAL, and AC forms the Gamma-Ray Imaging Detector (GRID).
A set of different on-board triggers enables the discrimination of background events (mainly cosmic rays in the AGILE Low Earth Orbit) from gamma-ray events. AGILE raw data are down-linked every about 100 min to the ASI Malindi ground station in Kenya, and transmitted first to the Telespazio Mission Control Center at Fucino, and then to the AGILE Data Center (ADC), which is part of the ASI Space Science Data Center (SSDC, previously known as ASDC) and then to the INAF/OAS Bologna for the real-time analysis of data.
Main AGILE websites:
IAPS/Rome: http://agile.rm.iasf.cnr.it/
AGILE Apps:
AGILE Science Tools¶
The AGILE/GRID Science Tools developed by the AGILE Team are used to analyse gamma-ray data starting from spacecraft files (called LOG), and the acquired events (EVT files or event list). They provide a way to generate gamma-ray counts, exposure and diffuse emission maps that are used as input for the binned maximum likelihood estimator (MLE). The analysis depends on the isotropic and Galactic diffuse emission, the gamma-ray photon statistics, and on the instrument response functions (IRFs). IRFs are matrices that characterise the effective area (Aeff), the point spread function (PSF), and the energy dispersion probability (EDP), that depend on the direction of the incoming gamma-ray in instrument coordinates, its energy and on the on-ground event filter.
The result of the MLE is an evaluation of the presence of one or more point-like or extended sources in the sky maps: this is the essential step for the scientific results of AGILE.
A full description and characterisation of the last release of the Science Tools is available in https://arxiv.org/abs/1903.06957. Science Tools, IRFs and Galactic emission model are publicly available from the AGILE website at SSDC: https://agile.ssdc.asi.it.
Agilepy analysis¶
The AGILE-GRID data analysis can be performed with Agilepy with different techniques:
using the maximum likelihood estimator analysis
wavelet techniques
Lomb-Scargle periodogram analysis (coming soon)
The likelihood analysis reach better sensitivity, more accurate flux measurement, better evaluation of the backgrounds and can work with a detailed source models where more sources can be considered at the same time.
AGILE-GRID light curves can be created in two different ways:
using the maximum likelihood estimator analysis
using aperture photometry.
Aperture photometry provides a raw measure of the flux of a sigle source and is less computing demanding.
Maximium Likelihood Estimator caveats¶
During the fitting process some values are fixed and others are variable, depending on the values of the flags. The execution time strongly depends on the number of the variable parameters. It is not possible to predict how long the fitting process will last or how it depends on the number of parameters, but the dependence is not linear. If all the diffuse coefficients are variable and all spectral parameters are free, for M maps and S sources the number of variable parameters will be 2M+4S. In the case of many maps and many sources, this may lead to a very long execution time.
The fitting process takes place in two steps, according to the method of Maximum Likelihood. During each step all the sources are considered one by one, and several fitting attempts are performed by invoking the function TH1D::Fit() provided by the ROOT library, developed by CERN and will find the related documentation on the CERN web site.
Installation¶
Agilepy is available as Anaconda package or into a ready-to-use Docker container (from 1.4.0)
Note
AGILE DATASET DOWNLOAD Now it possible to download all the public AGILE dataset stored on SSDC datacenter through a REST Api. Agilepy automatically handles the data and no actions are required from the user. For more information visit this page. This major release includes many new important features and a general refactoring.
Installation with Anaconda¶
Agilepy (and its dependencies) can be easily installed using Anaconda. You just need to decide the name of the virtual environment that will be created by anaconda.
conda config --add channels conda-forge
conda config --add channels plotly
conda create -n <virtualenv_name> -c agilescience agilepy
Note
If you want to try agilepy’s new features that are not officially released yet, a develpoment environment called agilepy-environment is available into Anaconda cloud. It contains all the dependencies unless agilepy, which must be installed by hand cloning the repository. Check the installation instructions here
Supported platforms:
linux-64
osx-64
Note
An experimental package for IBM POWER architecture(ppc64le) is available on Anaconda cloud. Due to some incompability this package does not contain ROOT and AGILE science tools that need to be installed from source. Check the instructions to install AGILE science tools here
Tested on:
CentOs 7.6
Ubuntu 18.04
Ubuntu 19.10
Ubuntu 20.04
macOs 10.14
macOs 10.15
macOS 12.0.1
In order to use the software you need to activate the virtual environment first:
conda activate <virtualenv_name>
or
source activate <virtualenv_name>
Running jupyter server:
start_agilepy_notebooks.sh
Installation with Docker¶
You can pull the image directly from dockerhub using the following command:
docker pull agilescience/agilepy:release-<version>
Note
Check the installation instructions for Docker here
Note
If you want to try agilepy’s new features that are not officially released yet, you need to pull a develop image available using agilepy:develop-latest tag
Using this command you can launch the container and automatically start jupyter notebook.
docker run --rm -it -p 8888:8888 \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
-v $PWD/shared_dir:/shared_dir \
agilescience/agilepy:release-<version> /bin/bash -c \
"source /opt/anaconda3/etc/profile.d/conda.sh && conda activate agilepydev && \
jupyter notebook --ip='*' --port=8888 --no-browser --allow-root --notebook-dir="/Agilepy/agilepy/notebooks" --NotebookApp.token='' --NotebookApp.password=''"
shared_dir must be created before launching the command, it is not necessary, but useful for several cases (exporting analysis outside the container, link another dataset etc.)
Jupyter server is at localhost:8888
Agilepy’s containers can be found at dockerhub page
Supported platforms:
linux-64
osx-64
win-64(see note)
Tested on:
CentOs 7.6
Ubuntu 18.04
Ubuntu 19.10
Ubuntu 20.04
macOs 10.14
macOs 10.15
Windows 10 v2004 (May 2020 Update)
Manual Installation¶
If the isntallation does not work with the instructions above, it is recommended to install agilepy and its dependencies from scratch. The dependencies required by Agilepy are:
Root 6.26 Cfitsio 4.1 Zlib
AGILE’s Science Tools (the correct tag to install is on sciencetools_version.txt in the repository main directory)
Uninstalling¶
Anaconda
conda env remove --name <virtualenv_name>
Docker
docker rmi agilescience/agilepy:release-<version>
Package distribution structure¶
The virtual environment <virtualenv_name> folder is under the “envs” folder within the root folder of your anaconda installation.
It contains all the dependencies Agilepy requires. Here, there is the “agiletools” directory, containing AGILE’s scientific software.
Quickstart guide¶
To import the library:
from agilepy.api import AGAnalysis
You can create the (required) yaml configuration file, calling the following static method:
AGAnalysis.getConfiguration(
"./agconfig.yaml", # the destination path of the configuration file
"username", # the name of the flare advocate
"OJ287", # the name of the source
58930, # tmin
58936, # tmax
"MJD", # time type
206.8121188769472, # glon
35.8208923457401, # glat
"$HOME/agilepy_analysis", # the destination path of the output directory
1, # the verbosity level
evtfile="/AGILE_PROC3/FM3.119_ASDC2/INDEX/EVT.index", # optional parameter
logfile="/AGILE_PROC3/DATA_ASDC2/INDEX/LOG.log.index" # optional parameter
)
In order to interact with the library you need to obtain an instance of the AGAnalysis class:
ag = AGAnalysis('agconfig.yaml')
Then you have to load the models of the sources (you can filter them by their distance (degree) from l,b provided within the configuration file):
sources = ag.loadSourcesFromCatalog('2AGL', rangeDist=(0, 10))
Keyword arguments can be passed via setOptions() to override configuration parameters:
ag.setOptions(binsize=0.50, outdir="./output")
To generate sky maps:
maplistfile = ag.generateMaps()
To display and interact with the sky maps:
ag.displayCtsSkyMaps(smooth=True, sigma=3)
ag.displayExpSkyMaps()
ag.displayGasSkyMaps()
To perform an maximum likelyhood estimation analysis:
sourcefiles = ag.mle()
You can query the sources with an arbitrary boolean expression string..
selectedSources = ag.selectSources("flux > 0 AND dist <= 1 OR sqrtTS > 3")
..and fix or free a source’s parameter:
sourcefiles = ag.freeSources('name == "CYGX3"', "flux", True)
You can generate a light curve data file with…
lightCurveData = ag.lightCurveMLE("CYGX3", tmin=58930 , tmax=58936, binsize=10800)
…and display the interactive light curve plot with:
ag.displayLightCurve("mle")
If you want to manually update the value of a source’s spectrum parameter, you can do it with:
sources = ag.selectSources('name == "2AGLJ2021+4029"')
source = sources.pop()
source.spectrum.set("index", 1.8)
Hint
Check out the API documentation and the Jupiter notebooks section!
Jupyter Notebooks¶
Several Jupyter notebooks are available. You can start a Jupyter server calling:
start_agilepy_notebooks.sh
Tutorial notebooks¶
There’re several categories of tutorial notebooks:
science_api_tutorial: the most important ones. They show the basic usage of Agilepy to perform a scientific analysis for sources detection. The following notebooks are useful example of the use of Agilepy and they can be runned using provided sample data:
Wavelet analysis: it shows how to use the Agilepy’s wavelet analysis API.
engineering_api_tutorial: they show how to use the Agilepy’s engineering analysis API.
The following notebook is another useful example of the use of Agilepy that is runned downloading AGILE data from SSDC website:
PKS1510-089: analysis of 2009 gamma-ray flare
Analysis notebooks¶
These notebooks have been developed for internal purposes of the AGILE Team. A template notebook is also provided to speed up the development of a new analysis notebook for AGILE Flare Advocate team.
Download AGILE-GRID data¶
The AGILE-GRID data can be downloaded in two ways: automated by Agilepy or manually.
Automated download using SSDC REST Api¶
The AGILE-GRID data is download automatically by agilepy.
Version 1.5.0 has implemented the SSDC REST Api in order to get the AGILE dataset from SSDC datacenter. All the required data is downloaded according to tmin and tmax values selected in configuration file. This feature works when generatesmaps method is called.
- Pre-requisites:
Internet connection (> 200 Mb/s)
- SSDC Data policy:
EVT files contain 15 days of data (2 files per month)
LOG files contain 1 day of data
- Eg for getting data from 10/10/2018 to 05/11/2018 it returns:
3 EVT files (30/09-15/10, 15/10-31/10, 31/10-15/11)
26 LOG files, one file for each day
Two query files are created to keep track of the query history and to implement the policy above. Before calling Rest api, Agilepy checks if the dates selected are in query files, if True download is not performed. If False Agilepy downloads the data in /tmp/ folder, it unpacks them into the selected datapath and it automatically calls indexgen tool for generating index files. Finally, it updates query files.
Example¶
AGAnalysis.getConfiguration(
confFilePath = confFilePath,
evtfile=None,
logfile=None,
userName = "username",
sourceName = "PKS1510-089",
tmin = 54891,
tmax = 54921,
timetype = "MJD",
glon = 351.29,
glat = 40.13,
outputDir = "$HOME/agilepy_analysis",
verboselvl = 0,
userestapi=True,
datapath="$HOME/agile_dataset"
)
Advanced Information¶
Query files¶
Agilepy uses text files called “qfiles”. These files contain the slots requested by the user, according to SSDC policies. With query file it is possible to avoid multiple downloads for the same dates (useful for slow connections and if there are no data in the selected range days).
Index files¶
Index file is created by Indexgen tool immediately after the download. No action is required from the user.
Plotting index files vs query files¶
A function to plot the dates from index and query file can be useful to check differences between asked data and real data. Sometimes could happen that data are not available for several reasons (instruments off etc), data will not be downloaded but agilepy writes in query files in order to not perform a second request.
AGILE Data Coverage¶
SSDC uploads AGILE dataset once per month, this means that it could not be possible to select a date close to the present day. In this particular case, query files must not be uploaded, because in the future data will be available. Agilepy gets AGILE data coverage from SSDC and writes it into a file called Agilepy_coverage, when AGDataset starts it checks if last coverage is more than 60 days old from the present date and it updates it if positive.
Manual download¶
The AGILE-GRID data obtained both in pointing and in spinning mode are publicly available and can be download manually from the ASI/SSDC https://www.asdc.asi.it/mmia/index.php?mission=agilemmia
Prepare index files¶
There’re two types of data files: events list (EVT) and log data (LOG). They both are compressed fits files. Each file refers to a specific time interval.
Example:
agql1511240600_1511240730.LOG.gz
agql1511240730_1511240900.EVT__FM.gz
In order to use Agilepy (or the Agile science tools) a special file, called “index”, is needed. This file is used by Agilepy to know the position of the data files and which file refers to which interval. Two index files are needed: one for the event data and one for the log data.
Those index files have four column:
file name
time start of the file in Terrestrial Time (TT)
time end of the file in Terrestial time (TT)
LOG or EVT marker to identify the fole types
Here some examples of LOG and EVT indexes:
head -n 3 /ASDC_PROC3/DATA_ASDCe/INDEX/LOG.log.index
/AGILE_PROC3/DATA_ASDCe/LOG/ag-107092735_STD0P_GO.LOG.gz 107092735. 107179134.9 LOG
/AGILE_PROC3/DATA_ASDCe/LOG/ag-107179135_STD0P_GO.LOG.gz 107179135. 107265534.9 LOG
/AGILE_PROC3/DATA_ASDCe/LOG/ag-107265535_STD0P_GO.LOG.gz 107265535. 107351934.9 LOG
head -n 3 /ASDC_PROC3/FM3.119_ASDCSTDk/INDEX/EVT.index
/ASDC_PROC3/FM3.119_ASDCSTDk/EVT/ag0910311200_0911301200_STD1Kal_FM.EVT.gz 184075134.000000 186667134.000000 EVT
/ASDC_PROC3/FM3.119_ASDCSTDk/EVT/ag0911301200_0912201200_STD1Kal_FM.EVT.gz 186667134.000000 188395134.000000 EVT
/ASDC_PROC3/FM3.119_ASDCSTDk/EVT/ag0912201200_1001151200_STD1Kal_FM.EVT.gz 188395134.000000 190641534.000000 EVT
You can use the AG_indexgen tool to generate the .index file:
AG_indexgen <path to data> <type> <output file>
Where <type> can be EVT or LOG.
Example:
AG_indexgen /AGILE_PROC3/FM3.119_ASDC2/EVT EVT /home/user/data.index
Agilepy test data¶
The Agilepy conda package gets shipped with two subsets of the AGILE data archive for the purpose of unit testing and to show how to run scientific analysis with the tutorial notebooks.
test_dataset_6.0¶
A test data to analyse Vela region. The provided period is MJD 58026.50-58031.50.
The index files are the following:
evtfile="$AGILE/agilepy-test-data/test_dataset_6.0/EVT/EVT.index"
logfile="$AGILE/agilepy-test-data/test_dataset_6.0/LOG/LOG.index"
test_dataset_agn¶
A test data to analyse the November’s 2010 flare of 3C454.3 source. The provided period is MJD 55513.00-55520.00.
The index files are the following:
evtfile="$AGILE/agilepy-test-data/test_dataset_agn/EVT/EVT.index"
logfile="$AGILE/agilepy-test-data/test_dataset_agn/LOG/LOG.index"
Configuration file¶
General¶
A yaml configuration file is required in order to run Agilepy.
It is composed by several sections and each section holds several configuration options, some of which are optional (having a default value), some others are required.
It supports environment variables (they can be used to define file system paths).
It can be created easily, calling the following static method and passing the minimal set of (required) configuration parameters.
AGAnalysis.getConfiguration(
confilepath="./agconfig.yaml", # the destination path of the configuration file
userName="username", # the name of the flare advocate
sourceName="OJ287", # the name of the source
tmin=58930, # tmin
tmax=58936, # tmax
timetype="MJD", # time type
glon=206.8121188769472, # glon
glat=35.8208923457401, # glat
outputDir="$HOME/agilepy_analysis", # the destination path of the output directory
verboselvl=1, # the verbosity level
evtfile="evt indexfile", # optional parameter
logfile="log indexfile", # optional parameter
datapath="datapath",
userestapi=True,
)
The method above will create the following configuration file:
input:
evtfile: None
logfile: None
userestapi: True
datapath: datapath
output:
outdir: $HOME/agilepy_analysis
filenameprefix: analysis_product
logfilenameprefix: analysis_log
sourcename: sourcename
username: username
verboselvl: 2
selection:
emin: 100
emax: 10000
tmin: 54935.0
tmax: 54936.0
timetype: MJD
glon: 355.447
glat: -0.2689
proj: ARC
timelist: None
filtercode: 5
fovradmin: 0
fovradmax: 60
albedorad: 80
dq: 0
phasecode: null
lonpole: 180
lpointing: null
bpointing: null
maplistgen: "None"
maps:
mapsize: 40
useEDPmatrixforEXP: false
expstep: null
spectralindex: 2.1
timestep: 160
projtype: WCS
proj: ARC
binsize: 0.25
energybins:
- 100, 10000
fovbinnumber: 1
offaxisangle: 30
model:
modelfile: null
galmode: 1
isomode: 1
galcoeff: null
isocoeff: null
emin_sources: 100
emax_sources: 10000
galmode2: 0
galmode2fit: 0
isomode2: 0
isomode2fit: 0
mle:
ranal: 10
ulcl: 2
loccl: 95
expratioevaluation: true
expratio_minthr: 0
expratio_maxthr: 15
expratio_size: 10
minimizertype: Minuit
minimizeralg: Migrad
minimizerdefstrategy: 2
mindefaulttolerance: 0.01
integratortype: 1
contourpoints: 40
edpcorrection: 0.75
fluxcorrection: 0
ap:
radius: 3
timeslot: 3600
plotting:
twocolumns: False
Updating the configuration options¶
The user should not directly manipulate the configuration file, because the configuration file is read only once, when the AGBaseAnalysis constructor is called. Hence, the configuration file modification will not affect the internal configuration object. Also, updating the values held by this object will not affect the original values written on disk.
In order to update the internal configuration object, the user can rely on the following methods:
For example:
ag.setOptions(binsize=0.50, energybins=[[100, 300], [500, 1000]])
print(ag.getOption("energybins"))
Configuration options¶
This section describes the configuration options.
Section: ‘input’¶
This section defines the input data files. The input data files are indexes: each row holds the file system position of an actual event data/log file, together with the time interval it refers to. If userestapi if True the selection of evtfile and logfile is not required, Agilepy creates its own index files automatically. See more details in this link.
Option |
Description |
Type |
Required |
Default |
---|---|---|---|---|
evtfile |
Path to index evt file name |
str |
no |
None |
logfile |
Path to index log file name |
str |
no |
None |
userestapi |
If true downloads date into datapath |
bool |
no |
True |
datapath |
the position of AGILE data |
str |
no |
None |
Section: ‘output’¶
The output section collects options related to the output files generation and logging.
The ‘outdir’ option sets the root directory of the analysis results where all output files are written.
Agilepy use two loggers, one logs messages on the console, the other writes messages on disk. The ‘verboselvl’ option sets the verbosity of the Agilepy console logger. The Agilepy file logger verbosity is set to 2 by default. There are 4 kind of messages based on their importance factor:
CRITICAL: a message describing a critical problem, something unexpected, preceding a program crash or an Exception raise.
WARNING: an indication that something unexpected happened, or indicative of some problem in the near future (e.g. ‘disk space low’). The software is still working as expected.
INFO: confirmation that things are working as expected.
DEBUG: detailed information, typically of interest only when diagnosing problems.
Option |
Description |
Type |
Required |
Default |
---|---|---|---|---|
outdir |
Path of the output directory |
str |
yes |
null |
filenameprefix |
The filename prefix of each output file |
str |
yes |
null |
logfilenameprefix |
The filename prefix of the log file |
str |
yes |
null |
sourcename |
The name of the source under analysis |
str |
yes |
null |
userName |
The name of the user performing the analysis |
str |
yes |
null |
verboselvl |
0 ⇒ CRITICAL and WARNING messages are logged on the console.
1 ⇒ CRITICAL, WARNING and INFO messages are logged on the console.
2 ⇒ CRITICAL, WARNING, INFO and DEBUG messages are logged on the console
|
int |
no |
1 |
Section: ‘selection’¶
The temporal, spatial and spectral binning of the data can be customized using the configuration options of this section.
The center of the ROI (region of interest) is defined by explicit Galactic sky coordinates (glon and glat).
Option |
Description |
Type |
Default |
Required |
---|---|---|---|---|
emin |
Energy min in MeV |
int |
100 |
no |
emax |
Energy max in MeV |
int |
10000 |
no |
glat |
Center of the ROI (’latitude’ or ‘b’) |
float |
null |
yes |
glon |
Center of the ROI (’longitude’ or ‘l’) |
float |
null |
yes |
tmin |
Minimum time (in MJD or TT) |
float |
null |
yes |
tmax |
Maximum time (in MJD or TT) |
float |
null |
yes |
timetype |
The date format of tmin and tmax.
Possibile values: [‘MJD’, ‘TT’]
|
str |
null |
yes |
timelist |
it’s a path to a file containing a list of time intervals in TT
format to generate maps
integrated within a time window.
If specified, ‘tmin’ and ‘tmax’ are ignored.
|
str |
null |
no |
filtercode |
filtercode = 5 select G filtercode = 0 select G+L+S |
int |
5 |
no |
fovradmin |
fovradmin < fovradmax |
int |
0 |
no |
fovradmax |
fovradmax > fovradmin (dq = 0 is necessary for setting) |
int |
60 |
no |
albedorad |
albedo selection cut (dq = 0 is necessary for setting) |
int |
80 |
no |
dq |
Data quality selection filter.
A combination of fovradmax and albedorad.
Possible values are [0,1,2,3,4,5,6,7,8,9]
dq = 0 -> albedorad and fovradmax are free and they must always be specified in setOption
dq = 1 -> albedorad=80, fovradmax=60
dq = 2 -> albedorad=80, fovradmax=50
dq = 3 -> albedorad=90, fovradmax=60
dq = 4 -> albedorad=90, fovradmax=50
dq = 5 -> albedorad=100, fovradmax=50
dq = 6 -> albedorad=90, fovradmax=40
dq = 7 -> albedorad=100, fovradmax=40
dq = 8 -> albedorad=90, fovradmax=30
dq = 9 -> albedorad=100, fovradmax=30
|
int |
0 |
no |
phasecode |
Photon list selection parameter based
on the orbital phase. If ‘None’, the
automated selection is done following
the ‘phasecode’ rule
|
int |
null |
no |
Phasecode rule¶
phasecode = 2 -> spinning mode, SAA excluded with AC counts method.
phasecode = 6 -> spinning mode, SAA excluded according to the magnetic field intensity (old definition of SAA, defined by TPZ)
phasecode = 18 -> pointing mode, SAA and recovery exluded.
It is suggested to use phasecode = 2 for data taken in spinning mode.
def setPhaseCode(tmax)
if @phasecode == -1
if tmax.to_f >= 182692800.0
@phasecode = 6 #SPIN
else
@phasecode = 18 #POINTING
end
end
end
filtercode rule¶
A set of different on-board triggers enables the discrimination of background events (mainly cosmic rays in the AGILE Low Earth Orbit) from gamma-ray events. The data processing of the GRID events use an additional on-ground filters and provides a classification of each event:
P : events classified as a charged particle and rejected
G : events classified as gamma-ray photons. This is the most useful class for the analysis
S : events classified as single-track: this is a special class of events with no separation between the electron and positron tracks
L : limbo events, not clearly classified.
The events provided in the EVT files are of type G, S, and L. The AGILE team recommends to use the G class for scientific analysis. Only for gamma-ray bursts or other short transient events, and for pulsar timing analysis the G, S and L classes should be used together.
Section: ‘maps’¶
These options control the behaviour of the sky maps generation tools. The ‘energybin’ and ‘fovbinnumber’ options set the number of maps that are generated:
number of maps = number of energy bins * fovbinnumber
The ‘energybin’ option is a list of strings with the following format:
energybins:
- 100, 1000
- 1000, 3000
The ‘fovbinnumber’ option sets the number of bins between ‘fovradmin’ and ‘fovradmax’ as:
number of fov bins = (fovradmax-fovradmin)/fovbinnumber
Note
One map is generated for each possible combination between the ‘energybin’ (emin, emax) and the ‘fovbinnumber’ (fovmin, fovmax). The order of map generation is described by the following pseudocode:
Option |
Description |
Type |
Default |
Required |
---|---|---|---|---|
mapsize |
Width of the ROI in degrees |
float |
40 |
no |
useEDPmatrixforEXP |
Use the EDP matrix to generate the exposure map. |
boolean |
False |
no |
expstep |
Step size of the exposure map, if ‘None’ it depends by
round(1 / binsize, 2) (e.g. 0.3->3, 0.25->4, 0.1->10)
|
int |
None |
no |
spectralindex |
Spectral index of the exposure map |
float |
2.1 |
no |
timestep |
LOG file step size of exposure map (LOG file are at 0.1s) |
float |
160 |
no |
projtype |
Projection mode. Possible values: [’WCS’] |
str |
WCS |
no |
proj |
Spatial projection for WCS mode.
Possible values: [’ARC’, ‘AIT’]
|
str |
ARC |
no |
skytype |
gasmap:
0) SKY000-1 + SKY000-5,
1) gc_allsky maps + SKY000-5,
2) SKY000-5
3) SKY001 (old galcenter, binsize 0.1, full sky),
4) SKY002 (new galcenter, binsize 0.1, full sky)
|
int |
4 |
no |
binsize |
Spatial bin size in degrees |
float |
0.25 |
no |
energybin |
The enegy bins of analysis. A list of value. | To configure: | 1) directly in the yaml configuration file; | 2) Use the method e.g. ag.setOptions(energybins=[[100, 300], [500, 1000]]) | 3) Use the method ag.setOptionEnergybin(value) |
List<String> |
[100, 10000] |
no |
fovbinnumber |
Number of bins between fovradmin and fovradmax.
Dim = (fovradmax-fovradmin)/fovbinnumber
|
int |
1 |
no |
Section: ‘model’¶
The ‘galcoeff’ and ‘isocoeff’ options values can take the default value of null or they can be a a list of values separated by a comma. If they are set to null it means they are free to change.
model:
galcoeff: 0.8, 0.6, 0.5, 0.4
isocoeff: 8, 10, 12, 14
In this case, you should pay attention on how the sky maps are generated: the following example show which iso/gal coefficients are assigned to which map.
selection:
fovradmin: 0
fovradmax: 60
maps:
energybins:
- 100, 300
- 300, 1000
fovbinnumber: 2
model:
galcoeff: 0.8, 0.6, 0.5, 0.4
isocoeff: 8, 10, 12, 14
Option |
Description |
Type |
Default |
Required |
---|---|---|---|---|
modelfile |
A file name that contains point
sources, diffuse and isotropic components
|
string |
null |
yes |
galmode |
int |
1 |
no |
|
isomode |
int |
1 |
no |
|
galcoeff |
set into .maplist if >= 0 |
null, float or str |
null |
no |
isocoeff |
set into .maplist if >= 0 |
null, float or str |
null |
no |
emin_sources |
energy min of the modelfile |
int |
100 |
no |
emax_sources |
energy max of the modelfile |
int |
10000 |
no |
galcoeff and isocoeff¶
galcoeff and isocoeff are the coefficients for the Galactic and isotropic diffuse emission components respectively. The values may be fixed during the fitting process or some or all of them may be optimized by allowing them to vary. Agilepy allows to evaluate these coefficient and fix them or to keep these coefficient free.
Positive values are considered fixed, while negative values are free to vary starting from their absolute values. These coefficients are affected by the galmode and isomode coefficients described in the following section.
galmode and isomode¶
‘galmode’ and ‘isomode’ are integer values describing how the corresponding coefficients ‘galcoeff’ or ‘isocoeff’ found in all the lines of the maplist will be used:
Section: ‘mle’¶
The maximum likelihood estimation analysis is configured by the following options:
Option |
Description |
Type |
Default |
Required |
---|---|---|---|---|
ranal |
Radius of analysis |
float |
10 |
No |
ulcl |
Upper limit confidence level, expressed as sqrt(TS) |
float |
2 |
No |
loccl |
Source location contour confidence level (default 95 (%)confidence level). Possible values: [ 99, 95, 98, 50] |
int |
95 |
No |
fluxcorrection |
Correction of the flux taking into account the spectral model. Possible values: [0 (no correction), 1 (enable correction)]. |
int |
0 |
No |
Exp-ratio evaluation options¶
See details in this link.
Option |
Type |
Default |
Required |
Description |
---|---|---|---|---|
expratioevaluation |
bool |
yes |
none |
|
expratio_minthr |
float |
0 |
none |
|
expratio_maxthr |
float |
15 |
none |
|
expratio_size |
float |
10 |
none |
Section: ‘ap’¶
This section describes the configuration parameters for the Aperture Photometry analysis.
Option |
Description |
Type |
Required |
Default |
---|---|---|---|---|
radius |
The radius of analysis |
float |
no |
3 |
timeslot |
The size of the temporal bin |
int |
no |
3600 |
Section: ‘plot’¶
This section defines the plotting configuration.
Option |
Description |
Type |
Required |
Default |
---|---|---|---|---|
twocolumns |
The plot is adjusted to the size of a two column journal publication |
boolean |
False |
no |
Sources file¶
The sources can be defined using one of two different formats: xml document and text file.
The flux parameter estimates are relevant in the fitting process, as the sources are considered one by one, starting with the one with the brightest initial flux value, regardless of the order they are given in the source file.
Spectral models¶
A full energy band spectral fit of the data is performed with different spectral model. The spectral representations used in the BUILD25 are PL, exponential cut-off PL, super-exponential cut-off PL, and log parabola (LP). More details are reported in https://arxiv.org/abs/1903.06957
The PL spectral model is used for all sources that are not significantly curved and have low exposure,

where N0 is the prefactor and alpha is the index explicitly evaluated by the MLE method. Our MLE spectral fitting does not explicitly output the prefactor value, which is internally calculated by the numerical procedure. The majority of the AGILE sources are described by a PL.
The exponential cut-off PL spectral model (PC) is

where N0 is the prefactor, α is the index, and Ec is the cut-off energy. The values Ec and α are explicitly provided by the MLE method.
The super exponential cut-off PL spectral model (PS) is

where N0 is the prefactor, α is the first index, β the second index, and Ec is the cut-off energy. The parameters α, Ec, and β are explicitly provided by the MLE method.
The LP spectral model is

where N0 is the prefactor, Ec is the pivot energy, α is the first index, β the curvature. The parameters α, Ec, and β are explicitly provided by the MLE method.
The selection of curved spectra followed the acceptance criteria described in bulgarelli19. Briefly, a source is considered significantly curved if T Scurved > 16, where T Scurved = 2 × (log L(curved spectrum)−log L(power law), where L is the likelihood function obtained changing only the spectral representation of that source and refitting all free parameters.
Source library format (xml document)¶
<?xml version="1.0" ?>
<source_library title="source library">
<!-- Point Sources -->
<source name="2AGLJ2202+4214" type="PointSource">
<spectrum type="PowerLaw">
<parameter name="flux" free="1" value="7.45398e-08"/>
<parameter name="index" free="1" scale="-1.0" value="1.96903" min="0.5" max="5"/>
</spectrum>
<spatialModel type="PointSource" location_limit="0">
<parameter name="pos" value="(92.4102, -10.3946)" free="0" />
</spatialModel>
</source>
<source name="2AGLJ0007+7308" type="PointSource">
<spectrum type="PLExpCutoff">
<parameter name="flux" free="1" value="41.6072e-08"/>
<parameter name="index" free="1" scale="-1.0" value="1.29082" min="0.5" max="5"/>
<parameter name="cutoffEnergy" free="1" scale="-1.0" value="2003.9" min="20" max="10000"/>
</spectrum>
<spatialModel type="PointSource" location_limit="0">
<parameter name="pos" value="(119.677, 10.544)" free="0" />
</spatialModel>
</source>
<source name="2AGLJ0835-4514" type="PointSource">
<spectrum type="PLSuperExpCutoff">
<parameter name="flux" free="1" value="969.539e-08"/>
<parameter name="index1" free="1" scale="-1.0" value="1.71345" min="0.5" max="5"/>
<parameter name="cutoffEnergy" free="1" value="3913.06" min="20" max="10000"/>
<parameter name="index2" free="1" value="1.3477" min="0" max="100"/>
</spectrum>
<spatialModel type="PointSource" location_limit="0">
<parameter name="pos" value="(263.585, -2.84083)" free="0" />
</spatialModel>
</source>
<source name="2AGLJ1801-2334" type="PointSource">
<spectrum type="LogParabola">
<parameter name="flux" free="1" value="35.79e-08"/>
<parameter name="index" free="1" scale="-1.0" value="3.37991" min="1" max="4"/>
<parameter name="pivotEnergy" free="1" scale="-1.0" value="2935.07" min="500" max="3000"/>
<parameter name="curvature" free="1" scale="-1.0" value="0.682363" min="0.1" max="3"/>
</spectrum>
<spatialModel type="PointSource" location_limit="0">
<parameter name="pos" value="(6.16978, -0.0676943)" free="1" />
</spatialModel>
</source>
</source_library>
Working with sources¶
The Source abstraction¶
The main abstraction of Agilepy is the Source class. It is described by several parameters, some of which can be free to vary, and they are changed by the mle() analysis.
The set of the parameters describing the source can vary, depending on the spectrum and spatial model types of the source.
The different types of sources are described here.
How to load or add new sources¶
In order to perform a scientific analysis with Agilpy, at least one Source model must be loaded. There are several ways to do that.
The loadSourcesFromCatalog(catalogName, rangeDist=0, inf, show=False) allows to load a source catalog, while filtering the sources by their distance (degree) from the l,b position provided within the configuration file.
sources = ag.loadSourcesFromCatalog('2AGL', rangeDist=(0, 10))
The loadSourcesFromFile(sourcesFilepath, rangeDist=0, inf, show=False) loads the sources, reading their model from a file.
The addSource(sourceName, sourceDict) method allows the user to define on the fly a source model with a python dictionary. Check the tutorial notebooks for an example.
How to select Source objects¶
The sources can be selected via the the selectSources(selection, show=False) method. The “selection” argument supports either lambda functions and boolean expression strings. The user can call selectSources (with show=True) to show the source description
source = ag.selectSources('name == "2AGLJ2254+1609"', show=False).pop()
print(source)
-----------------------------------------------------------
Source name: 2AGLJ2254+1609 (PointSource)
* Free parameters: flux
* Initial source parameters: (PowerLaw)
- flux(ph/cm2s): 7.50937e-07
- index: 2.20942
- Source position: (86.1236, -38.1824) (l,b)
- Distance from map center: 0.011 deg
-----------------------------------------------------------
Other examples:
sources = ag.selectSources('name == "PKS1510-089"', show=False)
sources = ag.selectSources('flux > 0', show=False)
sources = ag.selectSources(lambda name, sqrtTS: name == "2AGLJ2021+4029" AND sqrtTS> 0, show=False)
How to let the source’s parameters to vary¶
In order to free or fix a sources’ parameter, the user can rely on the freeSources(selection, parameterName, free, show=False) method. The “selection” argument is used like in selectSources, so you can free a parameter of multiple sources at once.
aganalysis.freeSources(lambda name, dist, flux : Name == "2AGLJ2021+4029" AND dist > 0 AND flux > 0, "flux", True)
ag.freeSources('name == "2AGLJ1513-0905"', "index", True, show=True)
Check the api documentation or the tutorial notebooks for additional examples.
How to check which source’s parameters are free to vary¶
The user can obtain this information by printing the Source object or calling the getFreeParams() method of the Source object.
print(source.getFreeParams())
['flux']
The “multi” description of a Source object¶
If the user performs an mle analysis, the Source object will contain also the analysis results.
print(source)
-----------------------------------------------------------
Source name: 2AGLJ2254+1609 (PointSource) => sqrt(ts): 10.2226
* Free parameters: flux index
* Initial source parameters: (PowerLaw)
- flux(ph/cm2s): 7.50937e-07
- index: 2.20942
- Source position: (86.1236, -38.1824) (l,b)
- Distance from map center: 0.011 deg
* Last MLE analysis:
- flux(ph/cm2s): 8.68363e-06 +/- 1.62474e-06
- index: 2.51001 +/- 0.173795
- upper limit(ph/cm2s): 1.13967e-05
- ergLog(erg/cm2s): 1.39217e-09 +/- 2.6048e-10
- galCoeff: [0.7, 0.7, 0.7, 0.7, 0.7]
- isoCoeff: [5.24757, 3.14662, 0.953512, 1.59944e-10, 0.557554] +/- [0.519283, 0.209362, 0.051478, 0.000108147, 0.000720814]
- exposure(cm2s): 13672200.0
- exp-ratio: 0.0
- L_peak: 86.1236
- B_peak: -38.1824
- Distance from start pos: 0.0
- position:
- L: -1.0
- B: -1.0
- Distance from start pos: -1.0
- radius of circle: -1.0
- ellipse:
- a: -1.0
- b: -1.0
- phi: -1.0
-----------------------------------------------------------
The values L_peak and B_peak set to the initial values in the source location is fixed. If it is allowed to vary then they are set to the position for which the TS is maximized. If a confidence contour was found, the parameters of the “ellipse” section describe the best-fit ellipse of the contour, described in detail below. The counts and fluxes are provided, as well as their symmetric, positive, and negative errors if the flux is allowed to vary. For convenience, the exposure of the source, used to calculate the source counts from the flux, is also provided. Finally, the spectral index and its error, or the other spectral parameters, if applicable, are provided.
How to manually inspect source’s attributes¶
The user can rely on a getter method get(sourceAttribute) method.
print(source.get("cutoffEnergy"))
print(source.get("index"))
print(source.get("pos"))
print(source.get("dist"))
print(source.get("locationLimit"))
print(source.get("multiFlux"))
How to manually change a source’s attributes¶
The user can rely on a setter method set(sourceAttribute) method.
source.set("index2", 1.34774)
The setAttributes() method allows to change the following attributes: value, free, scale, min, max, locationLimit. Example:
source.spectrum.cutoffEnergy.setAttributes(min=3000, max=5000)
In order to change the position of a source, the user can rely on the updateSourcePosition(sourceName, glon, glat) method.
Products¶
Sky maps¶
‘.cts.gz’ file¶
Counts maps are generated by the procedure AG_ctsmapgen embedded into Agilpy.
AG_ctsmapgen reads the event files listed in the event file index (see “AGILE data” section), bins the counts between tmin and tmax, and outputs a FITS image file. The image is a two-dimensional array in the ARC or AIT projection. The projection, size and resolution, the center and rotation of the map in Galactic coordinates, tmin, tmax, emin, and emax, along with various integration parameters (fovrad, fovradmin, albrad, phasecode) are managed by Agilepy.
The parameters are described in the “Configuration file” section.

‘.exp.gz’ file¶
Exposure maps are generated by the procedure AG_expmapgen embedded into Agilpy.
The task AG_expmapgen reads the log files listed in the LOG index (see “AGILE data” section), integrates the exposure between tmin and tmax, and outputs a FITS exposure image file. The image is a two-dimensional array in either the ARC or AIT projection. The projection, size and resolution, and center and rotation of the map in Galactic coordinates, tmin, tmax, emin, emax, and index file are managed by Agilepy, along with various integration parameters (fovrad, albrad, y tol, roll tol, earth tol, phasecode), and an interpolation step size (binstep). The interpolation procedure is a linear interpolation method in which only one bin each N is calculated (where N is the step size parameter). For a bin size of 0.5 deg or 0.25 deg with a step size of N = 4 it is possible to get a good approximation of the exposure map.
The parameters are described in the “Configuration file” section.

‘.gas.gz’ file¶
Diffuse emission maps are generated by the procedure AG_gasmapgen embedded into Agilpy. AG_gasmapgen reads an exposure map produced by AG_expmapgen and the master diffuse emission map and outputs a FITS image file, in the same format as the exposure map, in which each pixel contains the diffuse emission in that pixel. The image is a square array in the ARC projection. The diffuse emission map contain models of the diffuse emission convolved with the energy-dependent point spread function and combined into predefined observed energy ranges according to the appropriate energy dispersion function for G events using the FM3.119 background filter. The diffuse emission map automatically selected by Agilepy based on the energy range of the analysis; e.g. if the analysis is performed between 100 MeV and 50 GeV, Agilepy select the file 100_50000.0.1.SFMG_H0025.conv.sky.gz. The first number in the file name is the minimum energy and the second number is the maximum energy, followed by the resolution of the maps (0.1), the background event rejection filter (FM3.119) and the instrument response functions (IRFs).

‘.int.gz’ file¶
Intensity maps are generated by the procedure AG_intmapgen embedded into Agilepy. AG_intmapgen reads an exposure map produced by AG_expmapgen and a counts map produced by AG_ctsmapgen and outputs a FITS image file, in the same format as the counts map, in which each pixel contains the intensity in that pixel. The image is a square array in the ARC projection. The two input maps should have been produced using the same set of parameters. The intensity map is not used in scientific analysis; it is useful solely as a visualization tool.

Result of the Maximum Likelihood Estimator¶
Agilepy shows a high-level view of the results of the maximum likelihood estimator. See this link for more details.
The details of the output of the science tool AG_multi that performs the likelihood procedure is still accessible. This section describe the output of the AG_multi science tool, that performs a Maximum Likelyhood Estimator analysis to find the best position, flux and spectral parameters of a list of sources given set of count maps.
Confidence Contour files¶
If a confidence contour was found, the parameters on the following line describe the best-fit ellipse of the contour, described in detail below.
If source location was requested for a given source and a source location contour was found, then three additional files are generated for that source. These files are written using galactic coordinates in degrees and can be loaded by applications such as ds9 and overlaid on the maps provided as input to AG_multi to visualize the source location contours. One of the three files, with extension .con, contains the source contour as found by the ROOT functions, expressed as a list of galactic coordinates, one point per line, where the last line is a repetition of the first. It may depict any shape. The other two files describe the ellipse that best fits the contour. One has extension .ellipse.con and represents the ellipse as a contour in a format analogous to that of the .con file. The other has extension .reg and describes same ellipse by its axes and orientation.
Determination of the ellipse. If AG_multi was able to find a source contour, an ellipse is fit to the contour. The source contour is a list of points which defines a polygon by connecting each point sequentially. The value of Radius found in the HTML output is the radius in degrees of a circle with the same area as the polygon. AG_multi determines the ellipse which best fits the contour. This ellipse will have the same area as the polygon, and the distance between each contour point and the intersection between the ellipse and the line connecting that point to the centre will be minimized. The ellipse is completely described by three parameters: the two axes and the rotation (in degrees) of the first axis around the centre, as expected by the ds9 application. If the ellipse is a circle, its axes will both be equal to the Radius found in the HTML output. The ellipse is described by two files that are readable by ds9: one is a .reg file which contains the centre, the axes and the rotation of the ellipse, while the other describes the same ellipse as a list of points in galactic coordinates, thus using the same syntax of a contour file, and has extension .ellipse.con. This is an example of ellipse .reg file:
ExpRatio¶
Owing to the non-homogeneous sky coverage of the AGILE observations, it is possible that sources lie near the borders of certain pointings. In order to have an unbiased estimate of the coeffcients of the Galactic diffuse emission and isotropic background that could lead to to an incorrect evaluation of the flux and position of the source, exposure uniformity within the region of the analysis is required. We applied a specific check to verify the uniformity of the exposure within the 10-degree radius of the AGILE MLE analysis centred at each source candidate position, over the considered timescale. The fraction of pixels of the exposure map within the region of analysis having a value below a pre-defined threshold was calculated, and if it was more than 10% the region was considered unreliable and the candidate was discarded. The exposure threshold value is evaluated by calculating the mean exposure of the observation over the full FoV area and comparing this exposure with the values of some reference good exposures.
The parameters expratioevaluation, expratio_minthr, expratio_maxthr, expratio_size described here.
Light curves¶
AGILE-GRID light curves can be created in two different ways:
using a maximum likelihood estimator analysis
using aperture photometry.
The likelihood analysis reach better sensitivity, more accurate flux measurement, better evaluation of the backgrounds and can work with a detailed source models where more sources can be considered at the same time.
Aperture photometry provides a raw measure of the flux of a sigle source and is less computing demanding.
The likelihood light curve file contains the results of the generation of a light curve. The columns described are the following sections.
Time of the analysis in MJD:
time_start_mjd: time start (MJD)
time_end_mjd: time end (MJD)
Result of the analysis of the single source:
sqrt(ts): the square root of the Test Statistic value of the results of the maximum likelihood estimator (mle)
flux (ph/cm2/s/sr)
flux_err (ph/cm2/s/sr)
flux_ul (ph/cm2/s/sr)
gal: the value of the galactic diffuse emission (gal) parameter
gal_error: the error of the galactic diffuse emission (gal) parameter
iso: the value of the isotropic emission (iso) parameter
iso_error: the error of the isotropic emission (iso) parameter
(l_peak, b_peak): position in Galactic coordinate (l_peak, b_peak): peak coordinates. If it is allowed to vary then they are set to the position for which the TS is maximized.
dist_peak: distance between current l_peak, b_peak and previous position
(l, b): position in Galactic coordinate evaluated by mle with the determination of the 95% confidence level elliptical confidence region
r: radius of 95% c.l. circular confidence region, deg. Statistical error only
ell_dist: the distance between (l,b) and the initial position
a: the semimajer axis of the elliptical confidence region
b: the semiminor axis of the elliptical confidence region
phi: rotation of the elliptical confidence region
exposure
ExpRatio: see above section
counts
counts_err
Index
Index_Err
Par2
Par2_Err
Par3
Par3_Err
Erglog
Erglog_Err
Erglog_UL
Time of the analysis in UTC and TT:
time_start_utc
time_end_utc
time_start_tt
time_end_tt
The following are the initial parameters of the analysis of the source:
Fix: initial fixflag
index: initial spectral index
ULConfidenceLevel
SrcLocConfLevel
start_l: initial Galactic coordinate l
start_b: initial Galactic coordinate b
start_flux: inital flux for the MLKE
typefun: type of spectral model
par2: initial value of par2
par3: initial value of par3
The following are the parameters of the MLE analysis:
galmode2
galmode2fit
isomode2
isomode2fit
edpcor
fluxcor
integratortype
expratioEval
expratio_minthr
expratio_maxthr
expratio_size
Parameters of the maps:
Emin
emax
fovmin
fovmax
albedo
binsize
expstep
phasecode
Technical results of the fitting:
fit_cts
fit_fitstatus0
fit_fcn0
fit_edm0
fit_nvpar0
fit_nparx0
fit_iter0
fit_fitstatus1
fit_fcn1
fit_edm1
fit_nvpar1
fit_nparx1
fit_iter1
fit_Likelihood1
time_start_mjd time_end_mjd sqrt(ts) flux flux_err flux_ul gal gal_error iso iso_error l_peak b_peak dist_peak l b r ell_dist a b phi exposure ExpRatio counts counts_err Index Index_Err Par2 Par2_Err Par3 Par3_Err Erglog Erglog_Err Erglog_UL time_start_utc time_end_utc time_start_tt time_end_tt Fix index ULConfidenceLevel SrcLocConfLevel start_l start_b start_flux typefun par2 par3 galmode2 galmode2fit isomode2 isomode2fit edpcor fluxcor integratortype expratioEval expratio_minthr expratio_maxthr expratio_size Emin emax fovmin fovmax albedo binsize expstep phasecode fit_cts fit_fitstatus0 fit_fcn0 fit_edm0 fit_nvpar0 fit_nparx0 fit_iter0 fit_fitstatus1 fit_fcn1 fit_edm1 fit_nvpar1 fit_nparx1 fit_iter1 fit_Likelihood1
58026.49921296296 58027.49921296296 7.3538 944.812e-08 213.193e-08 1419.5e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3718660.0 1.99923 35.1344 7.92794 1.71345 0 3913.06 0 1.34774 0 1.67967e-09 3.7901e-10 2.52355e-09 2017-09-30T11:58:52.000 2017-10-01T11:58:52.000 433857532.0 433943932.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 92 -1 344.873 0.5 0 52 3 0 317.834 5.85172e-17 1 52 3 404.289
58027.49921296296 58028.49921296296 8.87831 1055.24e-08 211.709e-08 1524.18e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3843970.0 3.11419 40.5633 8.13802 1.71345 0 3913.06 0 1.34774 0 1.87599e-09 3.76372e-10 2.70965e-09 2017-10-01T11:58:52.000 2017-10-02T11:58:52.000 433943932.0 434030332.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 91 -1 341.314 0.5 0 52 3 0 301.902 5.88891e-17 1 52 3 391.515
58028.49921296296 58029.49921296296 7.31495 820.826e-08 198.261e-08 1266.82e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3788250.0 1.99923 31.095 7.51063 1.71345 0 3913.06 0 1.34774 0 1.45925e-09 3.52464e-10 2.25212e-09 2017-10-02T11:58:52.000 2017-10-03T11:58:52.000 434030332.0 434116732.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 96 -1 357.958 0.5 0 52 3 0 331.204 1.01185e-16 1 52 3 423.045
58029.49921296296 58030.49921296296 6.78978 840.67e-08 208.19e-08 1307.27e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3806190.0 3.11419 31.9975 7.92411 1.71345 0 3913.06 0 1.34774 0 1.49452e-09 3.70116e-10 2.32404e-09 2017-10-03T11:58:52.000 2017-10-04T11:58:52.000 434116732.0 434203132.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 110 -1 404.846 0.5 0 52 3 0 381.795 1.35163e-15 1 52 3 486.25
58030.49921296296 58031.49921296296 7.63221 820.4e-08 190.928e-08 1249.81e-08 0.7,0.7 0,0 4.08416,3.84041 0,0 263.638 -2.85605 0 -1 -1 -1 -1 -1 -1 -1 3793810.0 2.9604 31.1244 7.24344 1.71345 0 3913.06 0 1.34774 0 1.45849e-09 3.39428e-10 2.22189e-09 2017-10-04T11:58:52.000 2017-10-05T11:58:52.000 434203132.0 434289532.0 1 1.71345 2 5.99147 263.638 -2.85605 8.98066e-06 2 3913.06 1.34774 0 0 0 0 0.75 0 1 1 0 15 10 100,300 300,1000 0,0 60,60 80 0.4 0 6 97 -1 365.286 0.5 0 52 3 0 336.161 1.10047e-16 1 52 3 426.229
Advanced configuration¶
Selection¶
Option
Type
Default
Required
Description
lonpole
int
180
no
lpointing
float
TBD
no
l in Galactic coordinates of the center of the pointing during the ‘pointing period’
bpointing
float
TBD
no
b in Galactic coordinates of the center of the pointing during the ‘pointing period’
maplistgen
string
TBD
no
filename of a file for expmapgen with mapspec.fovradmin >> mapspec.fovradmax >> mapspec.emin >> mapspec.emax >> mapspec.index
Maps¶
Option
Type
Default
Required
Description
offaxisangle
float
30
No
Off axis pointing for mle analysis. Values are between 0 and 60 deg. Agilepy into .maplist
Model¶
Option |
Type |
Default |
Required |
Description |
---|---|---|---|---|
galmode2 |
int |
0 |
No |
Fix the gal parameters using the internal analysis of the MLE.
0) gal0 and gal1 are kept free
1) set gal0 for L0 and gal1 for L1
2) set gal0 for L0 and L1
3) set gal1 for L0 and L1
4) set gal1 - gal1err for L0 and L1
5) set gal1 + gal1err for L0 and L1
|
galmode2fit |
int |
0 |
No |
Fix the gal parameters for different energy bins performing a linear fit of gal values evaluated using the internal analysis of the MLE.
0) do not fit
1) pol0 fit
2) powerlaw fit
|
isomode2 |
int |
0 |
No |
Fix the iso parameters using the internal analysis of the MLE.
0) none
1) set iso0 for L0 and gal1 for L1
2) set iso0 for L0 and L1
3) set iso1 for L0 and L1
4) set iso1 - iso1err for L0 and L1
5) set iso1 + iso1err for L0 and L1
|
isomode2fit |
int |
0 |
No |
Fix the iso parameters for different energy bins performing a linear fit of iso values evaluated using the internal analysis of the MLE.
0) do not fit
1) pol0 fit
2) powerlaw fit
|
mle¶
Advanced options for optimizer¶
Option
Type
Default
Required
Description
minimizertype
string
Minuit
No
Use Minuit if position is free. For other values see below.
minimizeralg
string
Migrad
No
For other values see below.
minimizerdefstrategy
int
2
No
Default 2 for Minuit. For other values see below.
mindefaulttolerance
float
0.01
No
See below.
integratortype
int
1
No
1 gauss2 gaussht3 gausslefevre4 gausslefevrehtcontourpoints
int
40
No
Number of points to determine the contour (0-400).
minimizertype = Minuit (library libMinuit). Old version of Minuit, based on the TMinuit class. The list of possible algorithms (minimizeralg) are:1) Migrad (default one)2) Simplex3) Minimize (it is a combination of Migrad and Simplex)4) MigradImproved5) Scan6) Seekminimizertype = Minuit2 (library libMinuit2). New C++ version of Minuit. The list of the possible algorithms (minimizeralg) :1) Migrad (default)2) Simplex3) Minimize4) Scanminimizertype = Fumili . This is the same algorithm of TFumili, but implemented in the Minuit2 library.
minimizertype = GSLMultiMin (library libMathMore). Minimizer based on the Multidimensional Minimization routines of the Gnu Scientific Library (GSL). The list of available algorithms (minimizeralg) is | 1) BFGS2 (default) : second version of the vector Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm; | 2) BFGS : old version of the vector Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm; | 3) ConjugateFR : Fletcher-Reeves conjugate gradient algorithm; | 4) ConjugatePR : Polak-Ribiere conjugate gradient algorithm; | 5) SteepestDescent: steepest descent algorithm;
#*** * GSLMultiFit (library libMathMore). Minimizer based on the Non-Linear Least-Square routines of GSL. This minimizer can be used only for least-square fits.#*** * GSLSimAn (library libMathMore). Minimizer based on simulated annealing.#*** * Genetic (library libGenetic). Genetic minimizer based on an algorithm implemented in the TMVA package.Each minimizer can be configured using the ROOT::Math::MinimizerOptions class. The list of possible option that can be set are:
minimizertype:Minimizer type (MinimizerOptions::SetMinimizerType(const char * )) .* Print Level (MinimizerOptions::SetPrintLevel(int )) to set the verbose printing level (default is 0).mindefaulttolerance:* Tolerance (MinimizerOptions::SetTolerance(double )) tolerance used to control the iterations.* Precision (MinimizerOptions::SetTolerance(double )). Precision value in the evaluation of the minimization function. Default is numerical double precision.
Maximum number of function calls (MinimizerOptions::SetMaxFunctionCalls(int )).
Maximum number of iterations (MinimizerOptions::SetMaxIterations(int )). Note that this is not used by Minuit. FCN Upper value for Error Definition (MinimizerOptions::SetMaxIterations(int )). Value in the minimization function used to compute the parameter errors. The default is to get the uncertainties at the 68% CL is a value of 1 for a chi-squared function minimization and 0.5 for a log-likelihood function.
minimizerdefstrategy:* Strategy (MinimizerOptions::SetStrategy(int )), minimization strategy used. For each minimization strategy Minuit uses different configuration parameters (e.g. different requirements in computing derivatives, computing full Hessian (strategy = 2) or an approximate version. The default is a value of 1. In this case the full Hessian matrix is computed only after the minimization.
Advanced options for internal corrections¶
Option |
Type |
Default |
Required |
Description |
---|---|---|---|---|
edpcorrection |
float |
0.75 |
No |
Perform a flux correction based on EDP evaluation for highest energy channels. Default 0.75, otherwise any value between 0 and 1. EDP correction is enabled only for E>1000 MeV and if fluxcorrection=1, and only for point sources. flux = flux * edpcorrection
|
fluxcorrection |
int |
0 |
No |
Perform a flux correction of the flux using the source spectral model and considering that the exposure is calculated with a Power Law with spectral index of 2.1.
0) no correction
1) Flux calculation correction for spectral shape in output
2) correction in input and output
|
Analysis API¶
AGBaseAnalysis¶
- class core.AGBaseAnalysis.AGBaseAnalysis(configurationFilePath)[source]¶
-
- getAnalysisDir()[source]¶
It returns the path of the output directory
- Returns
the path of the output directory
- Return type
path (str)
- deleteAnalysisDir()[source]¶
It deletes the output directory where all the products of the analysis are written.
Args:
- Returns
True if the directory is succesfully deleted, False otherwise.
- setOptions(**kwargs)[source]¶
It updates configuration options specifying one or more key=value pairs at once.
- Parameters
kwargs – key-values pairs, separated by a comma.
- Returns
None
- Raises
ConfigFileOptionTypeError – if the type of the option value is not wrong.
ConfigurationsNotValidError – if the values are not coherent with the configuration.
CannotSetHiddenOptionError – if the option is hidden.
OptionNotFoundInConfigFileError – if the option is not found.
Note
The
config
attribute is initialized by reading the corresponding yaml configuration file, loading its contents in memory. Updating the values held by this object will not affect the original values written on disk.Example
>>> aganalysis.setOptions(mapsize=60, binsize=0.5) True
AGAnalysis¶
- class api.AGAnalysis.AGAnalysis(configurationFilePath, sourcesFilePath=None)[source]¶
This class contains the high-level API to run scientific analysis, data visualization and some utility methods.
This constructor of this class requires a
yaml configuration file
.- __init__(configurationFilePath, sourcesFilePath=None)[source]¶
AGAnalysis constructor.
- Parameters
configurationFilePath (str) – a relative or absolute path to the yaml configuration file.
sourcesFilePath (str, optional) – a relative or absolute path to a file containing the description of the sources. Defaults to None. Three different types of formats are supported: AGILE format (.txt), XML format (.xml) and AGILE catalog files (.multi).
- Raises
AGILENotFoundError – if the AGILE environment variable is not set.
PFILESNotFoundError – if the PFILES environment variable is not set.
Example
>>> from agilepy.api import AGAnalysis >>> aganalysis = AGAnalysis('agconfig.yaml', sourcesFilePath='sources.xml')
- static getConfiguration(confFilePath, userName, sourceName, tmin, tmax, timetype, glon, glat, outputDir, verboselvl, evtfile='/AGILE_PROC3/FM3.119_ASDC2/INDEX/EVT.index', logfile='/AGILE_PROC3/DATA_ASDC2/INDEX/LOG.log.index', userestapi=True, datapath=None)[source]¶
Utility method to create a configuration file.
- Parameters
confFilePath (str) – the path and filename of the configuration file that is going to be created.
userName (str) – the username of who is running the software.
sourceName (str) – the name of the source.
tmin (float) – the start time of the analysis.
tmax (float) – the stop time of the analysis.
timetype (str) – timetype format possible values: “TT” or “MJD”
glon (float) – the galactic longitude (L) of the analysis.
glat (float) – the galactic latitude (B) of the analysis.
outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’
verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug
evtfile (str, optional [default=/AGILE_PROC3/FM3.119_ASDC2/INDEX/EVT.index]) – the index file to be used for event data if userestapi is set to False. The time range starts from 107092735 TT, 54244.49924768 MJD, 2007-05-24 11:58:55 UTC
logfile (str, optional [default=/AGILE_PROC3/DATA_ASDC2/INDEX/LOG.log.index]) – the index file to be used for log data if userestapi is set to False. The time range starts from 107092735 TT, 54244.49924768 MJD, 2007-05-24 11:58:55 UTC
userestapi (bool, optional [default=True]) – If True, the SSDC REST API will be used to download missing data.
datapath (str, optional [default=None]) – Datapath to download AGILE data if userestapi is set to True. Index files will be generated into this path.
- Returns
None
- loadSourcesFromCatalog(catalogName, rangeDist=(0, inf), show=False)[source]¶
It loads the sources from a catalog.
You can also specify a rangeDist argument to filter out the sources which distance from (glon, glat) is not in the rangeDist interval.
If the catalog is 2AGL and if the energy range (emin, emax) specified by the user in the configuration file is different from the catalog’s energy range, the flux of every source will be scaled.
- Parameters
catalogName (str) – the catalog name (2AGL, 4FGL).
rangeDist (tuple, optional) – a interval (min, max) of distances (degree). It defaults to (0, +inf).
- Raises
FileNotFoundError – if the catalog is not supported.
SourceModelFormatNotSupported – if the input file format is not supported.
SourcesFileLoadingError – if any error occurs during the parsing of the sources file.
- Returns
The List of sources that have been succesfully loaded into the SourcesLibrary.
- loadSourcesFromFile(sourcesFilepath, rangeDist=(0, inf), show=False)[source]¶
It loads the sources, reading them from a file. Three different types of format are supported: AGILE format (.txt), XML format (.xml) and AGILE catalog files (.multi).
You can also specify a rangeDist argument to filter out the sources which distance from (glon, glat) is not in the rangeDist interval.
- Parameters
sourcesFilePath (str) – a relative or absolute path to a file containing the description of the sources. rangeDist (tuple): a interval (min, max) of distances (degree)
- Raises
SourceModelFormatNotSupported – if the input file format is not supported.
SourcesFileLoadingError – if any error occurs during the parsing of the sources file.
- Returns
The List of sources that have been succesfully loaded into the SourcesLibrary.
- selectSources(selection, show=False)[source]¶
It returns the sources matching the selection criteria from the
sourcesLibrary
.The sources can be selected with the
selection
argument, supporting eitherlambda functions
andboolean expression strings
.The selection criteria can be expressed using the following
Source
class’s parameters:name: the unique code identifying the source.
dist: the distance of the source from the center of the maps.
flux: the flux value.
sqrtts: the radix square of the ts.
Warning
The sqrtts parameter is available only after the maximum likelihood estimation analysis is performed.
- Parameters
selection (lambda or str) – a lambda function or a boolean expression string specifying the selection criteria.
quiet (boolean) (optional default=False) – if quiet is True, the method will not console log the selected sources.
- Returns
List of sources.
- freeSources(selection, parameterName, free, show=False)[source]¶
It can set to True or False a parameter’s
free
attribute of one or more source.Example of source model:
<source name="2AGLJ2021+4029" type="PointSource"> <spectrum type="PLExpCutoff"> <parameter name="flux" free="1" value="119.3e-08"/> <parameter name="index" free="0" scale="-1.0" value="1.75" min="0.5" max="5"/> <parameter name="cutoffEnergy" free="0" value="3307.63" min="20" max="10000"/> </spectrum> <spatialModel type="PointSource" location_limit="0"> <parameter name="pos" value="(78.2375, 2.12298)" free="0" /> </spatialModel> </source>
The supported
parameterName
are:flux
index
index1
index2
cutoffEnergy
pivotEnergy
curvature
index2
pos
- Parameters
selection (lambda or str) – a lambda function or a boolean expression string specifying the selection criteria.
parameterName (str) – the
free
attribute of this parameter will be updated.free (bool) – the boolean value used.
- Returns
The list of the sources matching the selection criteria.
Note
The
sourcesLibrary
attribute is initialized by reading the corresponding xml descriptor file, loading its contents in memory. Updating the values held by this object will not affect the original values written on disk.Example
The following calls are equivalent and they set to True the
free
attribute of the “Flux” parameter for all the sources named “2AGLJ2021+4029” which distance from the map center is greater than zero and which flux value is greater than zero.>>> aganalysis.freeSources(lambda name, dist, flux : Name == "2AGLJ2021+4029" AND dist > 0 AND flux > 0, "flux", True) [..]
>>> aganalysis.freeSources('name == "2AGLJ2021+4029" AND dist > 0 AND flux > 0', "flux", True) [..]
- fixSource(source)[source]¶
Set to False all freeable params of a source.
- Parameters
sourceName (Source) – the source object.
- Returns
True if at least one parameter of the source changes, False otherwise.
- addSource(sourceName, sourceDict)[source]¶
It adds a new source in the
Sources Library
. You can add a source, passing a dictionary containing the source’s data. Some keys are required, other are optional. Here’s an example:newSourceDict = { "glon" : 6.16978, "glat": -0.0676943, "spectrumType" : "LogParabola", "flux": 35.79e-08, "curvature": 0.682363 } newSource = aganalysis.addSource("2AGLJ1801-2334", newSourceDict)
- Parameters
sourceName (str) – the name of the source
sourceDict (dict) – a dictionary containing the source’s data.
- Raises
SourceParamNotFoundError – if
sourceName
is None or empty, OR thesourceDict
input dictionary does not contain at least one of the following keys: [“glon”, “glat”,”spectrumType”].- Returns
The source object if it is been loaded. None, otherwise.
- deleteSources(selection, show=False)[source]¶
It deletes the sources matching the selection criteria from the
sourcesLibrary
.- Parameters
selection (lambda or str) – a lambda function or a boolean expression string specifying the selection criteria.
- Returns
The list containing the deleted sources.
- updateSourcePosition(sourceName, glon, glat)[source]¶
It updates a source (l,b) position parameters. The ‘dist’ value will also be updated.
- Parameters
sourceName (str) – the name of the source
glon (float) – galactic longitude.
glat (float) – galactic latitude.
- Raises
SourceNotFound – if the source has not been loaded into the SourcesLibrary.
ValueError – if glon or glat values are out of range.
- Returns
True if the position changes, False otherwise.
- writeSourcesOnFile(outfileNamePrefix, fileFormat, sources=None)[source]¶
It writes on file the list of sources loaded into the SourceLibrary. The supported formats (‘txt’ AND ‘xml’) are described here: Sources file.
- Parameters
outfileNamePrefix (str) – the name of the output file (without the extension).
fileFormat (str) – Possible values: [‘txt’, ‘xml’, ‘reg’].
sources (List<Source>) – a list of Source objects. If is is None, every loaded source will be written on file.
- Raises
SourceModelFormatNotSupported – if the file format is not supported.
- Returns
Path to the file
- setOptionTimeMJD(tmin, tmax)[source]¶
It performs a time conversion MJD -> TT and set new values using setOptions functions
- Parameters
tmin (float) – tmin in MJD
tmax (float) – tmax in MJD
- setOptionEnergybin(value)[source]¶
An useful utility method that maps a value in a specific energybin and it calls the setOptions function.
- Parameters
Value (int) –
A value between 0 and 5
0: [[100, 10000]]
1: [[100, 50000]]
2: [[100, 300], [300, 1000], [1000, 3000], [3000, 10000]]
3: [[100, 300], [300, 1000], [1000, 3000], [3000, 10000], [10000, 50000]]
4: [[50, 100], [100, 300], [300, 1000], [1000, 3000], [3000, 10000]]
5: [[50, 100], [100, 300], [300, 1000], [1000, 3000], [3000, 10000], [10000, 50000]]
- Raises
ValueOutOfRange – if the value is not between 0 and 5.
- generateMaps(config=None, maplistObj=None, dqdmOff=False)[source]¶
It generates (one or more) counts, exposure, gas and int maps and a
maplist file
.The method’s behaviour varies according to several configuration options (see docs Configuration file).
Note
It resets the configuration options to their original values before exiting.
- Returns
The absolute path to the generated
maplist file
.- Raises
ScienceToolInputArgMissing – if not all the required configuration options have been set.
Example
>>> aganalysis.generateMaps() /home/rt/agilepy/output/testcase.maplist4
- calcBkg(sourceName, galcoeff=None, pastTimeWindow=14.0, excludeTmaxTmin=False)[source]¶
It estimates the isotropic and galactic background coefficients.
Note
The current value of the isotropic and galactic bkg coeffs will be set to -1 in order to be free to vary during the fit. It automatically updates the configuration.
- Parameters
sourceName (str) – the name of the source under analysis.
galcoeff (List, optional) – the galactic background coefficients (one for each map).
pastTimeWindow (float, optional) – the number of days previous tmin. It defaults to 14. If it’s 0, the background coefficients will be estimated within the [tmin, tmax] interval.
excludeTmaxTmin (bool, optional) – if this value is True (defalut=False) and pasttimewindow > 0 the calcbkg method computes coeffs between [tmin-pastTimeWindow, tmin] instead of [tmin-pastTimeWindow, tmax]
- Returns
the estimates of the background coefficients and the path to the updated maplist file.
- Return type
galactic coefficients, isotropic coefficients, maplist file (List, List, str)
- mle(maplistFilePath=None, config=None, updateSourceLibrary=True, position='ellipse')[source]¶
It performs a maximum likelihood estimation analysis on every source withing the
sourceLibrary
, producing one output file per source.The method’s behaviour varies according to several configuration options (see docs Configuration file).
The outputfiles have the
.source
extension.The method’s behaviour varies according to several configuration options (see docs here).
Note
It resets the configuration options to their original values before exiting.
- Parameters
maplistFilePath (str) – if not provided, the mle() analysis will use the last generated mapfile produced by
generateMaps()
.- Returns
A list of absolute paths to the output
.source
files.- Raises
MaplistIsNone – is the input argument is None.
Example
>>> maplistFilePath = aganalysis.generateMaps() >>> aganalysis.mle(maplistFilePath) [/home/rt/agilepy/output/testcase0001_2AGLJ2021+4029.source /home/rt/agilepy/output/testcase0001_2AGLJ2021+3654.source]
- lightCurveMLE(sourceName, tmin=None, tmax=None, timetype=None, binsize=86400, position='ellipse')[source]¶
It generates a cvs file containing the data for a light curve plot.
- Parameters
sourceName (str) – the name of the source under analysis.
tmin (float, optional) – starting point of the light curve. It defaults to None. If None the ‘tmin’ value of the configuration file will be used.
tmax (float, optional) – ending point of the light curve. It defaults to None. If None the ‘tmax’ value of the configuration file will be used.
timetype (str, optional) – the time format (‘MJD’ or ‘TT’). It defaults to None. If None the ‘timetype’ value of the configuration file will be used.
binsize (int, optional) – temporal bin size. It defaults to 86400.
position (str, optional) – the position of the source: {“ellipse”, “peak”, “initial”}
- Returns
The absolute path to the light curve data output file.
- aperturePhotometry()[source]¶
It generates a cvs file containing the data for a light curve plot. The method’s behaviour varies according to several configuration options (see docs Configuration file).
- Returns
The absolute path to the light curve data output file.
- Raises
ScienceToolInputArgMissing – if not all the required configuration options have been set.
- displayCtsSkyMaps(maplistFile=None, singleMode=True, smooth=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]¶
It displays the last generated cts skymaps.
- Parameters
maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.
singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.
smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.
saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.
fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .
title (str, optional) – the title of the image. It defaults to None.
cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html
regFiles (list, optional) – A list containing region file(s) absolute path.
regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles.
catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.
catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.
normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.
- Returns
It returns the paths to the image files written on disk.
- displayExpSkyMaps(maplistFile=None, singleMode=True, smooth=False, sigma=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]¶
It displays the last generated exp skymaps.
- Parameters
maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.
singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.
smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.
saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.
fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .
title (str, optional) – the title of the image. It defaults to None.
cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html
regFiles (list, optional) – A list containing region file(s) absolute path
regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles
catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.
catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.
normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.
- Returns
It returns the paths to the image files written on disk.
- displayGasSkyMaps(maplistFile=None, singleMode=True, smooth=False, sigma=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]¶
It displays the last generated gas skymaps.
- Parameters
maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.
singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.
smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.
saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.
fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .
title (str, optional) – the title of the image. It defaults to None.
cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html
regFiles (list, optional) – A list containing region file(s) absolute path
regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles
catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.
catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.
normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.
- Returns
It returns the paths to the image files written on disk.
- displayIntSkyMaps(maplistFile=None, singleMode=True, smooth=False, sigma=4.0, saveImage=False, fileFormat='.png', title=None, cmap='CMRmap', regFiles=None, regFileColors=[], catalogRegions=None, catalogRegionsColor='red', normType='linear')[source]¶
It displays the last generated int skymaps.
- Parameters
maplistFile (str, optional) – the path to the .maplist file. If not specified, the last generated maplist file will be used. It defaults to None.
singleMode (bool, optional) – if set to true, all maps will be displayed as subplots on a single figure. It defaults to True.
smooth (float, optional) – the sigma value of the gaussian smoothing. It defaults to 4.0.
saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.
fileFormat (str, optional) – the extension of the output image. It defaults to ‘.png’ .
title (str, optional) – the title of the image. It defaults to None.
cmap (str, optional) – Matplotlib colormap. It defaults to ‘CMRmap’. Colormaps: https://matplotlib.org/tutorials/colors/colormaps.html
regFiles (list, optional) – A list containing region file(s) absolute path
regFileColors (list, optional) – A list with a color for each region file, the length must be the same of regFiles
catalogRegions (str, optional) – a catalog name. The regions that belongs to the catalog will be loaded. It defaults to None.
catalogRegionsColor (str, optional) – the color of the regions loaded from the catalog.
normType (str, optional) – It selects a normalization function for the data, possible valuses {‘linear’, ‘sqrt’, ‘power’, log’, ‘asinh’}, default ‘linear’.
- Returns
It returns the paths to the image files written on disk.
- displayLightCurve(analysisName, filename=None, lineValue=None, lineError=None, saveImage=False, fermiLC=None)[source]¶
It displays the light curve plot. You can call this method after lightCurveMLE() or aperturePhotometry(). If you pass a filename containing the light curve data, this file will be used instead of using the data generated by the mentioned methods.
- Parameters
analysisName (str, required) – the analysis used to generate the light curve data: choose between ‘mle’ or ‘ap’.
filename (str, optional) – the path of the Lightcurve text data file. It defaults to None. If None the last generated file will be used.
lineValue (int, optional) – mean flux value. It defaults to None.
lineError (int, optional) – mean flux error lines. It defaults to None.
saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.
fermiLC (str, optional) – the path of FERMI lightcurve data
- Returns
It returns the lightcurve plot
- displayGenericColumns(filename, columns, um=None, saveImage=False)[source]¶
An utility method for viewing a generic column from the lightcurvedata
- Parameters
filename (str) – the path of the Lightcurve text data file. It defaults to None. If None the last generated file will be used.
columns (array of str) – the name of the columns to display(first display is always LC), possible values: (time_start_mjd | time_end_mjd | sqrt(ts) | flux | flux_err | flux_ul | gal | gal_error | iso | iso_error | l_peak | b_peak | dist_peak | l | b | r | ell_dist | a | b | phi | exposure | ExpRatio | counts | counts_err | Index | Index_Err | Par2 | Par2_Err | Par3 | Par3_Err | Erglog | Erglog_Err | Erglog_UL | time_start_utc | time_end_utc | time_start_tt | time_end_tt | Fix | index | ULConfidenceLevel | SrcLocConfLevel | start_l | start_b | start_flux | typefun | par2 | par3 | galmode2 | galmode2fit | isomode2 | isomode2fit | edpcor | fluxcor | integratortype | expratioEval | expratio_minthr | expratio_maxthr | expratio_size | Emin | emax | fovmin | fovmax | albedo | binsize | expstep | phasecode | fit_cts | fit_fitstatus0 | fit_fcn0 | fit_edm0 | fit_nvpar0 | fit_nparx0 | fit_iter0 | fit_fitstatus1 | fit_fcn1 | fit_edm1 | fit_nvpar1 | fit_nparx1 | fit_iter1 | fit_Likelihood1)
um (array of str) – unit of measurement, same length of columns
saveImage (bool) – if set to true, saves the image into the output directory. It defaults to False.
- Returns
If saveImage is true tha path of the image otherwise the plot
- convertCatalogToXml(catalogFilepath)[source]¶
It takes a catalog file in AGILE (.txt) format as input and converts it in the xml format.
- Parameters
catalogFilepath (str) – a relative or absolute path to the catalog file in AGILE (.txt) format.
- Raises
SourceModelFormatNotSupported – if the input file format is not supported.
- Returns
A string rapresenting the path to the output file.
- parseMaplistFile(maplistFilePath=None)[source]¶
It parses the maplistfile in order to return sky map files paths.
- Parameters
maplistFilePath (str) – if you don’t specify the maplistfile path, the
one (last generated) –
- Returns
A matrix where each row contains a cts, ext and gas map.
- Raises
MaplistIsNone – if no maplist file is passed and no maplist file have been generated yet.
AGAnalysisWavelet¶
- class api.AGAnalysisWavelet.AGAnalysisWavelet(configurationFilePath, sourcesFilePath=None)[source]¶
This class contains the high-level API methods you can use to run wavelet analysis.
This class requires you to setup a
yaml configuration file
to specify the software’s behaviour.WARNING: This class is deprecated due to the new Python wavelet tool in development
- __init__(configurationFilePath, sourcesFilePath=None)[source]¶
AGAnalysis constructor.
- Parameters
configurationFilePath (str) – a relative or absolute path to the yaml configuration file.
sourcesFilePath (str, optional) – a relative or absolute path to a file containing the description of the sources. Defaults to None. Three different types of format are supported: AGILE format (.txt), XML format (.xml) and AGILE catalog files (.multi).
- static getConfiguration(confFilePath, userName, outputDir, verboselvl, ctsmap, scaletype, scalemin, scalemax, scalenum, methistsize, cclsizemin, cclsizemax, cclradmin, cclradmax, cclscalemin, cclscalemax)[source]¶
Utility method to create a configuration file.
- Parameters
confFilePath (str) – the path and filename of the configuration file that is going to be created.
userName (str) – the username of who is running the software.
outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’
verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug
ctsmap (str) – the path of cts map generated by AGAnalys class
scaletype (str) –
scalemin (float) –
scalemax (float) –
scalenum (int) –
methistsize (float) –
cclsizemin (float) –
cclsizemax (float) –
cclradmin (float) –
cclradmax (float) –
cclscalemin (float) –
cclscalemax (float) –
- Returns
None
- waveletAnalysis()[source]¶
It runs the three science tools required to perform wavelet analysis, and it creates three directories respectively for any tool
- waveletDisplay(fitsfile, saveImage=False, multiImage=False)[source]¶
It displays the wevelets generated by waveletAnalysis.
- Parameters
fitsfile (str) – fits filepath
saveImage (bool, optional) – if set to true, saves the image into the output directory. It defaults to False.
multiImage (bool) – if set to true plots an image for each wavelet
- Returns
It returns the paths to the image files written on disk.
Engineering API¶
AGEngAgileOffaxisVisibility¶
- class api.AGEngAgileOffaxisVisibility.AGEngAgileOffaxisVisibility(configurationFilePath)[source]¶
This class contains the high-level API methods you can use to run engineering analysis.
This class requires you to setup a
yaml configuration file
to specify the software’s behaviour.Class attributes:
- config¶
it is used to read/update configuration values.
- Type
AgilepyConfig
- logger¶
it is used to log messages with different importance levels.
- Type
AgilepyLogger
- __init__(configurationFilePath)[source]¶
AGEngAgileOffaxisVisibility constructor.
- Parameters
configurationFilePath (str) – the relative or absolute path to the yaml configuration file.
Example
>>> from agilepy.api import AGEngAgileOffaxisVisibility >>> ageng = AGEngAgileOffaxisVisibility('agconfig.yaml')
- static getConfiguration(confFilePath, userName, outputDir, verboselvl)[source]¶
Utility method to create a configuration file.
- Parameters
confFilePath (str) – the path and filename of the configuration file that is going to be created.
userName (str) – the username of who is running the software.
outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’
verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug
- Returns
None
- visibilityPlot(logfilesIndex, tmin, tmax, src_x, src_y, ref, zmax=60, step=1, writeFiles=True, computeHistogram=True, saveImage=True, fileFormat='png', title='Visibility Plot')[source]¶
It computes the angular separations between the center of the AGILE GRID field of view and the coordinates for a given position in the sky, given by src_ra and src_dec.
- Parameters
logfilesIndex (str) – the index file for the logs files.
tmin (float) – inferior observation time limit to analize.
tmax (float) – superior observation time limit to analize.
src_x (float) – source position x (unit: degrees)
src_y (float) – source position y (unit: degrees)
zmax (float) – maximum zenith distance of the source to the center of the detector (unit: degrees)
step (integer) – time interval in seconds between 2 consecutive points in the resulting plot. Minimum accepted value: 0.1 s.
writeFiles (bool) – if True, two text files with the separions data will be written on file.
saveImage (bool) – If True, the image will be saved on disk
fileFormat (str) – The output format of the image
title (str) – The plot title
- Returns
the angular separations ti_tt (List): tf_tt (List): ti_mjd (List): tf_mjd (List): skyCordsFK5.ra.deg skyCordsFK5.dec.deg
- Return type
separations (List)
AGEngAgileFermiOffAxisVisibilityComparison¶
- class api.AGEngAgileFermiOffAxisVisibilityComparison.AGEngAgileFermiOffAxisVisibilityComparison(configurationFilePath)[source]¶
This class contains the high-level API methods to run offaxis and offaxis_ap_comparison tools. It’s a AGEng subclass
- __init__(configurationFilePath)[source]¶
AGEngAgileFermiOffAxisVisibilityComparison constructor.
- Parameters
configurationFilePath (str) – the relative or absolute path to the yaml configuration file.
Example
>>> from agilepy.api.AGEngAgileFermiOffAxisVisibilityComparison import AGEngAgileFermiOffAxisVisibilityComparison >>> ageng = AGEngAgileFermiOffAxisVisibilityComparison('agconfig.yaml')
- visibilityPlot(time_windows, ra, dec, fermi_datapath, agile_datapath, run, zmax, mode=<built-in function all>, step=1)[source]¶
It runs offaxis tools and creates a directory containing the result files
- Parameters
time_windws (2d float Array) – It contains the tstart-tstop intervals to process the data, the structure has developed as a 2d array(eg [[t1,t2],[t3, t4], …, [tn-1, tn]])
ra (float) – ra value
dec (float) – dec value
fermi_datapath (str) – fermi log filepath
agile_datapath (str) – agile log filepath
run (integer) – run number
zmax (float) – maximum offaxis degrees
mode (str) – options “agile” | “fermi” | “all”: Select all to plot both data, otherwise it will plot only agile/fermi data
step (float) – step value for plotting
- Returns
A new directory containing the results
- Return type
dir (str)
- apOffaxisComparation(agile_pathAP, fermi_pathAP, tstart_mjd, tstop_mjd, path_offaxis, vertical_boxes_mjd=[], zmax=60, timetype='MJD', data_column_name='cts', time_range=None, trigger_time_tt=None, add_rm=False, rm_files=None, rm_labels=None)[source]¶
It compares and shows aperture photometry data with offaxis results.
WARNING: This class should be instanced anew any time a new plot is generated, to ensure its correctness.
- Parameters
agile_pathAP (str) – agile ap filepath
fermi_pathAP (str) – fermi ap filepath
tstart_mjd (float) – time start in MJD
tstop_mjd (float) – time stop in MJD
path_offaxis (str) – directory path to offaxis results
vertical_boxes_mjd (list) – time in MJD
zmax (float) – maximum offaxis degrees
timetype (str) – time type to plot [MJD, TT]
data_column_name (str) – name of column to plot
time_range (list) – time xrange in MJD or TT depending on “timetype”
trigger_time_tt (float or None) – trigger time in TT
add_rm (bool) – add fourth plot with AGILE ratemeters
rm_files (list) – list of absolute paths for all RM to plot
rm_lables (list) – list of labels to pair to RM files
- Returns
void
- static getConfiguration(confFilePath, userName, outputDir, verboselvl)[source]¶
Utility method to create a configuration file.
- Parameters
confFilePath (str) – the path and filename of the configuration file that is going to be created.
userName (str) – the username of who is running the software.
outputDir (str) – the path to the output directory. The output directory will be created using the following format: ‘userName_sourceName_todaydate’
verboselvl (int) – the verbosity level of the console output. Message types: level 0 => critical, warning, level 1 => critical, warning, info, level 2 => critical, warning, info, debug
- Returns
None
AstroUtils API¶
- class utils.AstroUtils.AstroUtils[source]¶
- static distance(l1, b1, l2, b2)[source]¶
Computes the angular distance between two galatic coordinates.
- Parameters
l1 (float) – longitude of first coordinate
b1 (float) – latitude of first coordinate
l2 (float) – longitude of second coordinate
b2 (float) – latitude of second coordinate
- Returns
the angular distance between (l1, b1) and (l2, b2)
- static AP_filter(filename, threshold, tstart, tstop, outpath)[source]¶
This function filters an aperture photometry file using a threshold value for exposure, it discards the rows lower than threshold and returns a new file merging the continous rows.
- Parameters
filename (str) – path of the aperture photometry file
threshold (float) – exposure threshold
tstart (float) – time start in UTC
tstop (float) – time stop in UTC
- Returns
A filtered file result.txt
Source API¶
Source¶
Point Source¶
- class core.source.Source.PointSource(name)[source]¶
- get(parameterName)[source]¶
It returns a source’s parameter.
- Parameters
paramName (str) – the name of the source’s parameter.
- Raises
SourceParameterNotFound – if the source’s parameter is not found.
- Returns
A dictionary containing the source’s attributes.
Example
>>> s.get("index") >>> s.get("pos") >>> s.get("multiFlux")
- getVal(parameterName)[source]¶
It returns a source’s parameter value.
- Parameters
paramName (str) – the name of the source’s parameter.
- Raises
SourceParameterNotFound – if the source’s parameter is not found.
- Returns
The value of the source’s parameter.
Example
>>> s.getVal("index") >>> s.getVal("pos") >>> s.getVal("multiFlux")
- set(parameterName, attributeValueDict)[source]¶
It sets a source’s parameter.
- Parameters
parameterName (str) – the name of the source’s parameter.
- Returns
None
Example
>>> s.set("index",{"value":1, "min":10})
Science Tools¶
Name |
Short description |
Documents |
---|---|---|
AG_ctsmapgen |
Generates a counts map from the Agile satellite events list, given a list of time intervals, an energy range and a field of view interval. |
|
AG_ctsmapgenT |
Performs the same selection as AG_ctsmapgen, writing its output to text files for futher automatic usage. |
|
AG_diff_conv |
||
AG_expmapgen |
Generates an exposure map from the Agile satellite log files related to a given a list of time intervals. |
|
AG_expmapgenT |
Performs the same selection as AG_expmapgen, writing its output to text files for futher automatic usage. |
|
AG_gasmapgen |
Generates the Galactic diffuse emission maps |
|
AG_intmapgen |
Generates the intensity maps |
|
AG_iterativeGenSrcList |
||
Performs a Maximum Likelyhood Analysis to find the best position, flux and spectral index of a list of sources to explain the given set of count maps. |
AG_multiUserManual.html (outdated) CalcoloTS.html PSF_generation.pdf |
|
It does the same job as AG_multi also considering a set of extended sources. |
AG_multiExtUM.html |
|
Given a list of sources and a set of exposure maps it generates the corresponding counts map by means of a Monte Carlo simulation. It optionally performs on these maps the same analysis of AG_multi. The entire process can be iterated. The generated maps can be saved to disk or just analyzed on the fly. |
AG_multisimUM.html |
|
This application iteratively performs the same analysis of AG_multi, tentatively adding more sources, one at a time, from a second sources list. It produces the best sources list explaining the data. |
||
AG_pasteMap |
This utility program joins two sky maps together adapting their resolution. |
|
AG_spotfinder |
||
AG_thetamapgen |
Input files of the MLE¶
This page provides detailed description of the input files of the science tool AG_multi.
Map list input files¶
‘.maplist4’ file¶
The map list is a text file listing containing at least one line of text. Each line of text describes one set of maps and it is possible to include empty lines or comment lines. The comment lines begin with an exclamation mark.
Each line contains a set of maps:
<countsMap> <exposureMap> <gasMap> <offaxisangle> <galcoeff> <isocoeff>
where:
countsMap, exposureMap and gasMap are file system paths pointing to the corresponding sky maps (see `Sky Maps section<../manual/products.html>`_)
offaxisangle is in degrees;
galcoeff and isocoeff are the coefficients for the galactic and isotropic diffuse components. If positive they will be considered fixed (but see `galmode and isomode<manual/configuration_file.html#section-model>`_ section).
AGILE format (text file)¶
The source list is a text file listing at least one source. Each line of text describes one source, and it is possible to include empty lines or comment lines. The comment lines begin with an exclamation mark.
Each source is described by a line containing space-separated values, in the following order:
'flux' 'l' 'b' 'index' 'fixflag' 'minSqrt(TS)' 'name' 'locationlimit' 'funtype' 'par2' 'par3' 'index limit min' 'index limit max' 'par2 limit min' 'par2 limit max' 'par3 limit min' 'par3 limit max'
The ‘flux’ parameter is expressed in cm^-2 s^-1, and galactic longitude ‘l ‘and latitude ‘b’ are expressed in degrees.
minSqrt(TS) is the minimum acceptable value for the square root of TS: if the optimized significance of a source lies below this value, the source is considered undetected and will be ignored (set to flux = 0) when considering the other sources.
After the source’s name (which should not contain a space), an optional value for the location limitation (’locationlimit’) in degrees may be provided. If this value is present and not zero, the longitude and latitude of the source will not be allowed to vary by more than this value from its initial position.
According to the fixflag, some or all values will be optimized by being allowed to vary.
The ‘funtype’ specify the spectral model. PL indicates power-law fit to the energy spectrum; PC indicates power-law with exponential cut-off fit to the energy spectrum; PS indicates power-law with super-exponential cut-off fit to the energy spectrum; LP indicates log-parabola fit to the energy spectrum.
The ‘index’ of each source represents the initial estimates of the values for that source (a positive number) and could represent the spectral index of the source (see the following table). The other spectral parameters depend on the spectral shape of the source. ‘index limit min’ and ‘index limit max’ specifies the minimum and maximum range where the ‘index’ is searched.
The ‘par2’ and ‘par3’ parameters represent additional spectral parameters in the following table. ‘par2 limit min’, ‘par2 limit max’, ‘par3 limit min’, and ‘par3 limit max’ specify the minimum and maximum range of the ‘par2’ and ‘par3’ respectively.
funtype |
spectral model |
index |
par2 |
par3 |
||
---|---|---|---|---|---|---|
0 |
PL |
PowerLaw |
x^(-[index]) |
α |
||
1 |
PC |
PLExpCutoff |
x^(-[index]) * e^(- x / [par2]) |
α |
Ec |
|
2 |
PS |
PLSuperExpCutoff |
x^(-[index]) * e^(- pow(x / [par2], [par3])) |
α |
Ec |
β |
3 |
LP |
LogParabola |
( x / [par2] ) ^ ( -( [index] + [par3] * log ( x / [par2] ) ) ) |
α |
Ec |
β |
The match of the parameteres is:
index = α: Spectral index for PL, PC, and PS spectral models, first index for LP spectral model; Could be Index or Index1 in the XML format
par2 = Ec (MeV): cut-off energy for PC and PS spectral models, pivot energy for LP spectral model;
par3 = β: Second index for PS spectral models, curvature for LP spectral model;
The usual energy range used to calculate these parameters is 100 MeV – 10 GeV. The MLE procedure calculates also the 1σ uncertainty of the spectral parameters:
∆α: Statistical 1σ uncertainty of α;
∆Ec (MeV): Statistical 1σ uncertainty of Ec;
∆β: Statistical 1σ uncertainty of β.
According to the ‘fixflag’ some or all of those values will be optimized by being allowed to vary. The fixflag is a bit mask, each bit indicating whether the corresponding value is to be allowed to vary:
The user may combine these values, but the flux will always be allowed to vary if at least one of the other values are.
flux |
pos(free=1) |
index/index1 |
par2=cutoffEnergy/pivotEnergy |
par3=index2/curvature |
pos(free=2) |
|
---|---|---|---|---|---|---|
fixflag |
1 |
2 |
4 |
8 |
16 |
32 |
Output files of the MLE¶
This page provides detailed description of the output files of the science tool AG_multi.
The details of the output of the science tool AG_multi that performs the likelihood procedure is still accessible from agilepy. This section describes the “low level” results of the AG_multi procedure. The results are available in the $HOME/agilepy_analysis/<sourcename>_<username>_<date>-<time>/mle directory, where <sourcename> and <username> are defined in the yaml configuration file, <date> and <time> are defined by the system when the analysis starts.
At the end of the fitting process AG_multi generates two main files, describing the most relevant results for all the sources, and a set of source-specific files containing more detailed data about that source.
One of the two main files is in HTML format, and it includes both the input and output data grouped in tables. Having a look at this file the user should quickly understand the outcome of the fitting process and its main results. The next section describes the HTML output in more detail.
The second of the two main files contains the same data printed in text format. This file is divided in two sections. The first contains one line for each diffuse component and the second one line for each source. The first line of each section begins with an exclamation mark (a comment line for many applications) labeling the values printed beneath. In each line the values are separated by a space. This is an example of the text output of the analysis of the 2AGLJ2254+1609 (3C454.3) with the test dataset provided. For this analysis, only one set of maps and one source is used. The iotropic emission components coefficients are kep free and symmetric errors are provided. The flux and position of the source are allowed to vary, while the spectral index is fixed. The name, significance of the source detection, position, source counts with error, source flux with error, and spectral index with error are provided.
! DiffName, Flux, Err, +Err, -Err
Galactic 0.7 0 0 0
Isotropic 8.79898 0.969867 0.984804 -0.955381
! SrcName, sqrt(TS), L_peak, B_peak, Counts, Err, Flux, Err, Index, Err, Par2, Par2Err, Par3, Par3Err, TypeFun
2AGLJ2254+1609 35.5482 86.0638 -38.1753 719.369 35.2059 2.63371e-05 1.28894e-06 2.20942 0 0 0 0 0 0
index, par2, par3 and related errors depend by the spectral mode used.
The counts and fluxes are provided, as well as their errors if the flux is allowed to vary. Finally, the spectral index and its error, if applicable, are provided.
Note
If a source is outside the Galactic plane, fix the diffuse emission coefficient parameter (gal) to 0.7 with ag.setOptions(galcoeff=[0.7])
‘.source’ file¶
The .source file is an internal technical file produced by the maximum likelihood estimator mle() procedure for each source. It contains all the analysis results for each source that is part of the ensemble of models. Agilepy extract from this .source file the most important parameters useful for the final user.
When possible, two additional files describing the source contour (possibile only if position is kept free).
The text file contains some comment-like lines (first character is an exclamation mark) labeling the values printed beneath. This is an example of text output, consistent with the example given above:
! Label Fix index ULConfidenceLevel SrcLocConfLevel start_l start_b start_flux [ lmin, lmax ] [ bmin, bmax ] typefun par2 par3 galmode2 galmode2fit isomode2 isomode2fit edpcor fluxcor integratortype expratioEval expratio_minthr expratio_maxthr expratio_size [ index_min , index_max ] [ par2_min , par2_max ] [ par3_min , par3_max ] contourpoints minimizertype minimizeralg minimizerdefstrategy minimizerdeftol
! sqrt(TS)
! L_peak B_peak Dist_from_start_position
! L B Dist_from_start_position r a b phi
! Counts Err +Err -Err UL
! Flux(ph/cm2s) [0 , 1e+07] Err +Err -Err UL(ph/cm2s) ULbayes(ph/cm2s) Exp(cm2s) ExpSpectraCorFactor null null null Erglog(erg/cm2s) Erglog_Err Erglog_UL(erg/cm2s) Sensitivity FluxPerChannel(ph/cm2s)
! Index [0.5 , 5] Index_Err Par2 [20 , 10000] Par2_Err Par3 [0 , 100] Par3_Err
! cts fitstatus0 fcn0 edm0 nvpar0 nparx0 iter0 fitstatus1 fcn1 edm1 nvpar1 nparx1 iter1 Likelihood1
! Gal coeffs [0 , 100] and errs
! Gal zero coeffs and errs
! Iso coeffs [0 , 100] and errs
! Iso zero coeffs and errs
! Start_date(UTC) End_date(UTC) Start_date(TT) End_date(TT) Start_date(MJD) End_date(MJD)
! Emin..emax(MeV) fovmin..fovmax(deg) albedo(deg) binsize(deg) expstep phasecode ExpRatio
! Fit status of steps ext1, step1, ext2, step2, contour, index, ul [-1 step skipped, 0 ok, 1 errors]
! Number of counts for each step (to evaluate hypothesis)
! skytypeL.filter_irf skytypeH.filter_irf
2AGLJ2254+1609 1 2.20942 2 5.99147 86.1236 -38.1824 2.64387e-05 [ -1 , -1 ] [ -1 , -1 ] 0 0 0 0 0 0 0 0.75 0 1 1 0 15 10 [ 0.5 , 5 ] [ 20 , 10000 ] [ 0 , 100 ] 40 Minuit Migrad 2 0.01
47.8468
86.1236 -38.1824 0
-1 -1 -1 -1 -1 -1 -1
718.633 31.0247 31.4119 -30.6392 782.234
2.64387e-05 1.14141e-06 1.15565e-06 -1.12722e-06 2.87787e-05 2.01487e-05 2.71811e+07 1 0 0 0 4.27293e-09 1.8447e-10 4.6511e-09 0.0 2.64387e-05
2.20942 0 0 0 0 0
909 -1 2456.44 0.5 0 8 3 0 1311.78 7.28513e-16 1 8 3 1828.16
0.7 0
0.7 0
8.83231 0
8.83231 0
2010-11-13T00:01:06 2010-11-21T00:01:06 216691200.0000000 217382400.0000000 55513.0000000 55521.0000000
100..10000 0..60 80 0.25 0 6 0
-1 -1 -1 0 -1 -1 0
-1 2124 -1 2124 -1 -1 2124
SKY002.SFMG_H0025 SKY002.SFMG_H0025
The counts and fluxes are provided, as well as their symmetric, positive, and negative errors if the flux is allowed to vary. For convenience, the exposure of the source, used to calculate the source counts from the flux, is also provided. Finally, the spectral index and its error, if applicable, are provided.
‘.source’ Attributes¶
Parameter name |
UM |
Description |
default |
range |
---|---|---|---|---|
Label |
||||
Fix |
Value of the fixflag |
|||
index |
Initial value of the index parameter |
|||
ULConfidenceLevel |
Upper limit confidence level espressed as sqrt(TS) |
2 |
||
SrcLocConfLevel |
Source location contour confidence level % |
95 99 95 68 50 |
95 |
|
start_l |
||||
start_b |
||||
start_flux |
(ph/cm2s) |
|||
[ lmin lmax ] |
||||
[ bmin bmax ] |
||||
typefun |
||||
par2 |
||||
par3 |
||||
galmode2 |
||||
galmode2fit |
||||
isomode2 |
||||
isomode2fit |
||||
edpcor |
||||
fluxcor |
||||
integratortype |
||||
expratioEval |
||||
expratio_minthr |
||||
expratio_maxthr |
||||
expratio_size |
||||
[ index_min index_max ] |
||||
[ par2_min par2_max ] |
||||
[ par3_min par3_max ] |
||||
contourpoints |
||||
minimizertype |
||||
minimizeralg |
||||
minimizerdefstrategy |
||||
minimizerdeftol |
||||
sqrt(TS) |
||||
L_peak |
||||
B_peak |
||||
Dist_from_start_position |
||||
L |
||||
B |
||||
Dist_from_start_position |
||||
r |
||||
a |
||||
b |
||||
phi |
||||
Counts |
||||
Err |
||||
+Err |
||||
-Err |
||||
UL |
||||
Flux |
(ph/cm2s) |
|||
Err |
||||
+Err |
||||
-Err |
||||
UL |
(ph/cm2s) |
|||
ULbayes |
(ph/cm2s) |
|||
Exp |
(cm2s) |
|||
ExpSpectraCorFactor |
||||
Erglog |
(erg/cm2s) |
|||
Erglog_Err |
||||
Erglog_UL |
(erg/cm2s) |
|||
Sensitivity |
||||
FluxPerChannel |
(ph/cm2s) |
|||
Index |
||||
Index_Err |
||||
Par2 |
||||
Par2_Err |
||||
Par3 |
||||
Par3_Err |
||||
cts |
||||
fitstatus0 |
||||
fcn0 |
||||
edm0 |
||||
nvpar0 |
||||
nparx0 |
||||
iter0 |
||||
fitstatus1 |
||||
fcn1 |
||||
edm1 |
||||
nvpar1 |
||||
nparx1 |
||||
iter1 |
||||
Likelihood1 |
||||
Gal coeffs |
||||
errs |
||||
Gal zero coeffs |
||||
errs |
||||
Iso coeffs |
||||
errs |
||||
Iso zero coeffs |
||||
errs |
||||
Start_date(UTC) |
||||
End_date(UTC) |
||||
Start_date(TT) |
||||
End_date(TT) |
||||
Start_date(MJD) |
||||
End_date(MJD) |
||||
Emin..emax |
MeV |
|||
fovmin..fovmax |
deg |
|||
albedo |
deg |
|||
binsize |
deg |
|||
expstep |
||||
phasecode |
||||
ExpRatio |
||||
Fit status of steps ext1 |
||||
Fit status of steps step1 |
||||
Fit status of steps ext2 |
||||
Fit status of steps step2 |
||||
Fit status of steps contour |
||||
Fit status of steps index |
||||
Fit status of steps ul |
||||
Number of counts for ext1 |
||||
Number of counts for step1 |
||||
Number of counts for ext2 |
||||
Number of counts for step2 |
||||
Number of counts for contour |
||||
Number of counts for index |
||||
Number of counts for ul |
||||
skytypeL.filter_irf |
||||
skytypeH.filter_irf |
‘.source.con’ file¶
outfile.source.con: source contour (if found).
‘.source.reg’ file¶
outfile.source.reg: ellipse best fitting the source contour (if found).
‘.log’ file¶
Log file with a line for each step of the fitting process.
HTML output¶
AG_multi provides an HTML output of the results. The HTML output file is divided into two sections, input and output. The input section contains three subsections: the command line options, the map list and the source list contents. The command line options are listed in two tables, one with the names of the IRFs (PSD, SAR and EDP) files, the other with the rest of the command line. The maplist subsection also contains two tables. The first lists the mapfile contents and the second contains the data from the map files themselves. This table contains one map per row, and each column contains one value only if it is the same for all the maps. The last table of the input section contains the source list contents. The output section is also divided into three subsections. The first is a table showing the Galactic and isotropic coefficients and their errors. Also in this table some cells may be grouped together when the values are all the same. The second is a table showing the fit results for the sources and their errors. One of the listed values is the contour equivalent radius, explained in the next section. The last table shows the source flux per energy channel, and it is present only when different energy channels are considered. This table has one row for each source and one column for each energy channel.
Need Help¶
If you have troubles please email to:
leonardo.baroncelli[at]inaf.it
antonio.addis[at]inaf.it
andrea.bulgarelli[at]inaf.it
or open an issue of GitHub: https://github.com/AGILESCIENCE/Agilepy/issues
Known issues¶
The unit test “test_aperture_photometry” fails on macos, therefore the aperturePhotometry() method is not available on this OS.
Each notebook should instantiate only one AGAnalysis object, otherwise the logger will be duplicated.
In certain situations %matplotlib widget has a weird behaviour. If you have problems with map sizes or interactions, comment the line %matplotlib widget
Development¶
Install the development environment¶
If you want to try agilepy’s new features that are not officially released yet, a develpoment environment called agilepy-environment is available into Anaconda cloud. It contains all the dependencies unless agilepy, which must be installed by hand cloning the repository.
Anaconda¶
conda config --add channels conda-forge
conda config --add channels plotly
conda create -n agilepydev -c agilescience agiletools agilepy-dataset
conda activate agilepydev
git clone https://github.com/AGILESCIENCE/Agilepy.git
cd Agilepy && git checkout develop
conda env update -f environment.yml
python setup.py develop
Docker¶
docker pull agilescience/agilepy-recipe:latest
mkdir shared_dir && cd shared_dir && git clone https://github.com/AGILESCIENCE/Agilepy.git \
&& cd Agilepy && git checkout develop
docker run --rm -it -p 8888:8888 \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
-v $SHARED_DIR_PATH:/shared_dir \
agilescience/agilepy-recipe:latest
## -- Inside the container --
conda activate agilepydev
cd /shared_dir/Agilepy python setup.py develop
jupyter notebook --port 8889 --ip 0.0.0.0 --allow-root
Now you have the agilepy’s latest version installed in your environment.
Git flow¶
Branches¶
Two main branches:
master: contains only production releases.
develop: contains commits that will be included in the next production release.
Two support branches:
feature branch: each new feature (Trello’s card) should be developed in its own feature branch, branching from develop and merged back into it. The feature branch are not pushed into the remote.
hotfix branch: if an hotfix is needed it should be develop in its own branch, branching from master and merged back to it.

Versioning¶
The master branch contains only production releases: when the develop branch (or hotfix branch) is merged to master a new release tag must be created. Its name follows the semantic versioning.
x.y.z
Incrementing:
x version when you make incompatible API changes,
y version when you add functionality in a backwards compatible manner, and
z version when you make backwards compatible bug fixes.
Branches names¶
The master and the develop branch have an infinite lifetime, hence their name is fixed.
The feature branch takes the following format:
feature-#<card-number>-<short-description>
e.g. feature-#61-new-cool-feature
The hotfix branch name takes the following format:
hotfix-#<card-number>-<release-number>
e.g. hotfix-#57-1.0.0
The release number is the one of the production release from which it originates from.
Getting started¶
Development of a new feature¶
Create a new feature branch:
git checkout develop
git pull origin develop
git checkout -b feature-#61-new-cool-feature develop
Development and testing of the new feature.
When you have finished, update the CHANGELOG.md and commit your changes.
vim CHANGELOG
git commit -m "feature-#61-new-cool-feature done"
In the meantime it is possible that someone else have pushed his work into the develop branch. In this case you have to merge the changes in your feature branch.
git pull **origin** develop
Finally you can merge your feature branch back to develop branch.
git merge --no-ff feature-#61-new-cool-feature
git branch -d feature-#61-new-cool-feature
git push origin develop
Add configuration parameters¶
Let’s say we want to add the following configuration section to the AGAnalysis’ configuration file.
ap:
radius: 0.25
timeslot: 3600
Add the new section to the AGAnalysis.getConfiguration() method.
Add the type of the configuration parameters within the AGAnalysisConfig.checkOptionsType() method (in the corresponding lists).
If the parameters need some kind of validation (this is not the case), add a new method in ValidationStrategies and call it within the AGAnalysisConfig.validateConfiguration() (check examples).
If the parameters need some kind of transformation (this is not the case), add a new method in CompletionStrategies and call it within the AGAnalysisConfig.completeConfiguration() (check examples).
Add the new configuration section to all the unit test configuration files.
Document the new configuration parameters within the manual/configuration_file.rst file.
Add a new science tool¶
Let’s say we want to add a new (c++) science tool: AG_ap.
Add a new class within the api/ScienceTools.py script. You need to implement some abstract methods.
You can use the new class as follows:
apTool = AP("AG_ap", self.logger)
apTool.configureTool(self.config)
if not apTool.allRequiredOptionsSet(self.config):
raise ScienceToolInputArgMissing("Some options have not been set.")
products = apTool.call()
Release of a new version¶
Change the version of the software in setup.py. The version increment must be take in account all the commits of the develop branch. You can check the CHANGELOG.md to facilitate this process. Please, add the new tag within the CHANGELOG.md file.
git checkout master
git merge --no-ff develop
git tag -a <new-tag>
git push origin <new-tag>
DevOps¶
A high level description of agilepy’s devops is in the image below:

This scheme workflow produces three images:
base_image: It’s an image with all the dependencies except Agilepy python library, it’s used for developing purposes only by developers. Base image is built after a new commit in agilepy-recipe repository.
latest code image: It’s the base_image with Agilepy’s develop branch at latest commit, useful for using or testing agilepy’s updates not officially released. This image is not supported nor stable and is built by dockerhub after github’s testing pipelines are successful.
released image: The base_image with Agilepy’s release tag. By default the community shall be download this image. It’s built when a new tag is created.