Usage¶
The fiberassign package has some main scripts as well as lower-level python tools that can be used in custom scripts.
Basic Tools¶
There are several command-line programs which can be used to drive the fiber assignment process. The first is the fba_run script which takes one or more target files and other options
usage: fba_run [-h] --targets TARGETS [TARGETS ...] [--gfafile GFAFILE]
[--footprint FOOTPRINT] [--tiles TILES]
[--positioners POSITIONERS] [--status STATUS]
[--rundate RUNDATE] [--dir DIR] [--prefix PREFIX] [--split]
[--standards_per_petal STANDARDS_PER_PETAL]
[--sky_per_petal SKY_PER_PETAL] [--write_all_targets]
[--overwrite] [--sciencemask SCIENCEMASK] [--stdmask STDMASK]
[--skymask SKYMASK] [--safemask SAFEMASK]
[--excludemask EXCLUDEMASK]
optional arguments:
-h, --help show this help message and exit
--targets TARGETS [TARGETS ...]
Input file with targets of any type. This argument can
be specified multiple times (for example if standards
/ skies / science targets are in different files). By
default, the DESI_TARGET column and bitfield values
defined in desitarget are used to determine the type
of each target. Each filename may be optionally
followed by comma separated values that will be passed
to the arguments of the load_target_file() function.
See docs for details.
--sky SKY [SKY ...] Input file with sky or 'bad sky' targets. This option
exists in order to treat main-survey sky target files
as valid for other survey types. If you are running a
main survey assignment, you can just pass the sky file
to the --targets list.
--gfafile GFAFILE Optional GFA targets FITS file
--footprint FOOTPRINT
Optional FITS file defining the footprint. If not
specified, the default footprint from desimodel is
used.
--tiles TILES Optional text file containing a subset of the tile IDs
to use in the footprint, one ID per line. Default uses
all tiles in the footprint.
--positioners POSITIONERS
Optional FITS file describing the fiber positioner
locations. Default uses the file from desimodel.
--status STATUS Optional fiber status file in astropy ECSV format.
Default treats all fibers as good.
--rundate RUNDATE Optional date to simulate for this run of fiber
assignment, used with the fiber status file to
determine which fibers currently have problems.
Default uses the current date. Format is YYYY-MM-
DDTHH:mm:ss in UTC time.
--dir DIR Output directory.
--prefix PREFIX Prefix of each file (before the <tile>.fits).
--split Split output into tile prefix directories.
--standards_per_petal STANDARDS_PER_PETAL
Required number of standards per petal
--sky_per_petal SKY_PER_PETAL
Required number of sky targets per petal
--write_all_targets When writing target properties, write data for all
available targets, not just those which are assigned.
This is convenient, but increases the write time and
the file size.
--overwrite Overwrite any pre-existing output files
--sciencemask SCIENCEMASK
Default DESI_TARGET mask to use for science targets
--stdmask STDMASK Default DESI_TARGET mask to use for stdstar targets
--skymask SKYMASK Default DESI_TARGET mask to use for sky targets
--safemask SAFEMASK Default DESI_TARGET mask to use for safe backup
targets
--excludemask EXCLUDEMASK
Default DESI_TARGET mask to exclude from any
assignments
EXAMPLE: One could run this with mostly default options:
%> fba_run --targets mtl.fits standards.fits sky.fits other_targets.fits \
--write_all_targets \
--dir out_raw
The names of the target files above are arbitrary and they can contain any targets. The full list of targets is built up in memory from the contents of these files. Running this command will produce an output directory of files (one per tile) containing the basic target information and assignment results. For now, this “raw” output format is slightly different than the one used historically, and does not contain all the information that was in the original target catalog. This will be changed in the future. For now, files with this extra info can be created by post-processing the output with the fba_merge_results script:
usage: fba_merge_results [-h] --targets TARGETS [TARGETS ...] --dir DIR
[--prefix PREFIX] [--split] [--out OUT]
[--out_prefix OUT_PREFIX] [--out_split]
[--columns COLUMNS] [--skip_raw]
optional arguments:
-h, --help show this help message and exit
--targets TARGETS [TARGETS ...]
Input file with targets of any type. This argument can
be specified multiple times (for example if standards
/ skies / science targets are in different files).
--sky SKY [SKY ...] Input file with sky or 'bad sky' targets. This option
exists in order to treat main-survey sky target files
as valid for other survey types. If you are running a
main survey assignment, you can just pass the sky file
to the --targets list.
--dir DIR Directory containing fiberassign results.
--prefix PREFIX Prefix of each file (before the <tile>.fits).
--split Results are in tile prefix directories.
--out OUT Output directory for the merged files. Default is the
directory containing the fiberassign output.
--out_prefix OUT_PREFIX
Prefix of each output file.
--out_split Split output into tile prefix directories.
--columns COLUMNS Override the column names of target data to be copied
from the target files into the fiber assignment files.
This should be a comma-separated list.
--skip_raw Do not copy the raw fiberassign HDUs to the merged
output.
EXAMPLE: Continuing our minimal example from above:
%> fba_merge_results \
--targets mtl.fits standards.fits sky.fits other_targets.fits \
--dir out_raw \
--out out_merged
There are several built-in tools for doing quality assurance of the resulting assignment. One of these is the fba_run_qa script. This will compute the per-tile counts for the different target types and write these to JSON file (by default named “qa.json” in the output directory):
usage: fba_run_qa [-h] --dir DIR [--prefix PREFIX] [--split] [--out OUT]
[--footprint FOOTPRINT] [--tiles TILES]
[--positioners POSITIONERS] [--status STATUS]
[--rundate RUNDATE]
optional arguments:
-h, --help show this help message and exit
--dir DIR Directory containing fiberassign outputs.
--prefix PREFIX Prefix of each file (before the <tile>.fits).
--split Results are in tile prefix directories.
--out OUT Output file for QA. Default is a file in directory
containing the fiberassign output.
--footprint FOOTPRINT
Optional FITS file defining the footprint. If not
specified, the default footprint from desimodel is
used.
--tiles TILES Optional text file containing a subset of the tile IDs
to use in the footprint, one ID per line. Default uses
all tiles in the footprint.
--positioners POSITIONERS
Optional FITS file describing the fiber positioner
locations. Default uses the file from desimodel.
--status STATUS Optional fiber status file in astropy ECSV format.
Default treats all fibers as good.
--rundate RUNDATE Optional date to simulate for this run of fiber
assignment, used with the fiber status file to
determine which fibers currently have problems.
Default uses the current date. Format is YYYY-MM-DD or
YYYY-MM-DDTHH:mm:ss in UTC time.
EXAMPLE: One could run this on our example output with:
%> fba_run_qa --dir out_raw
It is also frequently useful to plot the results of the assignment. There are many customized plotting options possible using the low-level tools, but there is also a command-line script to create a PDF format plot of each tile. By default, this script plots all petals of all tiles using a “simple” representation of each positioner (two lines). Running fba_plot_results in this way will require several minutes per tile, but multiple processes will be used to plot tiles in parallel:
usage: fba_plot_results [-h] --dir DIR [--prefix PREFIX] [--split] [--out OUT]
[--out_prefix OUT_PREFIX] [--out_split]
[--petals PETALS] [--real_shapes]
[--footprint FOOTPRINT] [--tiles TILES]
[--positioners POSITIONERS] [--status STATUS]
[--rundate RUNDATE]
optional arguments:
-h, --help show this help message and exit
--dir DIR Directory containing fiberassign results.
--prefix PREFIX Prefix of each file (before the <tile>.fits).
--split Results are in tile prefix directories.
--out OUT Output directory for plots. Default is the directory
containing the fiberassign output.
--out_prefix OUT_PREFIX
Prefix of each output file.
--out_split Split output into tile prefix directories.
--petals PETALS Comma-separated list of petals to plot (default is all
petals)
--real_shapes Plot the actual positioner shapes. This looks better
but takes much longer and makes bigger files.
Recommended only for plotting limited tiles / petals.
--footprint FOOTPRINT
Optional FITS file defining the footprint. If not
specified, the default footprint from desimodel is
used.
--tiles TILES Optional text file containing a subset of the tile IDs
to use in the footprint, one ID per line. Default uses
all tiles in the footprint.
--positioners POSITIONERS
Optional FITS file describing the fiber positioner
locations. Default uses the file from desimodel.
--status STATUS Optional fiber status file in astropy ECSV format.
Default treats all fibers as good.
--rundate RUNDATE Optional date to simulate for this run of fiber
assignment, used with the fiber status file to
determine which fibers currently have problems.
Default uses the current date. Format is YYYY-MM-DD or
YYYY-MM-DDTHH:mm:ss in UTC time.
EXAMPLE: In our example, let’s plot only one tile and use options to plot the full positioner geometry. Notice how we made a text file with just one tile ID to use:
%> cat plot_tiles.txt
11108
%> fba_plot_results \
--tiles plot_tiles.txt \
--dir out_raw \
--real_shapes
In the plot above, science targets are red, sky is blue, standards are orange, and dual science / standard targets are green.
We can also make a crude visualization of the QA for the entire footprint using the fba_plot_qa script. Note that when plotting only a few tiles the resulting plot is not as useful (but in that case you can get the numbers directly out of the JSON):
usage: fba_plot_qa [-h] --qafile QAFILE [--outroot OUTROOT] [--labels]
optional arguments:
-h, --help show this help message and exit
--qafile QAFILE Input QA file.
--outroot OUTROOT Output root file name. Default uses input.
--labels Plot tile IDs at center of circles.
EXAMPLE: For our small test above we could do:
%> fba_plot_qa --qafile "out_raw/qa.json"
If you are only plotting a few tiles and want to see the tile IDs on the plot, use the –labels option.
Todo
Document the use of the qa-fiberassign tool, after verifying that it still works.
Interactive Debugging and Testing¶
The fiberassign codebase respects the DESI_LOGLEVEL environment variable. The default level if this is not set is “INFO”. Setting this environment variable to “DEBUG” will produce more detailed output. There are other environment variables that can be set to dump even more details about the internal assignment process. Examples:
%> export DESI_DEBUG_TARGET=123456789
%> export DESI_DEBUG_LOCATION=4321
%> export DESI_DEBUG_TILE=1111
These options are combined with a logical OR and any combination of tile, fiber, or target specified will have all possible info logged.
Warning
Use of these “extra” debug variables will have a HUGE impact on code performance. Do not use in large production runs.
Data File Testing¶
It is often useful to see if a given set of input target files are compatible with all of the commandline tools in fiberassign. This usage pattern was so common that a high-level script was created:
usage: fba_fulltest [-h] --targets TARGETS [TARGETS ...] [--sky SKY [SKY ...]]
[--gfafile GFAFILE] [--footprint FOOTPRINT]
[--tiles TILES] [--out OUT] [--plotpetals PLOTPETALS]
[--positioners POSITIONERS] [--status STATUS]
[--rundate RUNDATE]
[--standards_per_petal STANDARDS_PER_PETAL]
[--sky_per_petal SKY_PER_PETAL]
optional arguments:
-h, --help show this help message and exit
--targets TARGETS [TARGETS ...]
Input file with targets of any type. This argument can
be specified multiple times (for example if standards
/ skies / science targets are in different files). By
default, the '--mask_column' (default
DESI_TARGET)column and bitfield values defined in
desitarget are used to determine the type of each
target. Each filename may be optionally followed by
comma and then one of the strings 'science',
'standard', 'sky' or 'safe' to force all targets in
that file to be treated as a fixed target type.
--sky SKY [SKY ...] Input file with sky or 'bad sky' targets. This option
exists in order to treat main-survey sky target files
as valid for other survey types. If you are running a
main survey assignment, you can just pass the sky file
to the --targets list.
--gfafile GFAFILE Optional GFA targets FITS file
--footprint FOOTPRINT
Optional FITS file defining the footprint. If not
specified, the default footprint from desimodel is
used.
--tiles TILES Optional text file containing a subset of the tile IDs
to use in the footprint, one ID per line. Default uses
all tiles in the footprint.
--out OUT Top-level output directory.
--plotpetals PLOTPETALS
Comma-separated list of petals to plot (default is all
petals)
--positioners POSITIONERS
Optional FITS file describing the fiber positioner
locations. Default uses the file from desimodel.
--status STATUS Optional fiber status file in astropy ECSV format.
Default treats all fibers as good.
--rundate RUNDATE Optional date to simulate for this run of fiber
assignment, used with the fiber status file to
determine which fibers currently have problems.
Default uses the current date. Format is YYYY-MM-
DDTHH:mm:ss in UTC time.
--standards_per_petal STANDARDS_PER_PETAL
Required number of standards per petal
--sky_per_petal SKY_PER_PETAL
Required number of sky targets per petal
This runs the specified files through the assignment and merging, as well as running QA and plotting on both the raw outputs and the merged outputs.
Legacy Compatibility Wrappers¶
In order to maintain backwards compatibility with the previous version of
fiberassign, there is a replacement wrapper script called fiberassign
which
translates the command line options into arguments to pass to the newer
scripts.
Running fiberassign --help
gives the full set of command line
options:
usage: fiberassign [-h] --mtl MTL --sky SKY --stdstar STDSTAR
--fibstatusfile FIBSTATUSFILE
[--footprint FOOTPRINT]
[--positioners POSITIONERS] [--surveytiles SURVEYTILES]
[--telra TELRA] [--teldec TELDEC] [--tileid TILEID]
[--tileobsconditions TILEOBSCONDITIONS] [--outdir OUTDIR]
[--starmask STARMASK] [--rundate RUNDATE]
[--gfafile GFAFILE] [--nstarpetal NSTARPETAL]
[--nskypetal NSKYPETAL] [--nocleanup]
optional arguments:
-h, --help show this help message and exit
--mtl MTL input targets (FITS file)
--sky SKY input sky positions (FITS file)
--stdstar STDSTAR input std stars (FITS file)
--fibstatusfile FIBSTATUSFILE
list of positioners and its status (ECSV file)
--footprint FOOTPRINT
list of tiles defining the footprint (FITS file)
--positioners POSITIONERS
list of positioners on the focal plane (FITS file)
--surveytiles SURVEYTILES
set of tiles to run fiberassign on (text file)
--telra TELRA Right Ascension of arbitrary pointing - overrides
--surveytiles
--teldec TELDEC Declination of arbitrary pointing - overrides
--surveytiles
--tileid TILEID Integer ID of arbitrary pointing - overrides
--surveytiles
--tileobsconditions TILEOBSCONDITIONS
Mask describing observing program (DARK:1, GRAY:2,
BRIGHT:4) - overrides --surveytiles
--outdir OUTDIR output directory (default = ./)
--starmask STARMASK integer mask defining standard stars
--rundate RUNDATE run date [YYYY-MM-DD]
--gfafile GFAFILE GFA file (FITS tile)
--nstarpetal NSTARPETAL
number of standard stars per petal (default=10)
--nskypetal NSKYPETAL
number of sky fibers per petal (default=40)
--nocleanup
An example that provides the minimal set of required arguments would be:
fiberassign --mtl mtl.fits --stdstar std.fits --sky sky.fits
--fibstatusfile fiberstatus.ecsv --outdir $SCRATCH/temp/
In this example there are four files that must be explicitly provided:
mtl.fits
: DESI Merged Target List files contain a single binary table covering the entire footprint. They contain the variables in the Targets files plus other variables that define the priority and number of observations as required by fiber assignment. These variables are computed using the available information both in the target and the DESI redshift catalogs. The full datamodel can be found here.std.fits
: DESI standard star locations contain a single binary table covering the entire footprint. They contain the variables in the Targets files with the objects that were flagged as standards. This file follows the mtl datamodel.sky.fits
: DESI sky locations contain a single binary table covering the entire Legacy Surveys footprint. The imaging “blob maps” are bisected to achieve a requisite number of sky locations per sq. deg. Sky locations are placed within the bisected grid as far from blobs that contain sources as is possible. Flux is measured in an aperture at each sky location. This file follows the mtl datamodel.fiberstatus.ecsv
: