Examples

It is useful to consider several concrete examples of running fiber assignment and also describe the techniques for exploring the impact of the assignment algorithm.

Debugging Example

This example shows a recent debugging test done with a small dataset. This was using several separate target files. I ran them with:

%> export DESI_LOGLEVEL=DEBUG
%> fba_run --targets mtl.fits standards-dark.fits sky.fits

And the outputs were written by default to date-stamped output directory. Then I made some plots:

%> fba_plot_results \
   --dir "out_fiberassign_2018-11-24T11:09:48"

I opened one resulting image (fiberassign_001148.pdf) and zoomed into one petal. I noticed that device location 2233 was assigned to a standard. Let’s look at the details of how this device was assigned:

%> export DESI_DEBUG_LOCATION=2233
%> fba_run --targets mtl.fits standards-dark.fits sky.fits

The log showed that there were several targets available to this fiber on tile 1148:

DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599630147 (type=1), total priority 3200.07
DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599629415 (type=1), total priority 3000.91
DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599629372 (type=1), total priority 3000.24
DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599627959 (type=1), total priority 2000.04
DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599631299 (type=3), total priority 1500.79
DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599631166 (type=1), total priority 1500.1
DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599632448 (type=4), total priority 0.293402
DEBUG: targets avail:  tile 1148, loc 2233 append ID \
    288230398599632236 (type=4), total priority 0.179244

Looking at the target type (which are the 4 categories of target used internally in fiberassign, defined in targets.py / targets.h), we see that there are 2 sky targets, 5 science targets, and one target which is both a science target and a standard. Later, during the assignment of unused locations to science targets, we see:

DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599630147, subpriority 0.0692912
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599629415, subpriority 0.914937
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599629372, subpriority 0.241329
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599627959, subpriority 0.0415658
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599631299, subpriority 0.786657
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599631166, subpriority 0.0962114
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599632448 is wrong type (4)
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    available target 288230398599632236 is wrong type (4)
DEBUG: find_best: tile 1148, loc 2233, target \
    288230398599630147, type 1 accept with priority = 3200, \
    subpriority = 0.0692912, obs_remain = 2
DEBUG: find_best: tile 1148, loc 2233, target \
    288230398599630147, type 1 SELECTED
DEBUG: assign unused science: tile 1148, petal 4 loc 2233 \
    found best object 288230398599630147

So it skipped over the two sky targets and selected a high-priority science target. Later, during the redistribution step, we see:

DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 considering for swap...
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 considering tile indices 0 to 16070
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 1148,2233 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 6926,175 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12688,968 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12688,991 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12689,2561 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12689,2587 already assigned
DEBUG: redist: tile 1148, loc 2233, target \
    288230398599630147 keeping assignment

So the science target initially assigned to this location had no other unassigned available tile / locations for potential swapping. Further in the assignment process, we see that a standard was found available to this location. Here is the log snippet from this step:

DEBUG: assign force standard: tile 1148, petal 4, \
    loc 2233, found object 288230398599631299 \
    with weight 1500.79
DEBUG: assign force standard: tile 1148, petal 4, \
    class 1500, object 288230398599631299, subpriority \
    1500.79, available loc 2233 at target \
    288230398599630147 is wrong class (3200)
DEBUG: assign force standard: tile 1148, petal 4, \
    class 1600, object 288230398599631299, subpriority \
    1500.79, available loc 2233 at target \
    288230398599630147 is wrong class (3200)
DEBUG: assign force standard: tile 1148, petal 4, \
    class 2000, object 288230398599631299, subpriority \
    1500.79, available loc 2233 at target \
    288230398599630147 is wrong class (3200)
DEBUG: assign force standard: tile 1148, petal 4, \
    class 2100, object 288230398599631299, subpriority \
    1500.79, available loc 2233 at target \
    288230398599630147 is wrong class (3200)
DEBUG: assign force standard: tile 1148, petal 4, \
    class 3000, object 288230398599631299, subpriority \
    1500.79, available loc 2233 at target \
    288230398599630147 is wrong class (3200)
DEBUG: assign force standard: tile 1148, petal 4, \
    class 3200, object 288230398599631299, subpriority \
    1500.79, available loc 2233 bumping science \
    target 288230398599630147
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 considering for swap...
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 considering tile indices 0 to 16070
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 1148,2233 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 6926,175 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12688,968 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12688,991 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12689,2561 already assigned
DEBUG: reassign: tile 1148, loc 2233, target \
    288230398599630147 avail T/F 12689,2587 not OK to assign

What happened here is that a standard was found to replace the low-priority science target assigned to location 2233. The existing science target was tested for other available tile / locations, but all but one of those locations were already assigned, and that one remaining device would produce a collision. During the forced assignment of sky targets, this is what happens to this location:

DEBUG: assign force sky: tile 1148, petal 4, loc 2233, \
    found object 288230398599632448 with weight 0.293402
DEBUG: assign force sky: tile 1148, petal 4, loc 2233, \
    found object 288230398599632236 with weight 0.179244
DEBUG: assign force sky: tile 1148, petal 4, class 1500, \
    object 288230398599632448, subpriority 0.293402, \
    available loc 2233 at science target \
    288230398599631299 is also a standard- skipping
DEBUG: assign force sky: tile 1148, petal 4, class 1500, \
    object 288230398599632236, subpriority 0.179244, \
    available loc 2233 at science target \
    288230398599631299 is also a standard- skipping
DEBUG: assign force sky: tile 1148, petal 4, class 1600, \
    object 288230398599632448, subpriority 0.293402, available \
    loc 2233 at target 288230398599631299 is wrong class (1500)
DEBUG: assign force sky: tile 1148, petal 4, class 1600, \
    object 288230398599632236, subpriority 0.179244, available \
    loc 2233 at target 288230398599631299 is wrong class (1500)
DEBUG: assign force sky: tile 1148, petal 4, class 2000, \
    object 288230398599632448, subpriority 0.293402, available \
    loc 2233 at target 288230398599631299 is wrong class (1500)
DEBUG: assign force sky: tile 1148, petal 4, class 2000, \
    object 288230398599632236, subpriority 0.179244, available \
    loc 2233 at target 288230398599631299 is wrong class (1500)
DEBUG: assign force sky: tile 1148, petal 4, class 2100, \
    object 288230398599632448, subpriority 0.293402, available \
    loc 2233 at target 288230398599631299 is wrong class (1500)
DEBUG: assign force sky: tile 1148, petal 4, class 2100, \
    object 288230398599632236, subpriority 0.179244, available \
    loc 2233 at target 288230398599631299 is wrong class (1500)

So for this device, the existing assignment was recognized as both a science target and a standard, and was therefore not considered for bumping to place a sky target.

Small Reference Run

This example is run on cori.nersc.gov, using data files in the project space here:

/project/projectdirs/desi/datachallenge/reference_runs/19.10/targets

After building (and optionally installing) fiberassign you should get an interactive session on a compute node for up to 4 hours:

%> salloc -N 1 -C haswell -A desi --qos=interactive -t 04:00:0

Once that job launches and you are on the compute node, set up some environment variables:

%> export OMP_NUM_THREADS=32
%> export DESI_LOGLEVEL=DEBUG
%> export \
   targetdir=/project/projectdirs/desi/datachallenge/reference_runs/19.10/targets

Now run the assignment using the default footprint tiling from desimodel:

%> time fba_run \
    --targets ${targetdir}/mtl-dark.fits \
    ${targetdir}/sky.fits \
    --dir out_ref_19.10 | tee log_ref_19.10

Make a plot of all tiles (you can also plot only some tiles or petals- see options for fba_plot_results):

%> time fba_plot_results --dir out_ref_19.10

Merge all columns of the original target files into a new set of fiberassign outputs:

%> time fba_merge_results \
--targets ${targetdir}/mtl-dark.fits \
${targetdir}/sky.fits --dir out_ref_19.10

Debugging example based on small reference run

Examining the output from the above, in tile 001149, in petal 6 (at about 10 o’clock, near the periphery of the petal, close to the GFA), we find that location 6525 has a green positioner, meaning that this is both a science target and a standards target. How did this assignment come about? We can learn this by running again, but with:

%>export DESI_DEBUG_LOCATION=6525

%>time fba_run     --targets ${targetdir}/mtl-dark.fits\
${targetdir}/sky.fits --dir out_debug2 |tee log_debug

Then we can examine log_debug for the simultaneous occurrence of 1149 and 6525:

DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385719030 (type=1), total priority 3000.42
DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385719269 (type=1), total priority 3000.37
DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385719156 (type=1), total priority 3000.29
DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385719119 (type=1), total priority 3000.12
DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385719090 (type=1), total priority 3000.02
DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385721102 (type=3), total priority 1500.58
DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385720982 (type=1), total priority 1400.95
DEBUG: targets avail:  tile 1149, location 6525 append ID 288230398385720952 (type=1), total priority 1400.32

The eventual choice has ID 288230398385721102, but this was not the initial assignment for tile 1149, location 6525 since it didn’t have the highest priority.:

DEBUG: target 288230398385719030 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: target 288230398385719269 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: target 288230398385719156 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: target 288230398385719119 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: target 288230398385719090 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: target 288230398385721102 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: target 288230398385720982 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: target 288230398385720952 has available tile / location 1149, 6525 (fiber 3006)
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385719030, priority 3000, subpriority 0.415054
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385719269, priority 3000, subpriority 0.370536
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385719156, priority 3000, subpriority 0.294639
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385719119, priority 3000, subpriority 0.121289
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385719090, priority 3000, subpriority 0.0176536
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385721102, priority 1500, subpriority 0.583245
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385720982, priority 1400, subpriority 0.951584DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) available target 288230398385720952, priority 1400, subpriority 0.317493
DEBUG: find_best: tile 1149, loc 6525, target 288230398385719030 science target with no remaining obs
DEBUG: find_best: tile 1149, loc 6525, target 288230398385719269, type 1 accept with priority = 3000, subpriority = 0.370536, obsremain = 1
DEBUG: find_best: tile 1149, loc 6525, target 288230398385719269, type 1 SELECTED
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) found best object 288230398385719269
DEBUG: assign_tileloc: tile 1149, loc 6525, target 288230398385719269, type 1 N_tile now = 4045 N_petal now = 456

Target 288230398385719269 is selected:

DEBUG: reassign: tile 1149, location 6525, target 288230398385719269 considering for swap...
DEBUG: reassign: tile 1149, location 6525, target 288230398385719269 considering tile indices 0 to 16070
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 1148,443 already considered or swapped
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 1149,6525 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 6911,5272 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 12673,4365 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 18451,9252 already considered or swapped
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 24213,1165 already assigned
DEBUG: redist: tile 1149, location 6525 (fiber 3006), target 288230398385719269 keeping assignment
DEBUG: reassign: tile 5330, loc 6525, target 288230398276670339 avail T/F 1149,421 already considered or swapped
DEBUG: reassign: tile 5330, loc 6525, target 288230398276670339 avail T/F 1149,398 already considered or swapped
DEBUG: reassign: tile 11108, loc 6525, target 288230398377331091 avail T/F 1149,3222 already considered or swapped
DEBUG: reassign: tile 16870, loc 6525, target 288230398381528045 avail T/F 1149,5051 already considered or swapped
DEBUG: reassign: tile 16870, loc 6525, target 288230398381528045 avail T/F 1149,5059 already considered or swapped
DEBUG: reassign: tile 28408, loc 6525, target 288230398377330190 avail T/F 1149,2219 already considered or swapped
DEBUG: reassign: tile 34170, loc 6525, target 288230398381523339 avail T/F 1149,1003 already considered or swapped
DEBUG: reassign: tile 39932, loc 6525, target 288230398289252223 avail T/F 1149,8141 already considered or swapped
DEBUG: assign unused standard: tile 1149, petal 6 location 6525 (fiber 3006) is already assigned
DEBUG: assign force standard: tile 1149, petal 6, location 6525 (fiber 3006), found object 288230398385721102 with weight 1500.58
DEBUG: assign force standard: tile 1149, petal 6, class 1400, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) at target 288230398385719269 is wrong class (3000)
DEBUG: assign force standard: tile 1149, petal 6, class 1500, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) at target 288230398385719269 is wrong class (3000)
DEBUG: assign force standard: tile 1149, petal 6, class 1600, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) at target 288230398385719269 is wrong class (3000)
DEBUG: assign force standard: tile 1149, petal 6, class 2000, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) at target 288230398385719269 is wrong class (3000)
DEBUG: assign force standard: tile 1149, petal 6, class 2998, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) at target 288230398385719269 is wrong class (3000)
DEBUG: assign force standard: tile 1149, petal 6, class 3000, object 288230398385721102, total priority 1500.58, available l
DEBUG: assign force standard: tile 1149, petal 6, class 3000, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) at target 288230398385719269 (subpriority 0.370536) could be bumped
DEBUG: assign force standard: tile 1149, petal 6, class 3000, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) bumping science target 288230398385719269
DEBUG: assign force standard: tile 1149, petal 6, class 3000, object 288230398385721102, total priority 1500.58, available loc 6525 (fiber 3006) bumping science target 288230398385719269
DEBUG: reassign: tile 1149, location 6525, target 288230398385719269 considering for swap...
DEBUG: reassign: tile 1149, location 6525, target 288230398385719269 considering tile indices 0 to 16070
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 1148,443 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 1149,6525 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 6911,5272 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 12673,4365 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 18451,9252 already assigned
DEBUG: reassign: tile 1149, loc 6525, target 288230398385719269 avail T/F 24213,1165 already assigned
DEBUG: unassign_tileloc: tile 1149, loc 6525, target 288230398385719269, type 1 N_tile now = 3966 N_petal now = 433
DEBUG: assign_tileloc: tile 1149, loc 6525, target 288230398385721102, type 1 N_tile now = 3967 N_petal now = 434
DEBUG: assign_tileloc: tile 1149, loc 6525, target 288230398385721102, type 2 N_tile now = 71 N_petal now = 10
DEBUG: assign unused science: tile 1149, petal 6 location 6525 (fiber 3006) is already assigned
DEBUG: assign unused standard: tile 1149, petal 6 location 6525 (fiber 3006) is already assigned

Finally, in search of a standard, target 288230398385719269 is unassigned and 288230398385721102 is used in its place.

Large Run

This large DR7 example is run on cori.nersc.gov, using data files in the project space here:

/project/projectdirs/desi/target/fiberassign/dr7.1/0.10.3-dark

After building (and optionally installing) fiberassign you should get an interactive session on a compute node for up to 4 hours:

%> salloc -N 1 -C haswell -A desi --qos=interactive -t 04:00:0

Once that job launches and you are on the compute node, set up some environment variables:

%> export OMP_NUM_THREADS=32
%> export DESI_LOGLEVEL=DEBUG
%> export \
   targetdir=/project/projectdirs/desi/target/fiberassign/dr7.1/0.10.3-dark

Now run the assignment. This will use about half of the RAM on a cori haswell compute node and take about an hour- but half of that time is writing the output files (something to work on):

%> time fba_run \
--footprint ${targetdir}/input_tiles.fits \
--targets ${targetdir}/mtl_large.fits \
${targetdir}/std_large.fits \
${targetdir}/sky_large.fits \
--dir out_dr7.1_dark | tee log_dr7.1_dark

To save time for this example, only plot one of the petals on each tile:

%> time fba_plot_results \
   --dir out_dr7.1_dark \
   --petals 4

Merge results:

%> time fba_merge_results \
--targets ${targetdir}/mtl_large.fits \
${targetdir}/std_large.fits \
${targetdir}/sky_large.fits \
--dir out_dr7.1_dark