This notebook is available at arjunsavel/simmer

ShARCS Reduction Quickstart

import os

import simmer.insts as i
import simmer.drivers as drivers
from simmer.tests.tests_reduction import download_folder
import simmer.check_logsheet as check
import simmer.add_dark_exp as ad
import simmer.create_config as c

1. Collect your data

For our purposes, we’ll just download some quickstart data.

download_folder(
    "shane_quickstart"
)  # You don't need to run this if you have your own data to reduce.

raw_dir = "shane_quickstart/"  # change this based on where your raw data is.

2. Initializing the instrument

Similarly to the PHARO method, we begin by instantiating the code — but this time, with a different instrument and different data. The downloaded files will take up about a gigabyte of space, and they’ll be placed in a folder in your current directory.

Many specifics (plate scale, FITS headers) are bound as attributes and methods to their respective instrument class.

inst = i.ShARCS()

This instantiates inst as an instrument object prepared to reduced ShARCS data.

3. Creating the config file

Next, you’ll want to set up your config file path, raw data directory, and reduced data directory. In our case, this takes the form of the below. Note that reddir needs to be a directory that’s already been created.

In brief, a “logsheet” generally refers to records taken during observation: usually an Excel spreadsheet or CSV that lists the target you’re observing, the exposure time, the filter, and any comments that might be helpful (e.g. airmass, cloud coverage, adaptive optics performance). In (slight) contrast, an “config file” is strictly a CSV file that contains only the information that the reduction pipeline needs in the correct format: object, exposure time, filter, and reduction mode, with “saturated” being the default reduction mode.

If you have a logsheet you’d like to convert to a config file, run the following cells. Otherwise, skip to the last cell in this section.

First, we check that our logsheet has the correct values input — for example, that there aren’t any missing exposure times that could trip us up later on.

tab = None  # this is a CSV, so we don't have a tab name of interest.
log_name = "logsheet.csv"  # this is the path to the logsheet
failed_tests = check.check_logsheet(inst, log_name, tab)

Missing a coadd.
8/9 tests passed.

All tests passed, so we don’t need to make any changes to our logsheet. Normally, though, this function might suggest changes to be made so that your logsheet can be correctly parsed later on.

ShARCS users often use the automated script on site to take their darks. If this is true for you, you can use the add_dark_exp function to locate your darks in your raw data, read their headers, and write the corresponding data to the logsheet.

ad.add_dark_exp(
    inst,  # no need to run this if you already have rows in your logsheet corresponding to darks.
    log_name,
    raw_dir,
    tab,
)

s1356.fits
s1357.fits
s1356.fits
s1356.fits
s1377.fits
s1366.fits
s1366.fits
s1386.fits
s1386.fits

Finally, we can create our config file from our logsheet!

config_file = "logsheet_test.csv"

c.create_config(log_name, config_file)

/Users/arjunsavel/Desktop/SImMER/src/simmer/create_config.py:47: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  savedf["Filenums"] = pd.Series(filenums)

These last steps establish our config file path and set up the reduced data directory.

# we switch to a different config file here for convenience — one with "method" already filled in.
config_file = "config_shane.csv"

os.mkdir("reduced")  # this stays the same if you don't already have a "reduced" folder

reddir = "reduced/"  # same as above comment

4. Performing the reduction

Finally, run all drivers to perform the reduction!

drivers.all_driver(inst, config_file, raw_dir, reddir)

1137 1138 shane_quickstart/s1138.fits
1358 1359 shane_quickstart/s1359.fits
1141 1142 shane_quickstart/s1142.fits
1132 1133 shane_quickstart/s1133.fits
1261 1262 shane_quickstart/s1262.fits

Running darks: 0%| | 0/2 [00:00<?, ?it/s]

</pre>

Running darks: 0%| | 0/2 [00:00<?, ?it/s]

end{sphinxVerbatim}

Running darks: 0%| | 0/2 [00:00<?, ?it/s]

1146 1147 shane_quickstart/s1147.fits
1384 1385 shane_quickstart/s1385.fits
1149 1150 shane_quickstart/s1150.fits

Running darks: 100%|██████████| 2/2 [00:16<00:00,  8.12s/it]
Running flats: 100%|██████████| 3/3 [00:03<00:00,  1.11s/it]
Running skies: 100%|██████████| 1/1 [00:04<00:00,  4.16s/it]
Running image driver: 100%|██████████| 1/1 [00:08<00:00,  8.34s/it]
Running registration:   0%|          | 0/1 [00:00<?, ?it/s]

Too many images to plot.

Running registration: 100%|██████████| 1/1 [01:31<00:00, 91.91s/it]