<resource schema="ppakm31">
	<meta name="creationDate">2020-02-19T10:03:34Z</meta>
	<meta name="schema-rank">100</meta>

	<meta name="title">PPAKM31 – Optical Integral Field Spectroscopy
		of Star-Forming Regions in M31</meta>
	<meta name="description">
		These observations cover five star-forming regions in the Andromeda galaxy
		(M31) with optical integral field spectroscopy. Each has a field of view of
		roughly 1 kpc across, at 10pc physical resolution.  In addition to the
		calibrated data cubes, we provide flux maps of the Hβ, [OIII]5007, Hα,
		[NII]6583, [SII]6716 and [SII]6730 line emission.  Line fluxes
		have not been corrected for dust extinction.  All data products have
		associated error maps.
	</meta>

	<meta name="subject">andromeda-galaxy</meta>
	<meta name="subject">interstellar-medium</meta>
	<meta name="subject">h-ii-regions</meta>

	<meta name="creator">Kreckel, K.; Tomičić, N.;
		Sandstrom, K.; Groves, B.</meta>
	<meta name="instrument">Potsdam Multi-aperture Spectrophotometer
		(PMAS) with the PPaK fiber-bundle</meta>
	<meta name="facility">Calar Alto 3.5m telescope</meta>

	<meta name="source">2017ApJ...844..155T</meta>
	<meta name="contentLevel">Research</meta>
	<meta name="type">Archive</meta>

	<meta name="coverage.waveband">Optical</meta>

	<meta name="_longdoc" format="rst"><![CDATA[
A Use Case
''''''''''

For a quick idea of what you can do with this data, consider Kewley
et al's starburst criterion (:bibcode:`2001ApJ...556..121K`), which
compares [OIII]/Hb with ([SII]6716+[SII]6730)/Ha.  In this example, we will
investigate where in the this plane our pixels lie.

We will use pyVO_, TOPCAT_, and Aladin_.

Since we need to do somewhat complex operations on the image pixels,
we'll use pyVO to get the source images and convert them into a table of
fluxes.  This table will then be investigated using TOPCAT (and a bit of
Aladin).

*Produce a flux table:* You could use Aladin for going from the images to a
table for fluxes, but this is a bit of drudgery here.  Instead, here is a short
python script using pyVO and numpy; it uses TAP to retrieve the image metadata
and then get the images themselves by plain HTTP.

It then filters out all pixels with an SNR below 5 (with a bit of handwaving,
assuming what's ok in the generally weakest band will be ok in the others)
puts the remaining pixels into a nice relational table.  It finally sends
this table to TOPCAT, so start TOPCAT before running this::

    from astropy.io import fits as pyfits
    from astropy import table, wcs
    from pyvo import samp
    import numpy
    import pyvo


    # change the following to use other ppakm31 fields
    FIELD = "F3"


    def stringify(s):
        """returns s utf-8-decoded if it's bytes, s otherwise.

        (that's working around old astropy votable breakage)
        """
        if isinstance(s, bytes):
            return s.decode("utf-8")
        return s


    def get_image_metadata(field):
        """returns metadata rows from the ppakm31 service for field.

        Access URL and schema are taken from a TOPCAT exploration of the service.
        """
        svc = pyvo.dal.TAPService("http://dc.g-vo.org/tap")

        res = []
        for row in svc.run_sync(
                "select accref, field, imagetitle, bandpassid, cube_link"
                "  from ppakm31.maps"
                "  where field={}".format(repr(field))):
            res.append({
                "bandpassid": stringify(row["bandpassid"]),
                "accref": stringify(row["accref"]),})

        return res


    def get_line_maps(image_meta):
        """returns a dict band->hdus for image_meta as returned by
        get_image_maps.
        """
        res = {}
        for row in image_meta:
            if row["bandpassid"]=='[NII]6583':
                 # we don't need that one later, so skip it
                 continue
            res[row["bandpassid"]] = pyfits.open(row["accref"])
        return res



    if __name__=="__main__":
        line_maps = get_line_maps(get_image_metadata(FIELD))

        # Select pixels with a reasonable SNR; the errors are in extension
        # 1, and we use [OIII]5007, since the other lines ought to be stronger
        snr_cutoff = 5.
        snr_mask = (line_maps["[OIII]5007"][0].data
            > line_maps["[OIII]5007"][1].data*snr_cutoff)

        # The line maps all have identical WCS: use any to turn pixels to pos.
        # This is a somewhat tricky way to get the positions of all valid
        # pixels:
        w = wcs.WCS(header=line_maps["Halpha"][0].header, naxis=2)
        ras, decs = w.wcs_pix2world(
            snr_mask.nonzero()[1], snr_mask.nonzero()[0], 1)

        # Create table with ra & dec positions and our flux measurements.
        t = table.Table([
            ras,
            decs,
            line_maps["[OIII]5007"][0].data[snr_mask],
            line_maps["Hbeta"][0].data[snr_mask],
            line_maps["Halpha"][0].data[snr_mask],
            line_maps["[SII]6716"][0].data[snr_mask],
            line_maps["[SII]6730"][0].data[snr_mask]],
            names=('ra', 'dec', 'OIII_5007','Hbeta','Halpha','SII_6716','SII_6730'))

        # Send the table to TOPCAT via SAMP
        with samp.connection() as conn:
            samp.send_table_to(conn, t, "topcat")

You can change FIELD (or change the script so your flux table has all fields).

*Make a plot of the line ratios:*  To visualise where the data likes
with respect to the Kewley+2001 criterion, once you have the table in TOPCAT,
configure the plot rougly likes this:

* Graphics → Plane Plot
* In X, type the denominator of the Kewley criterion; in case you forget how
  the columns are called, check Views → Column Info.  Anyway, it's
  ``(SII_6716+SII_6730)/Halpha``
* Similarly, for Y, use ``OIII_5007/Hbeta``.
* In the ``Form`` tab, set the shading mode to aux, and set the Aux axis to
  Halpha
* Under ``Axes``, set both axes to log.
* Now add the Kewley criterion with Layers → Add Function Control and adding
  using ``exp10(0.72/(log10(x)-0.32)+1.30)`` as the function to plot; to see
  where this comes from, read :bibcode:`2001ApJ...556..121K`.

The result would be something like this (see below for the black squares):

.. image:: /ppakm31/q/cdl/static/shot1.png

*Sanity check:* To see how the various points on your plot look in reality,
start Aladin and tell it to either some common survey or PPAKM31 images
themselves; you can find the latter in the discovery tree left in Aladin's
window.

Then, configure TOPCAT can make Aladin follow its focus by clicking Views →
Activation Action and then checking “Send Sky Coordinates”.  See if you can
correlate the position in the plot with the visual (or infrared, or whatever)
appearance.

*Compare to known clusters:* To see what known star clusters fare in our plot,
get a catalogue of them.  In TOPCAT, use VO → Cone Search, and in keywords,
enter something like “M31 cluster”; to find what we've plotted above,
J/ApJ/827/33, you'll have to check “description” in the match fields before
sending off the query (they don't mention M31 in their title or subject); but,
really, other catalogues of clusters or H II regions should do just as well.

To quickly get a position to search for, click into your plot; this will fill
out RA and Dec in the dialog, and all that's left is to enter, perhaps, 0.1 into
the radius field (because that's about the FoV of our images) and fire off the
request.

Whatever catalogue you used, the entries will probably not have the fluxes we
need to put them into our flux ratio plot.  So: let's just add
them from our data by a positional crossmatch.  To do that, in TOPCAT's main
window say Joins → Pair Match.  Our pixel size is about 1 arcsec, so the default
match radius is ok.  Select our fluxes as table 1 and the cluster catalogue as
table 2, hit “Go”, and then return to the flux plot.

In there, do Layers → Add position control.  Configure this to plot the match
table, and again the flux expressions from above.  If you then tell TOPCAT to
plot the points as large black squares, you will have reproduced (more or less)
the plot above.


.. _pyVO: https://github.com/astropy/pyvo
.. _Aladin: https://aladin.u-strasbg.fr/aladin.gml
.. _TOPCAT: http://www.star.bris.ac.uk/~mbt/topcat/
	]]></meta>

	<macDef name="bandLabels">{
							"Ha6563": "Halpha",
							"Hb4861": "Hbeta",
							"NII6583": "[NII]6583",
							"OIII5007": "[OIII]5007",
							"SII6716": "[SII]6716",
							"SII6730": "[SII]6730",}</macDef>

	<table id="maps" onDisk="True" mixin="//siap#pgs" adql="True">
		<meta name="_associatedDatalinkService">
			<meta name="serviceId">cdl</meta>
			<meta name="idColumn">cube_id</meta>
		</meta>

		<meta name="description">
			Extracted narrow-band images.
		</meta>
		<mixin
			calibLevel="2"
			collectionName="'PPAKM31'"
			targetClass="'HII'"
		>//obscore#publishSIAP</mixin>

		<column original="centerAlpha"
			displayHint="sf=5"/>
		<column original="centerDelta"
			displayHint="sf=5"/>
		<column original="dateObs"
			description="Representative date of observation.  In reality,
				data contributing to this observation has typically been
				obtained over about a month."/>
		<column name="field" type="text"
			tablehead="Field"
			description="Internal designation of the field being observed."
			verbLevel="1"/>
		<column name="cube_id" type="text"
			ucd="meta.ref.ivoid"
			tablehead="PubDID"
			description="IVOA publisher DID of the originating cube."/>
		<column name="cube_link" type="text"
			tablehead="Cube"
			description="Datalink URL for the cube this line was extracted from.
				This can be used to access the cube using datalink."
			displayHint="type=url"
			verbLevel="1">
			<property name="targetType"
					>application/x-votable+xml;content=datalink</property>
			<property name="targetTitle">Datalink</property>
			<property name="anchorText">Original Cube</property>
		</column>
	</table>

	<coverage>
		<updater sourceTable="maps"/>
			<spatial>6/2623,2666-2667,2709,2752</spatial>
			<temporal>55819.5 55819.5</temporal>
			<spectral>2.95119e-19 4.08734e-19</spectral>
	</coverage>

	<data id="import">
		<property key="previewDir">preview</property>
		<sources pattern="data/*.fits"/>

		<fitsProdGrammar>
			<rowfilter procDef="//products#define">
				<bind key="table">"\schema.data"</bind>
				<bind name="preview_mime">"image/jpeg"</bind>
				<bind name="preview">\standardPreviewPath</bind>
				<bind name="owner">"ari"</bind>
				<bind name="embargo">"2020-05-01"</bind>
			</rowfilter>
			<rowfilter name="exclude_cubes">
				<code>
					if "_cube" in \srcstem:
						raise SkipThis("Cubes are extra")
					yield row
				</code>
			</rowfilter>
		</fitsProdGrammar>

		<make table="maps">
			<rowmaker>
				<apply name="guess_epoch">
					<code>
						mat = re.search("Date observed: ([0-9-]+) to ([0-9-]+)",
							str(@header_["comment"]))
						@meanDate = (dateTimeToMJD(parseDate(mat.group(1))
							)+dateTimeToMJD(parseDate(mat.group(2))))/2
					</code>
				</apply>

				<var name="line">\srcstem.split(".")[0].split("_")[-1]</var>

				<apply procDef="//procs#dictMap">
					<bind key="mapping">\bandLabels</bind>
					<bind key="key">"line"</bind>
				</apply>

				<apply name="compute_cube_name">
					<code>
						@cubename = re.sub(r"_[A-Za-z0-9]+$", "_cube", \srcstem)
					</code>
				</apply>
				
				<apply name="build_spectral_converter">
					<code>
						@specwcs = getWCSAxis(@header_, 3)
					</code>
				</apply>

				<apply procDef="//siap#setMeta">
					<bind key="dateObs">@meanDate</bind>

					<bind key="bandpassId">@line</bind>
					<bind key="bandpassRefval">@specwcs.pixToPhys(1)/1e10</bind>
					<bind key="bandpassLo">@specwcs.pixToPhys(0)/1e10</bind>
					<bind key="bandpassHi">@specwcs.pixToPhys(2)/1e10</bind>

					<bind key="pixflags">'F'</bind>
					
					<bind key="title">\srcstem</bind>
				</apply>

				<apply procDef="//siap#computePGS"/>

				<map key="field">\srcstem.split("_")[2]</map>

				<map key="cube_id"
					>getStandardPubDID("\schema/data/"+@cubename+".fits")</map>
				<map key="cube_link">"%s?ID=%s"%(
					targetTable.tableDef.rd.getById("cdl").getURL("dlmeta"),
					getStandardPubDID("\schema/data/"+@cubename+".fits"))</map>
			</rowmaker>
		</make>
	</data>

	<table id="cubes" onDisk="True" adql="True"
			mixin="//products#table"
			namePath="//obscore#ObsCore">
		<meta name="description">
			Spectral cubes of the fields.  Look for these with full metadata
			in the ivoa.obscore table, obs_collection 'ppakm31 cubes'.
		</meta>

		<mixin
			access_format="'application/fits'"
			access_url="accref"
			calib_level="2"
			dataproduct_subtype="NULL"
			dataproduct_type="'cube'"
			em_res_power="1500"
			em_ucd="'em.wl'"
			em_xel="1016"
			facility_name="'\metaString{facility}'"
			instrument_name="'\metaString{instrument}'"
			o_ucd="'phot.flux.density'"
			obs_collection="'ppakm31 cubes'"
			obs_creator_did="NULL"
			pol_states="NULL"
			pol_xel="NULL"
			preview="accref||'?preview=True'"
			s_ra="s_ra"
			s_dec="s_dec"
			s_region="s_region"
			s_fov="0.07"
			s_pixel_scale="0.0003"
			s_resolution="0.0003"
			t_exptime="NULL"
			t_resolution="2e6"
			t_xel="NULL"
			target_class="'HII'"
			target_name="'M31'"
			>//obscore#publishObscoreLike</mixin>

		<LOOP listItems="obs_id obs_publisher_did access_estsize
				obs_title s_ra s_dec t_min t_max s_region em_min em_max
				s_xel1 s_xel2">
			<events>
				<column original="\item"/>
			</events>
		</LOOP>
	</table>

	<data id="import_cubes">
		<property key="previewDir">preview</property>
		<sources pattern="data/*_cube.fits"/>
		<fitsProdGrammar>
			<rowfilter procDef="//products#define">
				<bind name="table">"ppmakm31.cubes"</bind>
				<bind name="preview_mime">"image/jpeg"</bind>
				<bind name="preview">\standardPreviewPath</bind>
			</rowfilter>
		</fitsProdGrammar>
		<make table="cubes">
			<rowmaker idmaps="*">
				<apply procDef="//siap#computePGS"/>

				<apply name="copy_to_obscore_fields">
					<code>
						vars["s_region"] = result["coverage"]
						vars["s_ra"] = result["centerAlpha"]
						vars["s_dec"] = result["centerDelta"]
					</code>
				</apply>

				<apply name="parse_times">
					<code>
						mat = re.search("Date observed: ([0-9-]+) to ([0-9-]+)",
							str(@header_["comment"]))
						@t_min = dateTimeToMJD(parseDate(mat.group(1)))
						@t_max = dateTimeToMJD(parseDate(mat.group(2)))
					</code>
				</apply>

				<apply name="eval_spect_wcs">
					<code>
						ax = getWCSAxis(@header_, 3)
						@em_min = ax.pixToPhys(1)
						@em_max = ax.pixToPhys(@NAXIS3)
					</code>
				</apply>

				<map key="access_estsize">@prodtblFsize</map>
				<map key="obs_id">\srcstem</map>
				<map key="obs_publisher_did">\standardPubDID</map>
				<map key="obs_title">"Cube of M31 SFR at %.5f %.5f"%(
					@CRVAL1, @CRVAL2)</map>
				<map key="s_xel1">@NAXIS1</map>
				<map key="s_xel2">@NAXIS2</map>
			</rowmaker>
		</make>
	</data>

	<service id="cdl" allowed="dlmeta,dlget,static">
		<meta name="title">PPAKM31 cube datalink service</meta>
		<property key="staticData">data</property>
		<datalinkCore>
			<descriptorGenerator procDef="//soda#fits_genDesc"
				name="genFITSDesc">
				<bind key="accrefPrefix">'\schema/data'</bind>
				<bind key="descClass">FITSProductDescriptor</bind>
				<bind key="contentQualifier">"#cube"</bind>
			</descriptorGenerator>

			<FEED source="//soda#fits_standardDLFuncs" spectralAxis="3"/>

			<metaMaker name="make_map_links" semantics="#derivation">
				<setup>
					<par name="bands">\bandLabels</par>
				</setup>
				<code>
					template = re.sub("_cube.fits", "_%s.fits", descriptor.accref
						).split("/", 1)[-1]
					
					for band, bandLabel in bands.items():
						yield descriptor.makeLinkFromFile(
							template%band,
							description=f"A map in {bandLabel} derived from this cube.",
							localSemantics=f"band{bandLabel}",
							contentQualifier="#image")
				</code>
			</metaMaker>

		</datalinkCore>
	</service>

	<service id="i" allowed="fixed">
		<template key="fixed">res/maplist.html</template>
		<meta name="shortName">ppakm31 web</meta>
		<publish render="fixed" sets="local"/>
		<fixedQueryCore
			query="select * from ppakm31.maps order by imageTitle">
			<outputTable namePath="maps" autoCols="imageTitle,centerAlpha,
				centerDelta,bandpassId,accref,accsize,cube_link">
			</outputTable>
		</fixedQueryCore>
	</service>

	<data id="pub" auto="False">
		<publish services="i"/>
		<make table="maps"/>
		<make table="cubes"/>
	</data>

	<regSuite title="ppakm31 regression">

		<regTest title="ppakm31 overview shows reasonable data">
			<url>i/fixed</url>
			<code><![CDATA[
				self.assertHasStrings(
					'title="Datalink URL for the cube',
					'<td>PPAK_M31_F4_OIII5007</td>',
					'<td>11.24364</td>',
					'<td>[OIII]5007</td>',
					'getproduct/ppakm31/data/PPAK_M31_F4_OIII5007.fits',
					'ppakm31/q/cdl/dlmeta?ID=ivo://org.gavo.dc/~?ppakm31'
						'/data/PPAK_M31_F4_cube.fits">Original Cube')
			]]></code>
		</regTest>

		<regTest title="ppakm31 map preview looks like a jpeg">
			<url>/getproduct/ppakm31/data/PPAK_M31_F4_OIII5007.fits?preview=True</url>
			<code>
				self.assertHasStrings("JFIF", "*456789:CDEFGHI")
			</code>
		</regTest>

		<regTest title="ppakm31 cube preview looks like a jpeg">
			<url>/getproduct/ppakm31/data/PPAK_M31_F4_cube.fits?preview=True</url>
			<code>
				self.assertHasStrings("JFIF", "*456789:CDEFGHI")
			</code>
		</regTest>

		<regTest title="ppamk31 cube datalink response looks right">
			<url ID="ivo://org.gavo.dc/~?ppakm31/data/PPAK_M31_F4_cube.fits"
				>cdl/dlmeta</url>
			<code>
				rows = self.getVOTableRows()
				for r in rows:
					if r["semantics"]=="#derivation" and "OIII5007" in r["access_url"]:
						self.assertEqual(r["description"],
							"A map in [OIII]5007 derived from this cube.")
						self.assertEqual(r["local_semantics"],
							"band[OIII]5007")
						break
				else:
					self.fail("OIII line map missing as #derivation")
					
				self.assertHasStrings(
					'MIN value="3.68516589077e-07"', # of the BAND param
					'MAX value="11.2436402393 41.9189435754 0.0436407329"', # CIRCLE
					'ucd="pos.cartesian.z;instr.pixel"' # auto pixel 3
					)
			</code>
		</regTest>
	</regSuite>
</resource>
