<resource schema="prdust">
	<meta name="title">Bayestar17 3D Dust Mapping with Pan-STARRS 1</meta>
	<meta name="description" format="rst">
		A 3D map of interstellar dust reddening, covering three
		quarters of the sky (declinations greater than -30 degrees) out to a
		distance of several kiloparsecs. The map is based on high-quality stellar
		photometry of 800 million stars from Pan-STARRS 1 and 2MASS.

		The map is available for five HEALPix levels (6 through 10), published
		here as separate tables, map6 through map10.  A union of the coverage is
		provided as map_union.  Use its coverage column to match against other
		tables.

		See http://argonaut.rc.fas.harvard.edu/ for more details.
	</meta>

	<meta name="_longdoc" format="rst">
		These tables lets you do dereddening of photometry provided you have:

		* Positions
		* An estimate for the distance modulus
		* Some idea of how to get from E(B-V) to your extinction.
		
		You will usually join two of the arrays in map_union to your table.
		Because of boring technicalities, that is a bit ideomatic; you need to
		compute an order-10 healpix and see if that is contained in this tables
		coverage.  In a cartoon::

		  SELECT
		    mytable.*,
		    best_fit, grdiagnostic
		  FROM mytable
		  JOIN prdust.map_union
		  ON (1=ivo_interval_has(
		    CAST(ivo_healpix_index(10, l, b) AS INTEGER), coverage))

		Yes, you have to use galactic longitude and latitude in the arguments
		of ivo_healpix_index; and the odd cast is necessary because coverage
		contains normal integer intervals.

		If you have equatorial coordinates ``ra`` and ``dec``, on this service
		there's the ``gavo_transform`` user defined function to your rescue.  Just
		write::

		  CAST(ivo_healpix_index(10,
		    gavo_transform('ICRS', 'GALACTIC', POINT(ra, dec))
		      ) AS INTEGER), coverage)

		in the second argument of ``ivo_interval_has`` above.
		
		The next step is to estimate a distance modulus.  If you have a good
		parallax estimate and your TAP service supports the ``in_unit`` UDF,
		you might get away with::

			5*log10(1./in_unit(parallax, 'arcsec'))-5 AS dist_mod

		You will certainly want to do better than this in almost all science
		use cases (see :bibcode:`2018arXiv180409376L` for a through discussion
		in the context of Gaia DR2).

		From ``dist_mod``, you can compute the distance bin in the ``best_fit``
		and ``grdiagostic`` columns with::

			ROUND((dist_mod-4)*2)+1 AS dist_mod_bin

		With this, you can add an E(B-V) to a local table somewhat like this::

			SELECT
				q.*,
				best_fit[dist_mod_bin] as eb_v
			FROM (
				SELECT (
					CAST(ivo_healpix_index(10, l, b) AS INTEGER) AS cat_hpx,
					ROUND((dist_mod-4)*2)+1 AS dist_mod_bin,
					mytable.*
				FROM mytable) AS q
			JOIN prdust.map_union
			ON (1=ivo_interval_has(cat_hpx, coverage))
			WHERE grdiagnostic[dist_mod_bin]&lt;1.2

		To turn E(B-V) into extinctions, there are several recipes.  See, for
		instance :bibcode:`1998ApJ...500..525S`.

		You'll find a worked-out example on this at
		https://blog.g-vo.org/deredden-using-tap/.
	</meta>
	<meta name="subject">interstellar-extinction</meta>
	<meta name="subject">interstellar-dust</meta>
	<meta name="subject">galaxy-structure</meta>

	<meta name="creationDate">2018-03-03T12:22:00</meta>
	<meta name="schema-rank">100</meta>
	<meta name="creator">Green, G.M.; Schlafly, E.F.; Finkbeiner, D.;
		Rix, H.-W.; Martin, N.; Burgett, W.; Draper, P.W.; Flewelling, H.;
		Hodapp, K.; Kaiser, N.; Kudritzki, R.-P.; Magnier, E.A.; Metcalfe, N.;
		Tonry, J.L.; Wainscoat, R; Waters, C.</meta>
	<meta name="instrument">Pan-STARRS 1</meta>

	<meta name="source">2018arXiv180103555G</meta>
	<meta name="contentLevel">Research</meta>
	<meta name="type">Survey</meta>

	<meta name="coverage.waveband">Optical</meta>
	<meta name="coverage.waveband">Infrared</meta>

	<FEED source="//procs#license-cc0" what="3D Dust Maps"/>

	<STREAM id="mapcolumns">
		<primary>healpix</primary>
		<index columns="coverage" method="GIST"/>

		<column name="healpix" type="integer" required="True"
			ucd="pos.healpix"
			tablehead="HEALPix"
			description="The healpix (of order \order) of which the extinction
				data is given.  This is in Galactic l, b."
			verbLevel="1"/>
		<column name="coverage" type="int4range"
			ucd="phys.angArea"
			tablehead="Coverage"
			description="Area (as order-10-healpix interval) covered by
				this healpix."
			verbLevel="30"/>
		<column name="converged" type="smallint" required="True"
			ucd="meta.code"
			tablehead="Converged?"
			description="1 if the line-of-sight reddening fit converged,
				0 otherwise."
			verbLevel="15"/>
		<column name="dm_reliable_min"
			unit="mag"
			tablehead="Min. Rel."
			description="Minimum reliable distance modulus."
			verbLevel="15"/>
		<column name="dm_reliable_max"
			unit="mag"
			tablehead="Max. Rel."
			description="Maximum reliable distance modulus."
			verbLevel="15"/>
		<column name="n_stars" type="integer"
			ucd="meta.number"
			tablehead="# Stars"
			description="Number of stars used to fit the line-of-sight reddening."
			verbLevel="15">
			<values nullLiteral="-1"/>
		</column>
		<column name="n_good" type="integer"
			ucd="meta.number"
			tablehead="# Good"
			description='The number of stars in the sightline with good
		   convergence, and which passed a cut on Bayesian evidence (termed
		   "good" stars).'
			verbLevel="15">
			<values nullLiteral="-1"/>
		</column>
		<column name="n_dwarfs" type="integer"
			ucd="meta.number"
			tablehead="# Dwarf"
			description='The number of "good" stars which are inferred
				to be Main-Sequence stars.'
			verbLevel="15">
			<values nullLiteral="-1"/>
		</column>
		<column name="grdiagnostic" type="real[31]"
			tablehead="G-R conv."
			description="Gelman-Rubin convergence diagnostic in each HEALpix;
				a HEALpix can be considered good if diagnostic is below 1.2."
			verbLevel="25"/>
		<column name="best_fit" type="real[31]"
			unit="mag" ucd="phys.absorption"
			tablehead="E(B-V) for 4 .. 19 mag (0.5 mag bin size)"
			description="The best-fit (maximum proability density) line-of-sight
				reddening, in units of SFD-equivalent E(B-V), in each distance bin;
				the bins are 4 .. 19 mag of distance modulus, spaced 0.5 mags."
			verbLevel="1"
			note="e"/>
		<!-- the following should really be type="real[18][31]" -->
		<column name="samples" type="real[558]"
			unit="mag" ucd="phys.absorption;src.sample"
			tablehead="18 samples/bin of E(B-V) from MCMC chain"
			description="18 samples of E(B-V) in each distance bin drawn from
				the MCMC chain; the bins are 4 .. 19 mag of distance modulus,
				spaced 0.5 mags.  This information can be used to estimate
				the reliability of best_fit (e.g., a standard deviation)."
			verbLevel="15"
			note="e"/>

		<meta name="note" tag="e"><![CDATA[
			Note that the reddening is given in units of “SFD-like” E(B-V). They are
			meant to be comparable to the E(B-V) reported by
			:bibcode:`1998wfsc.conf..297S`. Due to a subsequent recalibration of SFD
			(:bibcode:`2011ApJ...737..103S`), the true E(B-V) is 0.884 times the
			reddening reported here.
			
			To convert to extinction in Pan-STARRS 1 or 2MASS passbands, multiply the
			reddening reported here by

			==  =====
			g   3.384
			r   2.483
			i   1.838
			z   1.414
			y   1.126
			J   0.650
			H   0.327
			Ks  0.161
			==  =====

			The issue is discussed in somewhat more detail in
			http://argonaut.skymaps.info/usage#units.
		]]></meta>

	</STREAM>

	<STREAM id="tabletemplate">
		<table id="map\order" onDisk="True" adql="True">
			<meta name="description">
				The HEALPix map for order \order (nside=\nside); the pixel scale
				is about \pixelscale.

				In ADQL, all array indices are 1-based.
			</meta>

			<LFEED source="mapcolumns"/>
		</table>
	</STREAM>

	<LOOP source="tabletemplate">
		<csvItems>
			order, nside, pixelscale
			6,     64,    55'
			7,    128,    27'
			8,    256,    14'
			9,    512,    6.9'
			10,  1024,    3.4'
		</csvItems>
	</LOOP>

	<rowmaker id="rmk">
		<simplemaps>healpix:HEALPIX_INDEX, converged:CONVERGED,
			dm_reliable_min:DM_RELIABLE_MIN, dm_reliable_max:DM_RELIABLE_MAX,
			n_stars:N_STARS, n_good:N_GOOD, n_dwarfs: N_DWARFS,
			grdiagnostic:GRDIAGNOSTIC, best_fit:BEST_FIT, samples: SAMPLES
		</simplemaps>

		<apply name="compute_coverage">
			<code>
				order = int(round(math.log(@NSIDE)/math.log(2)))
				lowerBound = @HEALPIX_INDEX*2**(10-order)
				upperBound = (@HEALPIX_INDEX+1)*2**(10-order)
				# todo: make a MOC out of that when we have them
				result["coverage"] = '[%s,%s)'%(lowerBound, upperBound)
			</code>
		</apply>
		
	</rowmaker>

	<data id="import">
		<sources pattern="data/*.fits"/>
		<embeddedGrammar isDispatching="True">
			<iterator>
				<code>
					from gavo.utils import pyfits

					hdus = pyfits.open(self.sourceToken)
					fitsTable = hdus[1].data
					names = [n for n in fitsTable.dtype.names]
					for row in fitsTable:
						role = str(int(round(math.log(row["NSIDE"])/math.log(2))))
						yield role, dict(zip(names, row))
				</code>
			</iterator>
		</embeddedGrammar>
		<LOOP listItems="6 7 8 9 10">
			<events>
				<make table="map\item" role="\item" rowmaker="rmk"/>
			</events>
		</LOOP>
	</data>

	<table id="map_union" onDisk="True" adql="True">
		<meta name="title">Joined Bayestar17 3D Dust Map</meta>
		<meta name="description">This is a view of the dust maps in the
			five orders, generated by splitting the larger pixels from the higher
			orders into HEALPixes of order 10.  This means, in particular,
			that the spatial resolution for all pixels with original_order!=10.
			Use this for convenient joins to other tables.
		</meta>

		<publish sets="local,ivo_managed"/>
		
		<FEED source="mapcolumns" order="10"/>
		<column original="healpix"
			description="The healpix (in galactic l, b) for which this data applies.
			This is of the order given in the hpx_order column, and thus it's not
			usually useful for querying.  Query against the coverage column
			instead."/>
		<column name="hpx_order" type="smallint" required="True"
			ucd="meta.number;pos.healpix"
			tablehead="Src. order"
			description="The HEALPix order healpix is given for.  The
				dust map is not regularly sampled, which is why orders
				vary over the sky.  For querying, it is simpler to look
				at coverage than at healpix+order."/>
		<viewStatement>
			CREATE VIEW \curtable AS (SELECT \colNames FROM (
					SELECT *, 10 AS hpx_order FROM \schema.map10
				UNION ALL
					SELECT *, 9 AS hpx_order FROM \schema.map9
				UNION ALL
					SELECT *, 8 AS hpx_order FROM \schema.map8
				UNION ALL
					SELECT *, 7 AS hpx_order FROM \schema.map7
				UNION ALL
					SELECT *, 6 AS hpx_order FROM \schema.map6
			) AS q)
		</viewStatement>
	</table>

	<coverage>
		<!-- updater:

		select smoc('6/' || string_agg(format('%s', healpix), ',')) from (
			select healpix from prdust.map6 union
			select distinct healpix/4 from prdust.map7 union
			select distinct healpix/16 from prdust.map8 union
			select distinct healpix/64 from prdust.map9 union
			select distinct healpix/256 from prdust.map10) as q -->
		<spatial>0/0-2,5-6,9 1/15,17,33,35,42 2/53-55,58-59,76-77,79,122,129,131,137,139,162,172,174-175 3/207,209-211,227,229-231,261,263,313,315,494-495,506-507,509-513,515,521,523,640,642,654-655,694-695 4/821-823,833-835,903,907,913-915,1040-1041,1043,1049,1077,1079,1085,1249,1251,1257,1259,1930-1931,1968,1970-1971,1974-1975,2018-2019,2022-2023,2034-2035,2056-2057,2059,2081,2083,2180-2181,2183,2189,2191,2213,2215,2221,2572,2574-2575,2608,2610-2611,2666-2667,2770 5/3191,3197,3199,3303,3307,3309-3311,3329-3331,3622-3623,3625-3627,3649-3651,4169,4204-4205,4207,4304-4305,4307,4313,4315,4337,4349,4351,4693,4993,4995,5001,5003,5025,5027,5033,5035,7712,7714-7715,7736,7738-7739,7742-7743,7876,7878-7879,7890-7891,7894-7895,8063,8066-8067,8070-8071,8082-8083,8086-8087,8130-8131,8134-8135,8233,8235,8321,8357,8359,8365,8728-8729,8731,8753,8755,8761,8763,8849,8851,8857,8859,8893,10266,10292-10294,10344,10346-10347,10436,10438,10458,10658-10659,10662-10663,10680,10682-10683,11074-11075,11084,11086-11087,11098-11099,11103,11264-11266,11268-11269,11280 6/12757,12759,12787,12793,12795,13133-13135,13203,13206-13207,13209,13211,13233-13235,13313,13315,14429,14431,14439,14445-14447,14479,14487,14497-14499,14593-14595,16469,16471,16673,16684-16685,16687,16773,16805,16824-16825,16827,17172-17173,17224-17225,17227,17249,17357,17393,17401,17403,18769,18771,18780-18781,18783,18805-18807,18813,18815,18901,19968-19969,19971,19977,19979,20001,20009,20011,20097,20099,20105,20107,20137,20139,30760,30762,30854,30874,30878-30879,30950-30951,30962,31146-31147,31150,31508,31510-31511,31554-31555,31558-31559,31571,32238-32239,32249-32251,32515,32518,32530-32531,32534-32535,32928-32929,32931,32939,33293,33295,33425,33427,33469,33471,34837,34839,34845,34869,34920-34921,34923,35009,35011,35017,35019,35041,35043,35049,35051,35393,35395,35401,35403,35525,35527,35569,35580-35581,35583,41034,41056,41058-41059,41068,41070,41083,41180,41182-41183,41382,41402,41758-41759,41802,41824,41826-41827,41830,41836,41838-41839,41850-41851,41854-41855,42624,42626-42627,42642-42643,42646-42647,42698,42726,42746-42747,44290-44291,44314-44315,44318,44340,44342-44343,44385-44388,44390-44391,44410-44411,45068-45070,45084-45085,45088-45090,45092,45124-45125,45128,45136
		</spatial>
	</coverage>

	<data id="make_union_view">
		<make table="map_union"/>
	</data>
</resource>
