<resource schema="hdgaia">
	<meta name="creationDate">2018-09-20T08:28:29Z</meta>
	<meta name="schema-rank">100</meta>

		<meta name="title">The Henry Draper Catalog with Gaia IDs</meta>
	<meta name="description">
		This is the Henry Draper catalog (HD, Cannon &amp; Pickering 1918-1924) as
		distributed by the Astronomical Data Center in 1989 (Vizier III/135A),
		with Gaia DR2 source_ids and positions added.  The link to modern Gaia
		DR2 was done through Fabricius et al's match between HD and Tycho 2 (Vizier
		IV/25), TGAS to match Tycho 2 and Gaia DR1, and Gaia DR2 to match against
		Gaia DR1.
	</meta>

	<meta name="subject">surveys</meta>
	<meta name="subject">stellar-spectral-types</meta>
	<meta name="subject">astronomical-object-identification</meta>
	
	<meta name="source">1924AnHar..99....1C</meta>

	<meta name="creator">Cannon, A.J.; Fabricius, C.; Demleitner, M.</meta>

	<meta name="contentLevel">Research</meta>
	<meta name="type">Survey</meta>

	<meta name="copyright"><![CDATA[
			This work has made use of data from the European Space Agency (ESA)
			mission `Gaia <https://www.cosmos.esa.int/gaia>`_, processed by the
			Gaia `Data Processing and Analysis Consortium (DPAC)
			<https://www.cosmos.esa.int/web/gaia/dpac/consortium>`_. Funding for the
			DPAC has been provided by national institutions, in particular the
			institutions participating in the Gaia Multilateral Agreement.
	]]></meta>

	<meta name="coverage.waveband">Optical</meta>

	<meta name="_longdoc" format="rst"><![CDATA[
		This resource grew out of the realisation just how hard it is to
		do useful things with HD numbers in the VO.  In provides the original
		Henry Draper catalog together with Gaia DR2 source ids for the stars
		identified in Fabricius et al's Tycho 2-HD match
		(:bibcode:`2002A&A...386..709F`)

		It was generated as follows (cf. `sources`_):

		(a) create a database table for the HD catalog with empty columns
		    for the Gaia counterparts.  While doing this, precess the
		    B1900 positions to J2000.  We haven't bothered with diffrences
		    in the reference catalogs; at the 1 arcminute precision of the HD,
		    this doesn't matter at all. All astrometry coming from HD is
		    for debugging only.

		(b) Get a mapping from HD numbers to standard Tycho-2 ids from
		    VizieR::
		    	
		      select HD, n_HD, n_TYC,
		        TYC1 || '-' || TYC2 || '-' || TYC3 as tycho_id
		      from "IV/25/tyc2_hd"

		(c) Turn the Tycho 2 identifier to a Gaia DR2 source_id using the
		    ``dr2_tycho_best_neighbour`` table.  It does not exist on VizieR,
		    so we used the ARI-Gaia service::

		      select hd, n_hd, n_tyc, source_id
		      from gaiadr2.tycho2_best_neighbour
		      join TAP_UPLOAD.tyc
		      on (tycho_id=original_ext_source_id)

		    After this procedure, HD 45900, HD 260686, and HD 121169 did not
		    have unique matches.  It seemed excessive to complicate the data
		    model just for these three objects.  We have randomly dropped one
		    possible match for each of them; in addition, HD 121169 should have
		    had an second match, which did not work out in Gaia DR2.  These
		    doubtful cases can be identified by n_TYC=2.

		(d) Add a Gaia eDR3 source_id by matching the DR2 source_id
		    ``gaiaedr3.dr2neighbourhood``.  This frequently has
		    multiple eDR3 objects for a DR2 source_id.  In this case, we blindly
		    chose the brightest match, ignoring, for instance, component
		    information that might be present in the DM number.  Again, relatively
		    close binaries may be trouble in this catalogue.

		(e) Complete the source_id, gaia_ra, gaia_dec, n_hd, and n_tyc
		    columns of the table created in step (a) with the data obtained
		    in steps (c) and (d) using the local edr3lite table.

		.. _sources: http://svn.ari.uni-heidelberg.de/svn/gavo/hdinputs/hdgaia
	]]></meta>

	<table id="main" onDisk="True" adql="True" mixin="//scs#q3cindex">
		<mixin>//scs#pgs-pos-index</mixin>
		<primary>hd</primary>
		<index columns="spectral"/>
		<index columns="m_v"/>
		<nrows>272150</nrows>

		<column name="hd" type="integer" required="True"
			ucd="meta.id;meta.main"
			description="HD number for this object"
			verbLevel="1">
			<values min="" max=""/>
		</column>
		<column name="dm" type="text"
			ucd="meta.id.cross"
			tablehead="DM"
			description="Durchmusterung identifier, taken either from the
				Bonner Durchmusterung (BD), Cordoba Durchmusterung (CD), or
				Cape Photographic Durchmusterung (CPD); stars not given in any
				of them are referenced by their AGK number."
			verbLevel="15" note="d"/>

		<column name="ra_orig" type="text"
			ucd="pos.eq.ra"
			description="Original right ascension for equinox J1900.0.  Note that
				this position is only good to about an arcminute."
			verbLevel="21"/>
		<column name="dec_orig" type="text"
			ucd="pos.eq.dec"
			description="Original declination for equinox J1900.0. Note that this
				position is only good to about an arcminute."
			verbLevel="21"/>
		<column name="ra_orig_2000"
			ucd="pos.eq.ra"
			description="Original right ascension for equinox J2000.0 (precession
				done by DC staff).  Note that this position is only good to about
				an arcminute."
			verbLevel="21"/>
		<column name="dec_orig_2000"
			ucd="pos.eq.dec"
			description="Original declination for equinox J2000.0 (precession done
				by DC staff). Note that this position is only good to about an
				arcminute."
			verbLevel="21"/>

		<column name="mv_meas" type="char" required="True"
			ucd="meta.code"
			tablehead="m_V?"
			description="'0' if the m_v is a measured value, '1' if it's a value
				inferred from m_pg and the spectral type."
			verbLevel="25"/>
		<column name="m_v"
			unit="mag" ucd="phot.mag;em.opt.V"
			tablehead="m_V"
			description="Photovisual magnitude or the star, except for magnitudes
				>= 20, which are object type flags (see note)."
			verbLevel="5"
			note="m">
			<values min="-1.58" max="50.0"/>
		</column>
		<column name="mv_comb" type="char"
			ucd="meta.code"
			tablehead="Comb?"
			description="C if m_v is a combined magnitude (with a neighbouring
				star given in this catalogue), NULL otherwise."
			verbLevel="15">
			<values nullLiteral="-"/>
		</column>

		<column name="mpg_meas" type="char" required="True"
			ucd="meta.code"
			tablehead="Meas?"
			description="'0' if the m_pg is a measured value, '1' if it's a value
				inferred from m_v and the spectral type."
			verbLevel="25"/>
		<column name="m_pg"
			unit="mag" ucd="phot.mag;em.opt.B"
			tablehead="m_pg"
			description="Photographic magnitude or the star, except for magnitudes
				>= 20, which are object type flags (see note)."
			verbLevel="5"
			note="m"/>
		<column name="mpg_comb" type="char"
			ucd="meta.code"
			tablehead="Comb?"
			description="C if m_pg is a combined magnitude (with a neighbouring
				star given in this catalogue), NULL otherwise."
			verbLevel="15">
			<values nullLiteral=" "/>
		</column>

		<column name="spectral" type="text"
			ucd="src.sptype"
			tablehead="Spectral"
			description="Spectral type, given in upper and lower case as in the
				published catalog."
			verbLevel="1"/>
		<column name="pg_intensity" type="smallint"
			ucd="meta.code.qual"
			tablehead="Intensity"
			description="Photographic intensity of the spectrum estimated by Annie
				Cannon.  The faintest spectra which could be classified with certainty
				were assigned a value of 1, while the densest are given as 10.  For
				stars having two intensities in the published catalog, only the
				first is given here."
			verbLevel="25">
			<values nullLiteral="-1"/>
		</column>
		<column name="remarks" type="char"
			ucd="meta.code"
			tablehead="Rem"
			description="Remarks.  See note for what the letters mean."
			verbLevel="15"
			note="r">
			<values nullLiteral=" "/>
		</column>

		<column name="n_hd" type="smallint"
			ucd="meta.number"
			tablehead="#HD"
			description="Number of HD stars matched to a Tycho 2 source
				by Fabricius et al, 2002A&amp;A...386..709F."
			verbLevel="15">
			<values nullLiteral="0"/>
		</column>
		<column name="n_tyc" type="smallint"
			ucd="meta.number"
			tablehead="#Tyc"
			description="Number of Tycho 2 stars matched to a HD source
				by Fabricius et al, 2002A&amp;A...386..709F."
			verbLevel="15">
			<values nullLiteral="0"/>
		</column>

		<column name="source_id" type="bigint"
			ucd="meta.id.cross"
			tablehead="DR2 source_id"
			description="Gaia DR2 source_id matching the HD object through
				Fabricius' Tycho 2 match and Gaia DR2's tycho2_best_neighbour.
				For four ambiguous matches, a unique match was randomly
				chosen (see reference URL)."
			verbLevel="15">
			<values nullLiteral="-1"/>
		</column>

		<column name="source_id3" type="bigint"
			ucd="meta.id.cross"
			tablehead="Gaia source_id"
			description="Gaia eDR3 source_id matching the HD object through
				Fabricius' Tycho 2 match and Gaia DR2's tycho2_best_neighbour,
				and then Gaia eDR3's dr2_best_neighbour."
			verbLevel="1">
			<values nullLiteral="-1"/>
		</column>

		<column name="gaia_ra" type="double precision"
			unit="deg" ucd="pos.eq.ra;meta.main"
			tablehead="RA Gaia"
			description="ICRS Right Ascension from Gaia eDR3 (i.e., Epoch J2016)"
			verbLevel="5" displayHint="sf=9"/>
		<column name="gaia_dec" type="double precision"
			unit="deg" ucd="pos.eq.dec;meta.main"
			tablehead="Dec Gaia"
			description="ICRS Declination from Gaia eDR3 (i.e., Epoch J2016)"
			verbLevel="5" displayHint="sf=9"/>

		<meta name="note" tag="d">
			The Durchmusterung identifier includes, in place 10, a component
			identifier if there are two or more HD stars with the same DM number.
			For multiple systems included in the Washington Catalog of Visual Double
			Stars (WDS, Worley 1984) the same designations are given.

			In place 11, there is a lower case letter identification for BD
			supplemental stars (Warren and Kress 1980).
		</meta>

		<meta name="note" tag="m">
			The following special values photovisual magnitude indicate unusual
			object types:

			=====  ====================================
			20.0   nebula   (Neb  in published catalog)
			30.0   variable (var. in published catalog)
			40.0   nova     (Nov. in published catalog)
			50.0   cluster  (Cl.  in published catalog)
			NULL   no value in published catalog
			=====  ====================================

		</meta>
		
		<meta name="note" tag="r">
			The following codes are used in the remarks:

			D
				Entry deleted because it is a duplicate
				or does not refer to a real star (see
				Table 4)
			E
				Image is at edge of plate
			G
				Position and BD number taken directly
				from AGK and precessed to 1900
			M
				Multiple images used.  Only one intensity
				is included (both are in the published
				catalog, however)
			R
				There is a remark for this star in the
				published catalog
			\*
				Spectral type refers to a cluster, nebula
				or nebulous star.  (Occurs in the Magellanic
				Cloud fields only.)
		</meta>
	</table>

	<coverage>
		<updater sourceTable="main" mocOrder="4"/>
		<spatial>0/0-11</spatial>
	</coverage>

	<data id="import">
		<!-- import is three-step: first, get original data and leave
		space for our cross-match info, then to the matches with
		add_dr2 and add_dr3 via recreateAfter -->
		<recreateAfter>add_dr2</recreateAfter>

		<sources pattern="data/raw.dat"/>

		<columnGrammar>
			<colDefs>
				hd: 1-6
				dm: 7-18
				ra_hours: 19-20
				ra_decimins: 21-23
				dec_orig: 24-28
				mv_meas: 29
				m_v: 30-34
				mv_comb: 35
				mpg_meas: 36
				m_pg: 37-41
				mpg_comb: 42
				spectral: 43-45
				pg_intensity: 46-47
				remarks: 48
			</colDefs>
		</columnGrammar>

		<make table="main">
			<rowmaker idmaps="*">
				<apply name="add_rough_coords">
					<setup>
						<code>
							from gavo import stc
							toICRS = stc.getSimple2Converter(
								stc.parseSTCS("Position FK4 B1900.0"),
								stc.parseSTCS("Position ICRS"))
						</code>
					</setup>
					<code>
						# RA: first turn into minutes of time...
						rafloat = float(@ra_hours)*60+float(@ra_decimins)/10
						# ... and then into degrees
						rafloat = rafloat/4.
						decfloat = dmsToDeg(@dec_orig.replace(" ", "0"), "")
						@ra_orig = "%s %s.%s"%(
							@ra_hours,
							@ra_decimins[:-1],
							@ra_decimins[-1])
						@ra_orig_2000, @dec_orig_2000 = toICRS(rafloat, decfloat)
					</code>
				</apply>
				<map key="m_pg" nullExcs="ValueError"/>
				<map key="n_hd"></map>
				<map key="n_tyc"></map>
				<map key="gaia_ra">None</map>
				<map key="gaia_dec">None</map>
				<map key="source_id">None</map>
			</rowmaker>
		</make>
	</data>

	<table id="dr2match" onDisk="true" namePath="main" temporary="True">
		<primary>hd</primary>
		<LOOP listItems="hd n_hd n_tyc source_id">
			<events>
				<column original="\item"/>
			</events>
		</LOOP>
	</table>

	<data id="add_dr2">
		<recreateAfter>add_dr3</recreateAfter>
		<sources pattern="data/dr2-match.vot"/>

		<voTableGrammar/>

		<make table="dr2match">
			<script type="postCreation" lang="SQL">
				UPDATE \schema.main AS dest SET
					source_id=mat.source_id,
					n_hd=mat.n_hd,
					n_tyc=mat.n_tyc
				FROM \schema.dr2match AS mat
				WHERE mat.hd=dest.hd;
			</script>

			<rowmaker idmaps="*">
				<simplemaps>
					hd: HD,
					n_hd: n_HD,
					n_tyc: n_TYC,
					source_id: source_id
				</simplemaps>
			</rowmaker>
		</make>
	</data>

	<table id="dr3match" onDisk="true" namePath="main" temporary="True">
		<index columns="dr2_source_id"/>
		<column name="dr2_source_id" type="bigint" required="True"/>
		<column name="dr3_source_id" type="bigint" required="True"/>
	</table>

	<data id="add_dr3" auto="False">
		<sources pattern="data/dr3-match.vot"/>
		<voTableGrammar/>

		<make table="dr3match">
			<!-- the dr2 -> dr3 match is quite ambiguous.  We break the
				ambiguity by choosing the brightest DR3 object among multiple
				matches -->
			<script type="postCreation" lang="SQL">
				CREATE TEMP TABLE uniqued_dr3match AS (
					SELECT dr2_source_id, dr3_source_id from (
						SELECT dr2_source_id, dr3_source_id, row_number()
							OVER (PARTITION BY dr3_source_id
								ORDER BY phot_g_mean_mag desc) as rank
						FROM (
							SELECT a.* FROM
								dr3match AS a
								JOIN \schema.main ON (dr2_source_id=source_id)) AS i
						JOIN gaia.edr3lite ON (dr3_source_id=source_id)) AS q
					WHERE rank=1);

				UPDATE \schema.main AS dest SET
					source_id3=dr3_source_id
				FROM uniqued_dr3match
				WHERE dr2_source_id=source_id;

				UPDATE \schema.main AS dest SET
					gaia_ra=mat.ra,
					gaia_dec=mat.dec
				FROM gaia.edr3lite as mat
				WHERE mat.source_id=dest.source_id3;

				COMMIT;
				VACUUM ANALYZE \schema.main;
				REINDEX TABLE \schema.main;
			</script>
		</make>
	</data>

	<service id="cone" allowed="form,scs.xml">
		<meta name="shortName">hdgaia cone</meta>
		<meta name="testQuery">
			<meta name="ra">1.28692100414281</meta>
			<meta name="dec">67.8399729385174</meta>
			<meta name="sr">0.05</meta>
		</meta>

		<publish render="form" sets="ivo_managed, local"/>
		<publish render="scs.xml" sets="ivo_managed"/>

		<scsCore queriedTable="main">
			<FEED source="//scs#coreDescs"/>
			<condDesc>
				<inputKey original="spectral">
					<values fromdb="spectral from \schema.main order by spectral"/>
				</inputKey>
			</condDesc>
			<condDesc> <!-- don't buildFrom this as it's coerced to string when
					form sees it -->
				<inputKey original="hd"/>
			</condDesc>
			<condDesc buildFrom="m_v"/>
		</scsCore>

	</service>

	<regSuite title="hdgaia regression">
		<regTest title="hdgaia SCS serves some data">
			<url RA="303.254864"
					DEC="41.028339" SR="0.01"
				>cone/scs.xml</url>
			<code>
				row = self.getFirstVOTableRow()
				self.assertEqual(row["spectral"], "B")
				self.assertAlmostEqual(row["gaia_ra"], 303.254864900)
				self.assertEqual(row["hd"], '228365')
				self.assertEqual(row["source_id"], 2062599636658625024)
				self.assertEqual(row["source_id3"], 2062599636658625024)
			</code>
		</regTest>

		<regTest title="hdgaia form runs and has range hints.">
			<url parSet="form" hd="228365">cone/form</url>
			<code><![CDATA[
				self.assertHasStrings(
					"(Warren and Kress 1980)",
					'<option value="B3">B3</option>',
					'<td>303.254864900</td>',
					'placeholder="-1.58 .. 50.0"')
			]]></code>
		</regTest>
	</regSuite>
</resource>
