Online Logs

Introduction

In May 2005, about a month or so after I started to keep notes of my observing sessions, I decided that it would be a good idea to get those notes onto a computer. After a little thought I decided that the best way to approach this was to publish them on my website. The first attempt at doing this involved me hand-writing HTML files (with the help of some code I wrote for GNU Emacs) – it worked but it wasn’t very elegant. The main problem with this approach was that I was mixing data and the presentation of that data.

In September 2005 I decided to move the astronomy content of my site to a dedicated site. While writing the code to run that site I decided to re-think the approach to my logs. The following documents the method I devised for my logging system.

Overall Design

After some thought I decided that writing up my logs as XML files was the way to go. The two main reasons for this approach were:

  1. It more or less forces me to divorce presentation issues from data issues.
  2. There are lots of tools on the net that provide methods of extracting data from XML files so that it can be used in other forms. The main tool I had in mind when I made this decision was Sablotron.

After deciding to use XML the next issue I was faced with was that of using an existing approach or taking a Not Invented Here approach and creating my own design. I asked for feedback on the SPA BB and also sent an email to the British Astronomical Association seeking suggestions but I never got a definitive answer (actually, I didn’t get any answer from the BAA). The only two examples I found online were pchapin's logs and Observation Manager and XML Schema for astronomical data – both looked pretty good but also a little too rigid for my needs.

In the end I decided that, given that there didn’t appear to be any widely-used active effort out there I’d roll my own design. What follows is documentation for what I’ve come up with so far. Keep in mind that it isn’t ideal (see further down this document for details of the current problems), it just happens to work well for me and fits my current needs.

Format of a Log

A day’s worth of observation logging is held in a single XML file, all XML files are named YYYYMMDD.xml. The date used always relates to the date in Universal Time. For an observing session to be included in a given log file the session needs to have started on that date (it doesn’t matter if it ran over to the following date).

Because a single log file covers a whole day the file might contain multiple observing sessions (in retrospect I think this is a bad idea – see current problems for more details). Because of this a single log file might only contain a single <session>...</session> item or, if there is more than one session, all sessions will live inside a <sessions>...</sessions> item.

For example, a log file containing a single session will generally look like:

<session>
  <!-- Session data here -->
</session>

whereas a log file containing one or more sessions will generally look like:

<sessions>
 
  <session>
    <!-- Session 1 data here -->
  </session>
 
  <session>
    <!-- Session 2 data here -->
  </session>
 
</sessions>

Format of a Session

A session contains a number of items that explain the overall information about the session as well as details of each of the observations made (see below). The main “header” items in a session are as follows:

Details about the observer

The details of the observer are wrapped in a pair of observer tags. Information that can be given is the name of the observer, the email address of the observer and the home page of the observer – the last two items are optional.

Example:

<observer>
  <name>John Observer</name>
  <email>john@example.com</email>
  <homepage>http://www.jobserver.example.com/</homepage>
</observer>

Details of when the session took place

The details of when the session took place are wrapped in a pair of when tags. Within those tags there is a required from pair of tags and an optional to pair. Both from and to support an attribute called qualify with allows some text to be used to qualify the time – I generally use it to note if a recorded time was approximate.

Example of a session time where the from and to times are both known:

<when>
  <from>2005-08-15T20:24Z</from>
  <to>2005-08-15T22:55Z</to>
</when>

Example of a session time where the from and to times are both know but the times are approximate:

<when>
  <from qualify="(approximate)">2005-08-15T20:24Z</from>
  <to qualify="(approximate)">2005-08-15T22:55Z</to>
</when>

Example of a session time where the session took place in a single minute:

<when>
  <from>2005-08-15T20:24Z</from>
</when>

Details of where the session took place

The details of where the session took place are wrapped in a pair of location tags. Within those tags there is a required name pair of tags and an optional note pair. The idea is that the name will give just that: the name of the location, whereas the note will give extra information (perhaps “county” and country information).

There’s also a position pair of tags for given the actual position of the location – this is optional. Within it are required latitude and longitude tags and an optional altitude tag.

<location>
  <name>Billingborough</name>
  <note>South Lincolnshire, UK</note>
  <position>
    <latitude>52.893649</latitude>
    <longitude>-0.340233</longitude>
    <altitude units="m">10</altitude>
  </position>
</location>

Details of the equipment used during the session

Details of the equipment used during the session is recorded inside an optional pair of equipment tags. One or more items of equipment are listed, each one surrounded by a pair of item tags. The text can be anything – personally I ensure that the text for each item of equipment is the same each time I use it, this makes it easier to index on.

An example equipment list looks like:

<equipment>
  <item>Naked Eye</item>
  <item>Sky-Watcher Explorer 130M</item>
</equipment>

Optional details of the weather for the session

Optional details about the state of the weather at the time that the session started are wrapped in a pair of weather tags. Within those tags are the details of each recordable aspect of the weather (each one of these is optional too). At the moment I currently support the temperature, dew point, relative humidity, wind speed, wind direction and relative atmospheric pressure. For each recordable item a value (with associated unit) and source must be provided.

An example weather section would look like:

<weather>
  <temperature>
    <value units="C">7.6</value>
    <source>La Crosse WS3600</source>
  </temperature>
  <dewpoint>
    <value units="C">2.2</value>
    <source>La Crosse WS3600</source>
  </dewpoint>
  <humidity>
    <value units="%">69</value>
    <source>La Crosse WS3600</source>
  </humidity>
  <wind-speed>
    <value units="mph">2.9</value>
    <source>La Crosse WS3600</source>
  </wind-speed>
  <wind-direction>
    <value units="">West South West</value>
    <source>La Crosse WS3600</source>
  </wind-direction>
  <pressure>
    <value units="hPa">990.1</value>
    <source>La Crosse WS3600</source>
  </pressure>
</weather>

Various optional index lists relating to the session

To allow for various forms of indexing of a session an index tag is provided. While many types of index could be generated from the presence of specific markup tags this approach exists because it is more flexible and allows me to index on things that don’t have specific markup.

The format is as follows:

<index type="type-name">
  <item><!-- First item --></item>
  <item><!-- Second item --></item>
  <!-- More items -->
</index>

The type parameter of the index tag supplies the type of the index – this can be anything at all. At the moment I use it for a specific set of indexes are are used on my site. They are:

An example of a general index is:

<index type="general">
  <item>Planet</item>
  <item>Uranus</item>
  <item>Messier</item>
  <item>Caldwell</item>
  <item>Cluster</item>
  <item>Galaxy</item>
</index>

General notes about the session

A session can optionally have some general notes associated with it. The notes go in a pair of <note>...</note> tags. The content of those tags should be valid XHTML but with the addition of any of the log markup items documented below.

For example:

<note>
  <p>
    A clear, warm and calm evening. The sky was still very light (the
    Sun hadn't long set). I decided to set up the
    <equipment>905</equipment> to have a session observing the
    <terrestrial>Moon</terrestrial> (which was just past first
    &frac14;). I also intended to observe <planet>Saturn</planet> and,
    if I was out late enough and the position was right,
    <planet>Jupiter</planet>.
  </p>
  <p>
    All being well I also wanted to try and further observe
    <comet id="73P">73P Schwassmann-Wachmann 3</comet> although I didn't
    hold out too much hope given how bright the Moon was going to be.
  </p>
</note>

Format of an Observation

Observations are contained within a session and are wrapped in a pair of <observations>..</observations> tags. Within the observations tags there is a <observation>..</observation> pair, each one containing a single observation. For example:

<observations>
 
  <observation>
    <!-- Observation 1 -->
  </observation>
 
  <observation>
    <!-- Observation 2 -->
  </observation>
 
  ...
 
  <observation>
    <!-- Observation n -->
  </observation>
 
</observations>

Observations should be listed in chronological order – earliest first.

Details of when the observation took place

The rules for giving the time of an observation are the same as those used for a session.

The title of the observation

The title of an observation should be a simple, short, one line description of the observation. It is enclosed in a pair of <title>...</title> tags. For example:

<title>Saturn</title>

Optional details of the weather for the observation

The rules for noting the weather conditions for an observation the same as those used for a session.

I tend not to use this that much now that I’ve got my weather station as this records a full history that I can refer back to.

The notes of the observation

The notes for an observation contain the details and text of the observation. The text is wrapped in a pair of <notes>...</notes> tags (notes, not note) and the rules for the content are the same as those used for a session.

Markup for use in notes

The notes of a session and an observation can contain various forms of custom markup. These items of markup include:

  • asterism – Refer to an asterism.
  • caldwell – Refer to a Caldwell object.
  • comet – Refer to a comet.
  • const – Refer to a constellation.
  • equipment – Refer to an item of equipment.
  • filter – Refer to a filter.
  • gloss – Mark a word or phrase as a glossary item.
  • image – Refer to an image.
  • jovian – Refer to an object associated with Jupiter.
  • logref – Refer to another log.
  • lunar – Refer to an object associated with the Moon.
  • messier – Refer to a Messier object.
  • meteorshower – Refer to a meteor shower.
  • ngc – Refer to an NGC object.
  • obsref – Refer to a specific observation.
  • planet – Refer to a planet.
  • saturnian – Refer to an object associated with Saturn.
  • star – Refer to a star.
  • sunspot – Refer to a sunspot/active region on the Sun.
  • sunspotreturn – Refer to a return of a sunspot/active region on the Sun.
  • terrestrial – Refer to an object associated with Earth.
  • time – Refer to a specific time.

An Example Log File

The following is an annotated version of a session that took place on 2005-08-15, the full source can be downloaded from here.

First we have a DOCTYPE to ensure that the observing session entities file is included:

<!DOCTYPE session [<!ENTITY % sessionEntities SYSTEM "session.ent"> %sessionEntities; ]>

The start of the session:

<session>

Declare some details about the observer:

  <observer>
    <name>David Pearson</name>
    <email>davep@davep.org</email>
    <homepage>http://www.davep.org/</homepage>
  </observer>

Declare the start and end times of the session:

  <when>
    <from>2005-08-15T20:24Z</from>
    <to>2005-08-15T22:55Z</to>
  </when>

Declare the location of the session:

  <location>
    <name>Billingborough</name>
    <note>South Lincolnshire, UK</note>
    <position>
      <latitude>52.893649</latitude>
      <longitude>-0.340233</longitude>
      <altitude units="m">10</altitude>
    </position>
  </location>

List the equipment used during the session:

  <equipment>
    <item>Naked Eye</item>
    <item>Sky-Watcher Explorer 130M</item>
  </equipment>

List some general terms that might be included in an index:

  <index type="general">
    <item>Messier</item>
    <item>Cygnus</item>
    <item>Lyra</item>
    <item>Ursa Major</item>
    <item>Hercules</item>
    <item>Andromeda</item>
    <item>Galaxy</item>
    <item>Drawing</item>
    <item>Double</item>
    <item>Iridium Flare</item>
    <item>Satellite</item>
    <item>Meteor</item>
    <item>Mars</item>
    <item>Planet</item>
    <item>Cluster</item>
  </index>

List the Messier objects observed during the session:

  <index type="messier">
    <item>13</item>
    <item>31</item>
    <item>32</item>
  </index>

List items from the New General Catalogue observed during the session:

  <index type="NGC">
    <item>6205</item>
    <item>224</item>
    <item>221</item>
  </index>

List constellations pertaining to observations made during the session:

  <index type="constellation">
    <item>And</item>
    <item>Lyr</item>
    <item>Cyg</item>
    <item>Her</item>
    <item>UMa</item>
  </index>

Overall notes for the session as a whole – in this case outlining the reason and the plan for the session as well as giving some idea of the conditions at the time:

  <note>
    <p>
      The main reason for venturing out was to give the 130M a proper "run
      under the stars" after
      <a class="ext" href="http://davep-astro.blogspot.com/2005/08/fear-of-mirrors-ii-when-mirrors-attack.html">center
      spotting the primary mirror</a> on 2005-08-13. Also, before heading
      out, I use a laser collimator to try and improve the collimation. I
      didn't plan on doing any star-tests tonight &mdash; I just wanted to
      see how well I got on with the 'scope having actually had the mirror
      out of it.
    </p>
    <p>
      Moon was a waxing gibbous quite low in the sky (not visible from my
      position). There was some thin haze in parts of the sky but no
      noticeable clouds. Temperature was cool but still warm enough to be
      out in a t-shirt.
    </p>
    <p>
      When I started out the sky was still somewhat light.
    </p>
  </note>

Start of the list of observations made during the session:

  <observations>

Start of the first observation made during the session:

    <observation>

Start and end times for the observation:

      <when>
        <from>2005-08-15T20:24Z</from>
        <to>2005-08-15T21:05Z</to>
      </when>

The title for the observation:

      <title>General testing</title>

The notes for the observation:

      <notes>
        <p>
          Started out by pointing the 'scope at <star>Mizar</star>. With the
          25mm eyepiece there was some obvious "flaring" of the brighter
          stars in the field. I could also see, from time to time, a faint
          "rainbow" effect in the flare. At this point I had trouble
          recalling exactly how bright stars used to look in the 'scope.
          I've always seen some flaring but &mdash; never having really made
          a point of noting <strong>exactly</strong> how it appeared &mdash;
          I didn't really have anything to compare. Lesson for the future:
          make notes about the really obvious things such as how stars look
          before you do some work on your 'scope.
        </p>
        <p>
          With the 15mm eyepiece the flare (which, at times, looked like a
          very tight double image of each bright star) had a noticeable
          difference in appearance either side of best focus. When unfocused
          either side there was the impression of an oval effect to the
          unfocused stars. Either side of focus the orientation of the oval
          would rotate 90&degree;. As I understand it this is evidence of
          astigmatism in the primary or secondary mirror! I don't think I've
          ever noticed this before (not that I've ever really gone looking
          for it before).
        </p>
        <p>
          I tried a few things to see if the oval effect would differ: I
          changed eyes (no difference), changed my orientation at the
          eyepiece (no difference) and rotated the eyepiece in the focuser
          (no difference).
        </p>
        <p>
          I then tested with the 10mm and then 6mm eyepieces and, as best as
          I could tell, the oval effect wasn't noticeable. Most confusing.
        </p>
        <p>
          The more I thought about it the more I felt that what I was seeing
          actually wasn't any worse than the 'scope used to be. Also,
          there's the fact that I don't generally know what a bright star
          should look like through a smallish Newtonian Reflector.
        </p>
        <p>
          I wished that I'd had Jupiter or Saturn around still so that I
          could compare how things looked with a more "substantial" target.
        </p>
      </notes>

The end of the first observation for the session:

    </observation>

Start of the second observation:

    <observation>

The time of the second observation. Note that, in this case, there is only a from time given which means that the observation happened at a specific time, it didn’t cover a period of time:

      <when>
        <from>2005-08-15T21:01Z</from>
      </when>

The rest of the observation:

      <title>Meteor</title>
 
      <notes>
        <p>
          Saw a reasonably bright meteor pass roughly North to South through
          <const>Lyr</const>.
        </p>
      </notes>
 
    </observation>

Further observations:

    <observation>
 
      <when>
        <from>2005-08-15T21:10Z</from>
      </when>      
 
      <title>Probable Iridium Flare</title>
 
      <notes>
 
        <p>
          Saw a very bright Iridium Flare in <const>UMa</const> &mdash; just
          below the "handle" of "The Plough". I got the impression that it
          was one of the brightest flares I've ever seen. It was a
          <strong>lot</strong> brighter than any of the stars in
          <const>UMa</const>.
        </p>
      </notes>
 
    </observation>
 
    <observation>
 
      <when>
        <from>2005-08-15T21:25Z</from>
        <to>2005-08-15T21:45Z</to>
      </when>      
 
      <title>M13</title>
 
      <notes>
        <p>
          After the slight annoyance and frustration early on in the session
          I decided to try the 'scope out on a DSO and opted for an easy
          target: <messier>13</messier>. Initial impression with the 25mm
          eyepiece was that it looked magnificent! While it looked like a
          cometary-like "blob" (as I'd noted in a 
          <logref log="20050606">previous observing session</logref>) there was,
          this time, the occasional faint hint that it was comprised of lots
          of stars. It wasn't so much that I could see stars, it was more a
          case of it looking slightly "grainy" from time to time.
        </p>
        <p>
          Made a sketch via the 25mm eyepiece:
        </p>
        <image big="2005-08-15-M13.jpg" width="498" height="512" alt="Sketch of M13">2005-08-15-M13-Half.jpg</image>
      </notes>
 
    </observation>
 
    <observation>
 
      <when>
        <from>2005-08-15T21:49Z</from>
      </when>      
 
      <title>Possible "late" Perseid</title>
 
      <notes>
        <p>
          Saw a faint meteor pass through <const>And</const>. Was very quick
          (less than a second I'd have said) and, given the direction of
          travel, it looked like it might have been a "late" Perseid.
        </p>
      </notes>
 
    </observation>
 
    <observation>
 
      <when>
        <from>2005-08-15T21:52Z</from>
      </when>      
 
      <title>Satellite between Cygnus and Lyra</title>
 
      <notes>
        <p>
          Watched a faint satellite go roughly North to South, more or less
          via the zenith, and pass between <const>Cyg</const> and
          <const>Lyr</const>. It seemed to occult a faint (to the naked eye)
          star somewhere between the two constellations. Unfortunately, at
          the time, I wasn't in a position to note which one it was.
        </p>
      </notes>
 
    </observation>
 
    <observation>
 
      <when>
        <from>2005-08-15T21:53Z</from>
        <to>2005-08-15T22:01Z</to>
      </when>      
 
      <title>M13</title>
 
      <notes>
        <p>
          Went back to M13, this time with the 15mm eyepiece. Appeared
          slightly brighter. There was now a hint that it's made of actual
          stars with the grainy appearance mentioned above being much more
          pronounced. While doing viewing a thin but obscuring line of cloud
          (might even have been a <gloss>contrail</gloss>) moved into the
          area and made observing rather hard. Somewhat annoying as I was
          about to start a sketch of what it looked like with the 15mm
          eyepiece.
        </p>
      </notes>
 
    </observation>
 
    <observation>
 
      <when>
        <from>2005-08-15T22:12Z</from>
        <to>2005-08-15T22:55Z</to>
      </when>      
 
      <title>M31 (and possibly M32)</title>
 
      <notes>
        <p>
          <messier>31</messier> is now in a position where I can see it
          with the 'scope. First looked at it with the 25mm eyepiece. M31
          itself was obvious but, at the same time, indistinct. There was an
          obvious brightness difference between what I assume is the central
          bulge and between the disk. There was no hint of any sort of
          structure and the whole thing had the appearance of a sort of
          light-gray "mist". The fact that I was looking in the direction of
          a street-light and that there was still a very faint haze in the
          sky probably wasn't helping matters.
        </p>
        <p>
          After a short while I noticed that a star in the field was
          actually rather "fuzzy" when compared to all the other stars.
          Started to wonder if what I was seeing was <messier>32</messier>.
          My initial impression was that it was further away from M31 than
          I'd imagined it would appear to be but, that said, that impression
          is formed from the photographs I've seen of M31 (which obviously
          show a lot more of the galaxy than I'd be seeing through my
          'scope).
        </p>
        <p>
          Checking with a chart I had to hand the fuzzy object did appear to
          be in about the right location for M32. To be sure I went and
          checked with my copy of <em>Sky Atlas 2000</em> and, looking at
          that, I convinced myself that I wasn't seeing M32 (based on the
          pattern of stars near it which seemed to be in SA2000 but not in
          the correct position for M32). Lesson here: be sure of the width
          of the field of view of the eyepiece so you can make good
          estimates of separation of objects.
        </p>
        <p>
          Switched to the 15mm eyepiece. The "fuzzy star" still had a fuzzy
          appearance and still looked quite different from all other stars
          in the field.
        </p>
        <p>
          Switched back to the 25mm eyepiece and made the following sketch:
        </p>
        <image big="2005-08-15-M31.jpg" width="546" height="512" alt="Sketch of M31">2005-08-15-M31-Half.jpg</image>
        <p>
          At <time>2005-08-15T22:55Z</time> I finished the session.
        </p>
      </notes>
 
    </observation>
 
    <observation>
 
      <when>
        <from>2005-08-15T22:55Z</from>
      </when>      
 
      <title>Mars pops up</title>
 
      <notes>
        <p>
          As I was starting to pack up I noticed that <planet>Mars</planet>
          had popped up over the roofs of the houses to the East of me. I
          did consider setting up the 'scope again to have a look at it but
          given that it was still low down and given that it was very close
          to a street-light I decided to save that for another night when
          conditions were a little more favourable.
        </p>
      </notes>
 
    </observation>

The end of the list of observations for this session:

  </observations>

The end of the data for this session:

</session>

Current Problems

At the moment there is only one design problem that I consider to be a serious issue and that is the decision to have multiple sessions in one file when I have more than one observing session in a day. The problem came about when I purchased my Solarscope – up until then I’d only been observing at night so the issue of more than one session in a day hadn’t been a problem. Once I started doing some solar observing the problem of having multiple sessions in a day became apparent.

The solution to this problem (given that I had a whole site built around the existing single-log-single-session design) was to allow multiple sessions in a single log file. While this works and neatly solved the problem the result can appear a little confusing to someone who doesn’t know what they’re looking at in my online logs.

My current thinking is that the long-term solution should be to get rid of the whole “one file per day” approach and, instead, have one file per session (the name of the file probably being YYYYMMDDHHMM.xml where HHMM is the time of the start of the session). This will, however, result in me needing to re-write most of the current log presentation system on my site so I’ve not rushed to do this.


Navigation
Personal Tools