From: Dr. Ann Loraine <Ann.Loraine@uncc.edu>
Date: Thursday, October 02, 2014 3:22 PM
To: >Nicholas Provart <nicholas.provart@utoronto.ca>,Meyer, Mason; Clabaugh, Ivory; Norris, David
Cc: Vaishali Deswal <v.deswal@mail.utoronto.ca>
Subject: IGB metadata file format?

Hello Nick,

It is great to hear from you.

Everyone, please meet Mason Meyer, who will be the point person on this.
He¹s on the IGB support team, and if he can¹t answer a particular
question, he can find the person who can.

Mason, please say hi!

To start, I¹d recommend sending the links to the documentation about
IGBQuickLoad and annots.xml. Mason, can you do that?

Also, please meet Ivory Blakley, who has used QuickLoad a lot to share
RNA-Seq and ChIP-Seq data files with other labs. She is pretty busy on
other projects, but she can weigh in with some advice from time to time as
needed.

Lastly, please meet David Norris, lead IGB developer. My guess is that as
you start to build out infrastructure to support IGB QuickLoad, you may
notice ways we can improve the software and get better performance for
users. So if you have questions or comments related to software
engineering of IGB, he would be the best person to contact.

All the best,

Ann

From: Mason Meyer <Mason.Meyer@my.lr.edu>
Date: Thursday, October 02, 2014 at 5:37 PM
To: Nicholas Provart <nicholas.provart@utoronto.ca>Loraine, Ann; Clabaugh, Ivory; Norris, David
Cc: Vaishali Deswal <v.deswal@mail.utoronto.ca>
Subject: IGB metadata file format?

Hello Nick and Vaishali ,

My name is Mason Meyer and I'll be here to answer whatever questions you may have for us about IGB. Feel free to e-mail me at anytime and I will promptly get back to you. For the moment, Dr. Loraine has requested that I send you the links to the documentation on IGB QuickLoad and annots.xml. I have included the links below:

• Creating Quickload Sites: http://wiki.transvar.org/confluence/display/igbman/Creating+QuickLoad+Sites

• Annots.xml: http://wiki.transvar.org/confluence/display/igbman/Creating+QuickLoad+Sites#CreatingQuickLoadSites-Annots.xmloptionsspecifytrackcolor%2Cannotationstyle%2Candmore

If there is anything else I can do to help assist you please do not hesitate to contact me. I look forward to hearing from you.

Thanks again,

Mason Meyer

From: Vaishali Deswal <v.deswal@mail.utoronto.ca>
Date: Mon, Oct 6, 2014 at 10:28 AM
To: Mason Meyer <Mason.Meyer@my.lr.edu>
Subject: IGB metadata file format?

Hi Mason

Thank you for the links. I will keep in touch.

best,
Vaishali

From: Mason Meyer <Mason.Meyer@my.lr.edu>
Date: Mon, Oct 6, 2014 at 11:19 AM
To: Vaishali Deswal <v.deswal@mail.utoronto.ca>
Subject: IGB metadata file format?

Hello Vaishali,

It was no problem sending the links. Let me know if you need something else, I will be here to answer your IGB questions anytime!

Thanks again,

Mason Meyer

Hi Ann,
From: Nicholas Provart <nicholas.provart@utoronto.ca>
Date: Thu 5/28/2015 1:19 PM
To: Dr. Ann Loraine <Ann.Loraine@uncc.edu>
Subject: IGB metadata file format?

So Araport (the new Arabidopsis Information Portal) and iPlant have mapped the existing Arabidopsis RNA-seq data sets from the SRA and have made them available on iPlant. We're just putting our combined RNA-seq track and eFP viewer together and, as we've discussed for a long time, we'd like to use an XML file that IGB can also use to explore the RNA-seq data on not just a gene-by-gene basis (that's what our viewer will do) but genome-wide.

I'm attaching a mockup of the XML file put together by a work-study student Richard Song, as described in the instructions from Mason from October last year. We can't figure out if it's working - how would we get the TAIR10 genome to load, do we need to specify this in the XML file (we've extended the XML with extra tags that our viewer would need)? There are two RNA-seq data sets in that attached mockup, and we'll have the 113 data set XML file ready in a couple of weeks.

Best, Nick

From: Dr. Ann Loraine <Ann.Loraine@uncc.edu>
Date: Thu 5/28/2015 1:57 PM
To: >Nicholas Provart <nicholas.provart@utoronto.ca>,Meyer, Mason; Clabaugh, Ivory; Norris, David
Subject: IGB metadata file format?

OK - we’ll look into it.

From: Dr. Ann Loraine <Ann.Loraine@uncc.edu>
Date: Thu 5/28/2015 2:22 PM
To: Meyer, Mason; Eckstein, John; Norris, David
Subject: IGB metadata file format?

Hello,

I set up TWO sample QL sites using DropBox here:

Site using Nick’s “annots.xml"
https://dl.dropboxusercontent.com/u/15094167/NicksAIP

Site using my edited version of Nick’s “annots.xml”

My edited version of the annots.xml:
https://dl.dropboxusercontent.com/u/15094167/AnnsEditsAIP

I tried adding both sites to IGB using both IGB 8.3.4 and IGB 8.4.0 (development version) and observed different behavior, including a bug in how IGB 8.4 is handling the file “name” attribute.

If “name” points to a URL, e.g., http://newland.iplantcollaborative.org/iplant/home/araport/rnaseq_bam/aerial/ERR274310/accepted_hits.bam

Then IGB shouldn’t append the QL genome version root URL to it. But it looks like IGB 8.4 is doing that.

IGB 8.3.4 is doing the expected thing, however.

Mason could you take a look and add these issues to the testing queue?

Also, note that Nick’s XML has attributes and child elements not recognized by IGB. The IGB 8.4 parser is able to tolerate these, apparently. The iGB 8.3.4 parser can’t, but it gives very useful and informative error messages, which I appreciated very much. Many kudos to the developer who implemented this! Thanks to that, it was very easy to debug their annots.xml file.

Probably we need to talk w/ Nick’s team to find out how critical (or not) it is for him to be able to include tags and attributes not intended for IGB’s use — that is, they may be using this XML both to control IGB but also as meta-data for their site.

Using David’s new API, we might even add a new kind of data provider for them. Something else to put in the grant

-Ann

From: <Norris>, David <dcnorris@uncc.edu<dcnorris@uncc.edu>>
Date: Thursday, May 28, 2015 at 4:52 PM
To: "Loraine, Ann" <Ann.Loraine@uncc.edu<Ann.Loraine@uncc.edu>>, "Meyer, Mason" <mmeyer20@uncc.edu<mmeyer20@uncc.edu>>
Subject: RE: IGB metadata file format?

For clarification, I am able to enforce stricter annots.xml parsing if it is preferred. I think the 8.4.0 code is smart to ignore the unrecognized fields as harmless, but we have discretion. The old annots xml parser also seems to be ignoring extra attributes on glancing over it, so maybe the error handling you are referring to was elsewhere?

The new Quickload code is designed to always provide useful feedback to make it clear when things are in an invalid state, so if you have any example at all of something more user friendly about the old code I would like to know about it so I can append this type of feedback to the new implementation. I actually think you will find it already has fully coverage, but if I missed anything let me know.

From: Loraine, Ann
Sent: Thursday, May 28, 2015 5:08 PM
To: Norris, David; Meyer, Mason
Subject: Re: IGB metadata file format?

OK - agreed. Let’s allow the parser to tolerate any and all unused (by QuickLoad) tags. Let’s not alert the user by showing a popup.

Try accessing QL site

https://dl.dropboxusercontent.com/u/15094167/NicksAIP

Using both 8.4 and 8.3.4.

It has data sets from Arabidopsis.

Main problem is that the annots.xml Nick provided references a DTD file:

java.io.FileNotFoundException: /Applications/Integrated Genome Browser 8.3/bamlocator.dtd (No such file or directory) at java.io.FileInputStream.open0(Native Method) ~[na:1.8.0_40] at java.io.FileInputStream.open(FileInputStream.java:195) ~[na:1.8.0_40] at java.io.FileInputStream.<init>(FileInputStream.java:138) ~[na:1.8.0_40]

From: <Norris>, David <dcnorris@uncc.edu<dcnorris@uncc.edu>>
Date: Thu 5/28/2015 5:58 PM
To: "Loraine, Ann" <Ann.Loraine@uncc.edu<Ann.Loraine@uncc.edu>>, "Meyer, Mason" <mmeyer20@uncc.edu<mmeyer20@uncc.edu>>
Subject: RE: IGB metadata file format?

I also fixed the name parsing differences. IGB 8.4.0 now tolerates the remote url format and the relative path format which is more typical on our own servers. The original unmodified quickload site now works as expected on IGB 8.4.0.

From: nicholas.provart@gmail.com [nicholas.provart@gmail.com] on behalf of Nicholas Provart [nicholas.provart@utoronto.ca]
Sent: Thursday, June 11, 2015 12:28 PM
To: Loraine, Ann
Cc: Meyer, Mason; Blakley, Ivory; Norris, David; Alison Wu; Richard Song
Subject: Re: IGB metadata file format?

Hi Ann and Mason,

Were you able to take a look at our annots.xml file and can you tell us why we're not seeing those RNA-seq tracks from iPlant?

Thanks!

Nick

From: <Norris>, David <dcnorris@uncc.edu>
Date: Thursday, June 11, 2015 at 12:45 PM
To: "nicholas.provart@utoronto.ca" <nicholas.provart@utoronto.ca>, "Loraine, Ann" <Ann.Loraine@uncc.edu>
Cc: "Meyer, Mason" <mmeyer20@uncc.edu>, "Blakley, Ivory" <ieclabau@uncc.edu>, Alison Wu <alison.wu@mail.utoronto.ca>, Richard Song <richardmc.song@mail.utoronto.ca>
Subject: RE: IGB metadata file format?

Hi Nick,

I was able to look at your annots.xml, and can tell you why IGB wasn't able to handle it in the form you were using. You were including a local dtd reference in the xml (i.e. <!DOCTYPE files SYSTEM "bamlocator.dtd">), which caused issues with XML validation. I have corrected this on our development branch, and the upcoming release of IGB will be able to handle the inclusion of an unexpected dtd reference. If you would like to work with the preview version of our upcoming IGB release, you can get a copy from http://test.bioviz.org/igb/index.html. Don't hesitate to reach out directly if you have any additional problems.

Best Regards,
David

From: <Loraine>, Ann <Ann.Loraine@uncc.edu>
Date: Thursday, June 11, 2015 at 12:59 PM
To: "Norris, David" <dcnorris@uncc.edu>, "nicholas.provart@utoronto.ca" <nicholas.provart@utoronto.ca>, "Loraine, Ann" <Ann.Loraine@uncc.edu>
Cc: "Meyer, Mason" <mmeyer20@uncc.edu>, "Blakley, Ivory" <ieclabau@uncc.edu>, Alison Wu <alison.wu@mail.utoronto.ca>, Richard Song <richardmc.song@mail.utoronto.ca>
Subject: Re: IGB metadata file format?

I created a couple of public Dropboxes for testing the annots.xml files.

Instructions for using a Dropbox “Public” folder to share data via IGBQuickLoad: https://wiki.transvar.org/display/igbman/Set+up+a+Quickload+in+Dropbox

Dropbox Quickload with my edits to the annots.xml: https://dl.dropboxusercontent.com/u/15094167/AnnsEditsAIP (works fine with released version of IGB)

Dropbox Quickload with your original annots.xml, minus the reference to a nonexistent local “DTD” file:

https://dl.dropboxusercontent.com/u/15094167/NicksAIP

Also, to get your data sets to appear in folders, add a “slash” character to the title, as with this annots.xml file:

http://www.igbquickload.org/abiotic/A_thaliana_Jun_2009/annots.xml

Also, an image. See attached.