New App repo has been created: https://bitbucket.org/KarthikRavee91/filter-by-down-sample/src/master/

Implement Down Sampler filter in IGB code base. See commit

Fetched Karthik's branch for testing. The Down Sampler filter appears to be downsampling by the specified amount. I checked this by highlighting the number of reads in the main view before and after filtering. The filter was easy enough to find in the drop-down menu, as well. However, see below for an error that needs to be addressed as well as some questions/suggestions.

Caught an error:
After adding a Down Sampler filter, clicking Edit and attempting to close the Edit filter window with the red X in the top left corner causes the following error in the Log:

Exception in thread "AWT-EventQueue-0" java.lang.ClassCastException: class java.lang.String cannot be cast to class java.lang.Integer (java.lang.String and java.lang.Integer are in module java.base of loader 'bootstrap')
	at com.affymetrix.igb.util.ConfigureFilters.beginEditing(ConfigureFilters.java:256)
	at com.affymetrix.igb.util.ConfigureFilters.editButtonActionPerformed(ConfigureFilters.java:156)
	at java.desktop/javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1972)
	at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2314)
	at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:407)
	at java.desktop/javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262)
	at java.desktop/javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279)
	at java.desktop/java.awt.Component.processMouseEvent(Component.java:6621)
	at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3398)
	at java.desktop/java.awt.Component.processEvent(Component.java:6386)
	at java.desktop/java.awt.Container.processEvent(Container.java:2266)
	at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:4996)
	at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2324)
	at java.desktop/java.awt.Component.dispatchEvent(Component.java:4828)
	at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4948)
	at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4575)
	at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4516)
	at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2310)
	at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2780)
	at java.desktop/java.awt.Component.dispatchEvent(Component.java:4828)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:775)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:720)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:714)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:400)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:98)
	at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:747)
	at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:745)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:400)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:744)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

Things to consider:

• The term "downsampling" is one word - should the name of the filter reflect this? E.g., "Downsampler"?
• It's not immediately apparent that the input is a percentage, maybe a "%" next to the box for user input would be more clear. Also, instead of "sampler", something like "Downsample by X%" would be clearer.
• Adding another filter (or Color by information) triggers the downsampling to happen again. Is this desired behavior?

Errors mentioned above has been fixed. See commit

The final point in the 'Things to consider' in the above comment by Paige Kulzer: When a new filter and/or color by condition is added, the whole chain of filters are run again, and the downsampler chooses reads randomly, so the observed behavior is bound to happen with the current workflow of chain filters. Applying changes to this behavior will be a significant undertaking, as it requires changing how the workflow of a chain of filters are applied which, for the purposes of downsampler, might not be warranted.

The error has been fixed and is no longer appearing when X'ing out of the "Edit filter" window.

My only other suggestion is a bit picky, but I think the name of the filter should just be "Downsampler" and the text in the filter should read, "Downsample by [text box] %".

My suggestion aside, recommending PR!

PR Submitted https://bitbucket.org/lorainelab/integrated-genome-browser/pull-requests/1079

Paige Kulzer the suggestion you gave in the above comment maybe possible to do but it would require some minor custom UI changes. The UI is based on a predefined template that includes limited dynamic changes.

Change request:

• Please re-implement this is as an IGB App so that it can work in the released version of IGB as well as 10.2.0.

I think it would work great as an App!

Re-implemented as an IGB App and submitted to IGB App store for review.

Thank you Karthik Raveendran!

For the next steps:

• Create a bitbucket repository for your App code and add your code there.
• Incude a bitbucket.yml file and a "README.md" file that will be displayed in Bitbucket. Explain there what the App does and let potential new developers understand how they can build it. The audience for this file are other developers. Tell them what you think they need to know about how to build it.
• Ensure that the app can build within bitbucket pipelines - it needs to create repository.xml and a jar file. It's possible you will not be able to copy these two files to the Downloads part of your Bitbucket repo. Please just check that the pipeline can run and create the artifacts.
• If you haven't already done it, make an App Description file that will be displayed in the IGB App Manager and also in the App Store. It should explain what the App does, why a user might want to run it, and how they can run it. You could maybe use saideepthi jagarapu's App Description as an example. You'll be able to see how the file gets used during the build process by looking at her code.

Update:

• I am attempting to access the App using the pending apps repository, a repository we added to IGB but, by default, is not active.
• When I did that, the App appeared in the App Manager as expected. I installed it, but got this error message in IGB log:

12:40:23.649 INFO o.l.i.p.manager.AppManagerFxPanel - Installing plugin: Downsampler Filter
ERROR: Resolver: Install error - Downsampler-Filter
java.io.IOException: Error accessing https://apps.bioviz.org/installapp/pending_releases/pending_releases/Downsampler-Filter-0.0.0.jar
at org.apache.felix.bundlerepository.impl.FileUtil.openURL(FileUtil.java:217)
at org.apache.felix.bundlerepository.impl.FileUtil.openURL(FileUtil.java:199)
at org.apache.felix.bundlerepository.impl.ResolverImpl.deploy(ResolverImpl.java:575)
at org.lorainelab.igb.plugin.manager.BundleActionManager.installBundle(BundleActionManager.java:173)
at org.lorainelab.igb.plugin.manager.BundleActionManager$1.get(BundleActionManager.java:149)
at org.lorainelab.igb.plugin.manager.BundleActionManager$1.get(BundleActionManager.java:139)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Caused by: java.io.IOException: Server returned HTTP response code: 500 for URL: https://apps.bioviz.org/installapp/pending_releases/pending_releases/Downsampler-Filter-0.0.0.jar
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:2011)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1602)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:223)
at org.apache.felix.bundlerepository.impl.FileUtil.openURL(FileUtil.java:209)
... 12 more
12:40:24.186 INFO o.l.i.p.manager.BundleActionManager - Installed app: Downsampler-Filter,0.0.0
12:40:24.187 INFO com.affymetrix.igb.IGB - bundle Downsampler-Filter;0.0.0 installed

• I don't think this is due to Karthik Raveendran's code at all but is a problem with the App Store code. However, I would like to run the App before I approve it for release on the App Store.

As per today's scrum, Karthik Raveendran already has created a repository for the App.

Karthik Raveendran please provide a link to the repository here for review.

Downsampler repo: https://bitbucket.org/KarthikRavee91/downsampler_filter/src/main/

Thanks Karthik Raveendran!

I was excited to see that you were able to get your build process to copy the jar file to the Downloads section!

However, to use the Downloads folder as an App repository, it needs a "repository.xml" file as well. Based on the pom.xml, it looks like your build process should be creating a repository.xml file, but I don't see it in the Downloads section.

Could you please look into why that file is not being copied?

(My apologies if you are already working on this currently - my request may be redundant with what you are already doing!)

Downsampler Filter is now available in repo downloads with the repository.xml : https://bitbucket.org/KarthikRavee91/downsampler_filter/downloads/

Two change requests:

(1)

I was able to install the filter App and observe that the filter interface says:

Show only: Downsampler by %

Downsampler by % [ default option is 10.0 ]

[ OK button ]

I am confused about what will happen when I click "OK."

How much data will be removed? Will it be 10% of the existing data will be hidden, or will only 10% be left as visible when I do this?

Please clarify and improve the interface so that this is more clear

(2) Please add a description markdown so that the App Manager interface shows information about the App. It should explain what the App does and how to use it.

Returning to Karthik Raveendran for clarification.

The percentage that is added by the user is the size of the subset of data that the user wants to see. So 10% means 10% of the data. I have changed the label text to represent that: https://bitbucket.org/KarthikRavee91/downsampler_filter/commits/ab87a64e410aea8bee71ab9f1f038fd164bcf414

I have also added a AppDescription.md file in that commit

Thank you for the improvements Karthik Raveendran!

For the next steps:

• Please increment the version (if you have not already done so) and merge the new branch into your main branch.
• Then please submit the App again to the App Store.

1) Incremented the version to 0.0.1
2) App submitted to App Store

I logged onto the App Store as "admin" user and accepted the App. It is now publicly accessible.
Also, the description text that Karthik Raveendran created is correctly displayed on the App's page here: https://apps.bioviz.org/apps/downsampler-filter

For the next steps, Karthik Raveendran please do the following:

• Log in again to the App Store and visit your App page: https://apps.bioviz.org/apps/downsampler-filter
• Select EDIT to open the editing interface for the page
• Next, please fill in fields: contact email, code repository URL (your repository), author
• Under Curated categories data type, select RNA-Seq, DNA-Seq, transcriptome, plus anything else that makes sense given the fact that the user has to choose a BAM (binary alignments) track to apply the filtering

Also, if you make any edits to the BUNDLE DESCRIPTION section, make sure you manually add that new text to your App description file in your repository. Otherwise, when the App is updated, your edits performed on the App Store site will be lost.

Do also please make some type of cool logo for the App. Once you create the logo, add it to your BitBucket repository as the icon for the repository.

Last but not least, I have a request for the filter option text!

The window that appears when I select the filter says:

"Show Only [ thing to be shown ]."

I think it will make more sense if "thing to be shown" states what will be shown if I choose the option. In this case, a percentage of the data will be shown. Is that correct? If so, then a good way to explain this to a user would be to use the phrase "% of alignments". Another option that might be better is "% of alignments, a random sample" so that the user understands that a random process is selecting which alignments will appear and that all alignments are equally likely to be chosen. Also, change "% of Reads to Display:" to "% of alignments to show:"

I think you use the term "alignment" instead of "read." This is because the graphical items IGB shows in a "bam" track are alignments. The term "reads" refers to the sequences that are being aligned to the genomic sequence. The visual entities we see in IGB include the sequence reads, but they show more than that: They show the details of how the read sequences match up to the genomic sequence. For example, consider how the same read sequence may can appear in many different alignments, if the read sequence is from a redundant part of the genome.

All changes mentioned above have been implemented, however, after running the pipeline the appstore jar was not updated.

I'm sorry - I don't know what you mean by "appstore jar was not updated". Did you re-submit the App to AppStore with new version coordinates? I'm not sure what I need to do next.

I submitted a new version of the app with the proposed changes in app store. I assumed when a new jar is created appstore will automatically update it.

Change requests:

• Add the image you created as a new file in the repository - do it like this: https://bitbucket.org/lorainelab/app-for-testing-app-workflow/src/testing/src/main/resources/
• Within Bitbucket, change the repository settings so that it uses the image as the repository logo / avatar
• Create a better "README.md" for the repository (see: https://bitbucket.org/KarthikRavee91/downsampler_filter/src/main/)

The 3 changes requested above has been implemented in the repo .

Thank you Karthik Raveendran! I updated the Loraine Lab fork with your new changes.

One last request !

Since the README.md is basically a copy of the AppDescription.md text, how about changing the POM.xml so that it uses the README.md for the "app description" variable? Then we won't have to maintain two different files, and you can delete the AppDescription.md.

Resubmitted the app with the requested changes.

Tested the Downsampler Filter on Mac using IGB 10.2.0. The app is available from the App Manager and installed with no errors. The app also appears in the IGB App Store with the correct version number (0.0.3). I'm able to use the app to filter reads within a BAM file track. The instructions in the app description are clear and easy to follow. Based on the above comments, I made sure that the app's JAR files were present in the Downloads section on BitBucket. Similarly, I checked that the logo is present in the repo, as well.

All looks good to me, so I'm closing this ticket now.