The new app for Make scaled coverage graph (with bigWig output) and Make coverage graph (with bigWig output) has been created as a prototype owned by the nowlanf account.

Make coverage graph bigWig: 6cd156c4-3dd4-11ed-8d36-008cfa5ae621
Make scaled coverage graph bigWig: 3adaa6b0-39c5-11ed-8bdd-008cfa5ae621

The app appeared to run successfully and the output was visible in IGB.

However, I noticed that there is no default analysis name or output file name for any of the analyses. This seems like a bug and requires further investigation.
UPDATE: I have created IGBF-3186 to address this issue.

This sounds like something that would need to be resolved before deployment.

The new apps have been added to the IGB community, but have not been made public. It was my understanding that previously the apps would need to be made public in order for them to be visible to other users, i.e., an app that is not public is not visible to other users unless it is either made public or is shared with that specific user. However, while testing the new apps on bvctest3 I noticed that they are available to all 3 of my user accounts. While this is unexpected it may be valuable, as once an app has been made public it cannot be modified, thus if we were to leave the apps as private, but within the IGB community we could more easily modify them when needed.

Next step will be to check if Karthik's user accounts can also see the new apps on bvctest3.

UPDATE: While user's can see the new apps on bvctest3.bioviz.org that have been added to the IGB community but have not yet been made public, attempting to run a job with the new app leads to an error and the analysis is not submitted. The apps must be made public and we must avoid running the syncapps endpoint until the apps have been made public in order to avoid confusing users.

The two new apps have been created under the nowlanf account and appear to be working correctly.

Next steps which must be completed in this order:

1. Decide on a naming convention for the new apps, as they are effectively replacing the original apps but will require a different public name.
2. Move test input/output data to BioViz community folder.
3. Publish apps with appropriate documentation which can be copied from the current Make Scaled Coverage Graph app.
4. Sync Apps to database.
5. Remove previous apps from IGB Community (or leave them as they provide bedgraph output?).

Some thoughts / questions:

Does CyVerse "apps" endpoint include a version field? If yes, we could configure BioViz Connect to enable the user to select a particular app version. And, we could show what we consider to be the best version of the app.

CyVerse apps do include a version field, but the apps are classified as "Unversioned". Tools (i.e. Samtools) that apps use are versioned, but my understanding of apps is that they are meant to be fixed/static once published. I looked through the most recent DE documentation and cannot find any mention of versioning for apps nor do I see a way of changing the app version via Terrain.

However, we created the IGB community so that we could control what apps are shown in BioViz Connect. This effectively gives us a way to control what "version" of an app we show to BioViz Connect users. We can keep the previous apps that produced bedgraph output and let users see and select between the two versions. I think there is also a way to add a dropdown selector for selecting desired output (bedgraph/bigwig) from deeptools, so we could just make that an option in the new apps.

I created two new apps that allow the user to select bigwig or bedgraph as the output. However, BioViz Connect seems to be having issues with the new "list" parameter included as part of the apps. When the apps were added to the IGB community and the sync apps database endpoint hit, it caused an error that led to the apps not appearing correctly in BioViz Connect. Unclear if we want to spend the time fixing this issue or if we should just release the apps with the output set as bigwig.

UPDATE: I created IGBF-3193 to address the issue with BioViz Connect not working with dropdown menus. I have moved the ticket to the backlog as we have decided to move forward with publishing the bigWig output only versions of the Apps. Follow the steps in the previous comment to complete this ticket.

1. App names are identical to the original apps.
2. Test input/output data are in iplant/home/shared/BioViz/example_data
3. Both apps have been published.
4. Apps have been synced to bvctest3.bioviz.org for testing.
5. Previous version of apps that produced bedgraph output have been removed from IGB Community.

Final steps for testing:

1. Test the new apps on bvctest3.bioviz.org using various accounts (not nowlanf).
2. Run sync apps database on BioViz Connect production.
3. Test the new apps on BioViz Connect production using various accounts (not nowlanf).

There seems to be an issue where two jobs are being submitted simultaneously for either scaled or unscaled. I have tested twice, and either scaled or unscaled analyses were duplicated.

This requires further investigation.

Details of the duplicate analyses. Note that the app_id are the same and the startdate are just slightly different.

        {
            "description": "",
            "name": "nowlanUnscaled_1",
            "can_share": true,
            "username": "nowlan@iplantcollaborative.org",
            "app_id": "6cd156c4-3dd4-11ed-8d36-008cfa5ae621",
            "system_id": "de",
            "app_disabled": false,
            "batch": false,
            "enddate": "1665088115037",
            "app_version_id": "6cd1ba9c-3dd4-11ed-8d36-008cfa5ae621",
            "status": "Completed",
            "id": "3fba5bee-45b4-11ed-b5bd-008cfa5ae621",
            "startdate": "1665087598141",
            "app_description": "Make a coverage graph file in bigWig format from a bam sequence alignment file using deepTools bamCoverage version 3.3.0 with options --binSize 1. See deeptools.readthedocs.io. %INPUT%bam",
            "notify": true,
            "resultfolderid": "/iplant/home/nowlan/analyses",
            "app_name": "Make coverage graph"
        },
        {
            "description": "",
            "name": "nowlanUnscaled_1",
            "can_share": true,
            "username": "nowlan@iplantcollaborative.org",
            "app_id": "6cd156c4-3dd4-11ed-8d36-008cfa5ae621",
            "system_id": "de",
            "app_disabled": false,
            "batch": false,
            "enddate": "1665087948568",
            "app_version_id": "6cd1ba9c-3dd4-11ed-8d36-008cfa5ae621",
            "status": "Completed",
            "id": "3fb2e92c-45b4-11ed-8f0c-008cfa5ae621",
            "startdate": "1665087598084",
            "app_description": "Make a coverage graph file in bigWig format from a bam sequence alignment file using deepTools bamCoverage version 3.3.0 with options --binSize 1. See deeptools.readthedocs.io. %INPUT%bam",
            "notify": true,
            "resultfolderid": "/iplant/home/nowlan/analyses",
            "app_name": "Make coverage graph"
        },

Launching analyses from the DE itself does not appear to create duplicates. This would imply something is behaving incorrectly in BioViz Connect.

Tested on BioViz Connect with nowlan account on October 7. Running scaled and unscaled coverage graph analyses on community data did not produce duplicate analyses. Possible this was a momentary glitch.

Final testing on Mac on connect.bioviz.org

I ran four analyses using the nowlanf account:
Scaled and unscaled coverage graphs on files in user's home folder.
Scaled and unscaled coverage graphs on files in community folder.

All four analyses completed successfully and the output were visible in IGB.
Text description for each analyses appeared correct.

Closing ticket