Details
-
Type:
Bug
-
Status: Closed (View Workflow)
-
Priority:
Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:None
-
Story Points:3
-
Epic Link:
-
Sprint:Fall 3 2022 Sep 26, Fall 4 2022 Oct 10
Description
Situation: When using the Search bar in BioViz Connect, the returned list of files do not have the View in IGB button.
Task: Fix BioViz Connect so that when the Search bar is used any resulting files that can be viewed in IGB have the View in IGB button.
Attachments
Activity
| Field | Original Value | New Value |
|---|---|---|
| Epic Link | IGBF-2376 [ 18533 ] |
| Issue Type | Task [ 3 ] | Bug [ 1 ] |
| Assignee | Karthik Raveendran [ karthik ] |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
| Story Points | 1 | 2 |
Formats that are validated in the function mentioned above
2bit,axml,bam,bar,bb,bdg,bed,bedgraph,bgn,bgr,bigbed,bigWig,bigwig,bnib,bp1,bps,broadPeak,broadpeak,
brpt,brs,bsnp,bw,chp,cn_segments,cnchp,cnt,cyt,das,das2xml,dasxml,ead,egr,egr.txt,fa,fas,fasta,
fna,fsa,fsh,gb,gen,gff,gff3,gr,gtf,link.psl,loh_segments,lohchp,map,mpfa,narrowPeak,narrowpeak,
psl,psl3,pslx,sam,sgr,sin,tally,useq,var,vcf,wig
See Commit
| Assignee | Karthik Raveendran [ karthik ] |
| Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
| Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
| Assignee | Nowlan Freese [ nfreese ] |
Here's my interpretation of the current BioViz Connect code:
In cyverse.js we populate tables of files/folders retrieved (populateTable) and there is a call to check if tableData[data].isViewable is true.
In ResponseParser.py isViewable compares the file extension against the array of igbfileormats which contains all of the file formats accepted by IGB (can be seen in ManageAppDetails.py).
So it seems like isViewable is performing the same function as the previous validateFileExtension. But why is isViewable being set to false for files found during search? Is there something in the search data that's not being parsed correctly?
After discussion with Karthik we found that the searchForFile.py is calling:
ResponseParser.parse_searchResponse -> parse_filesearchResponse -> parse_individual_search_folder
At no point in the parsing of the search responses is isViewable being called to check if the files are viewable by IGB. I think we just need to make a call at some point, maybe within the parse_individual_search_folder method. May be worth investigating why both files and folders are being parsed by parse_individual_search_folder.
| Assignee | Nowlan Freese [ nfreese ] | Karthik Raveendran [ karthik ] |
| Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Changes have been pushed. See commit
Note: parse_individual_search_folder has been renamed to parse_individual_search_item which makes more sense
| Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
| Assignee | Karthik Raveendran [ karthik ] |
My only complaint is that the variable is named fileBool instead of isFile.
Code looks good. I vote for doing one more commit and changing fileBool to isFile and then squashing all of the commits before doing pull request.
| Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
| Status | First Level Review in Progress [ 10301 ] | Ready for Pull Request [ 10304 ] |
| Status | Ready for Pull Request [ 10304 ] | Pull Request Submitted [ 10101 ] |
| Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
Merged.
Will need to push latest changes to one of the bvctest sites and test once CyVerse is available after today's maintenance.
| Status | Reviewing Pull Request [ 10303 ] | Merged Needs Testing [ 10002 ] |
AL: to re-run playbooks for test sites
| Assignee | Ann Loraine [ aloraine ] |
| Status | Merged Needs Testing [ 10002 ] | Post-merge Testing In Progress [ 10003 ] |
| Assignee | Ann Loraine [ aloraine ] | Nowlan Freese [ nfreese ] |
Testing on bvctest3.bioviz.org
View in IGB appears in the search results for the appropriate files. Clicking on View in IGB for a public file successfully opens the file in IGB.
However, when a file in the search results is not public and the user clicks View in IGB, a modal is supposed to appear that informs the user the file is not public and there is the option to make the file public. Instead what is happening is that the toast is appearing telling the user to contact the owner of the file to make the file public. This would imply that when BioViz Connect checks if the search file is owned/not owned/public it is coming back as not owned, even though the file I was testing was owned by the user.
This is probably due to the differences in how Search results are parsed and should be addressed as part of this ticket.
Moving back to to-do.
| Status | Post-merge Testing In Progress [ 10003 ] | To-Do [ 10305 ] |
| Assignee | Nowlan Freese [ nfreese ] |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
| Assignee | Nowlan Freese [ nfreese ] |
Here are the search results for a specific file in the nowlanf home directory. The file is NOT public.
Note that the results show the file permissions are set to "own".
{
"total": 1,
"max_score": null,
"hits": [
{
"matched_queries": null,
"_id": "63bb44e8-3dd7-11ed-9bfe-90e2ba675364",
"_explanation": null,
"_routing": "",
"highlight": null,
"fields": {
"permission": [
"own"
]
},
"_type": "file",
"_parent": "",
"_score": null,
"_index": "data-1-2017.03.13",
"inner_hits": null,
"_version": null,
"_source": {
"userPermissions": [
{
"permission": "own",
"user": "nowlanf#iplant"
},
{
"permission": "own",
"user": "rodsadmin#iplant"
}
],
"path": "/iplant/home/nowlanf/CyVerse_Webinar_doNotAlter/reads/2022-9-26_bigwig_893_scaled_test1.bigwig",
"fileSize": 63082526,
"creator": "nowlanf#iplant",
"fileType": "generic",
"dateModified": 1664223081000,
"label": "2022-9-26_bigwig_893_scaled_test1.bigwig",
"id": "63bb44e8-3dd7-11ed-9bfe-90e2ba675364",
"dateCreated": 1664223081000,
"metadata": [
{
"value": "bigwig",
"unit": "ipc-info-typer",
"attribute": "ipc-filetype"
},
{
"value": "3adaa6b0-39c5-11ed-8bdd-008cfa5ae621",
"unit": "UUID",
"attribute": "ipc-analysis-id"
},
{
"value": "c7dd9ad7-01d4-4508-8d3e-f9f1def9904e",
"unit": "UUID",
"attribute": "ipc-execution-id"
}
]
},
"_uid": "",
"sort": [
1664223081000
]
}
]
}
Looks like the issue is similar to the isViewable problem we tackled above. In ResponseParser.py we never set the value for "permission" in parse_individual_search_item whereas we do set permission in parse_file_system_info. I'm guessing the default value for permission is "not own".
There is a parse_each_Search method in ResponseParser.py but it looks like it is only called when CheckFilePermission.py is being used.
| Status | In Progress [ 3 ] | To-Do [ 10305 ] |
| Assignee | Nowlan Freese [ nfreese ] | Karthik Raveendran [ karthik ] |
| Sprint | Fall 3 2022 Sep 26 [ 155 ] | Fall 3 2022 Sep 26, Fall 4 2022 Oct 10 [ 155, 156 ] |
| Rank | Ranked higher |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | To-Do [ 10305 ] |
Search results in community folder for file that is not owned by nowlanf:
{
"total": 2,
"max_score": null,
"hits": [
{
"matched_queries": null,
"_id": "05330864-eaf8-11e9-8c7d-90e2ba675364",
"_explanation": null,
"_routing": "",
"highlight": null,
"fields": {
"permission": [
"read"
]
},
"_type": "file",
"_parent": "",
"_score": null,
"_index": "data-1-2017.03.13",
"inner_hits": null,
"_version": null,
"_source": {
"userPermissions": [
{
"user": "aloraine#iplant",
"permission": "own"
},
{
"user": "anonymous#iplant",
"permission": "read"
},
{
"user": "kraveend91#iplant",
"permission": "write"
},
{
"user": "nfreese#iplant",
"permission": "own"
},
{
"user": "nowlanf#iplant",
"permission": "read"
},
{
"user": "public#iplant",
"permission": "read"
},
{
"user": "rodsadmin#iplant",
"permission": "own"
}
],
"path": "/iplant/home/shared/BioViz/rnaseq/A_thaliana_Jun_2009/SRP220157/reads/SRR10060893.bam",
"fileSize": 1730631085,
"creator": "nfreese#iplant",
"fileType": "generic",
"dateModified": 1570665474000,
"label": "SRR10060893.bam",
"id": "05330864-eaf8-11e9-8c7d-90e2ba675364",
"dateCreated": 1570665474000,
"metadata": [
{
"value": "bam",
"unit": "ipc-info-typer",
"attribute": "ipc-filetype"
}
]
},
"_uid": "",
"sort": [
1570665474000
]
},
{
"matched_queries": null,
"_id": "323ab5d4-eae2-11e9-8c7b-90e2ba675364",
"_explanation": null,
"_routing": "",
"highlight": null,
"fields": {
"permission": [
"read"
]
},
"_type": "file",
"_parent": "",
"_score": null,
"_index": "data-1-2017.03.13",
"inner_hits": null,
"_version": null,
"_source": {
"userPermissions": [
{
"user": "aloraine#iplant",
"permission": "own"
},
{
"user": "anonymous#iplant",
"permission": "read"
},
{
"user": "kraveend91#iplant",
"permission": "write"
},
{
"user": "nfreese#iplant",
"permission": "own"
},
{
"user": "nowlanf#iplant",
"permission": "read"
},
{
"user": "public#iplant",
"permission": "read"
},
{
"user": "rodsadmin#iplant",
"permission": "own"
}
],
"path": "/iplant/home/shared/BioViz/rnaseq/A_thaliana_Jun_2009/SRP220157/reads/SRR10060893.bam.bai",
"fileSize": 243280,
"creator": "nfreese#iplant",
"fileType": "generic",
"dateModified": 1570659282000,
"label": "SRR10060893.bam.bai",
"id": "323ab5d4-eae2-11e9-8c7b-90e2ba675364",
"dateCreated": 1570659282000,
"metadata": [
{
"value": "unknown",
"unit": "ipc-info-typer",
"attribute": "ipc-filetype"
}
]
},
"_uid": "",
"sort": [
1570659282000
]
}
]
}
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | To-Do [ 10305 ] |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Changes have been made for the 'Make public' modal issue with search results when View in IGB button is clicked. See commit
| Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
| Assignee | Karthik Raveendran [ karthik ] |
| Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
| Assignee | Nowlan Freese [ nfreese ] |
It seems like we could avoid some additional logic by looking at the fields:permission instead of the source:userPermissions. The fields:permission should have the correct user permission status for the user.
| Assignee | Nowlan Freese [ nfreese ] | Karthik Raveendran [ karthik ] |
| Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
Changes have been done. See commit
| Assignee | Karthik Raveendran [ karthik ] |
| Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
| Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
| Assignee | Nowlan Freese [ nfreese ] |
Drop the username from parse_individual_search_item since we won't need it and then squash the commits and should be good.
| Status | First Level Review in Progress [ 10301 ] | To-Do [ 10305 ] |
| Assignee | Nowlan Freese [ nfreese ] | Karthik Raveendran [ karthik ] |
| Status | To-Do [ 10305 ] | In Progress [ 3 ] |
New Pull request has been created. https://bitbucket.org/nfreese/bioviz-connect/pull-requests/276
| Status | In Progress [ 3 ] | Needs 1st Level Review [ 10005 ] |
| Status | Needs 1st Level Review [ 10005 ] | First Level Review in Progress [ 10301 ] |
| Status | First Level Review in Progress [ 10301 ] | Ready for Pull Request [ 10304 ] |
| Status | Ready for Pull Request [ 10304 ] | Pull Request Submitted [ 10101 ] |
| Status | Pull Request Submitted [ 10101 ] | Reviewing Pull Request [ 10303 ] |
| Status | Reviewing Pull Request [ 10303 ] | Merged Needs Testing [ 10002 ] |
| Assignee | Karthik Raveendran [ karthik ] |
Merged
| Status | Merged Needs Testing [ 10002 ] | Post-merge Testing In Progress [ 10003 ] |
| Assignee | Nowlan Freese [ nfreese ] |
Tested on bvctest3.bioviz.org using the nowlanf account on macOS 12.6 with FireFox browser 105.0.3.
When I click View in IGB in Search on a file that is not public the modal appears and I am able to make the file public and it is then viewed in IGB.
Closing ticket.
| Story Points | 2 | 3 |
| Assignee | Nowlan Freese [ nfreese ] | Karthik Raveendran [ karthik ] |
| Resolution | Done [ 10000 ] | |
| Status | Post-merge Testing In Progress [ 10003 ] | Closed [ 6 ] |
For "View in IGB" button to be added against a file, early in development we used "validateFileExtension" function to check the file extensions in file names. Later, we changed it to use the "isViewable" attribute of a file to check if the file is viewable in IGB and it worked for the most part. However, when the search function is used "isViewable" attribute is set as false for all files. I am not sure if this is new development in the Cyverse API response for files and folders but it seems like it.
To work around this issue, for search function alone, the "validateFileExtension" function from
IGBF-1621was revived which requires rigorous testing as I am not sure why its use was discontinued.