Details
-
Type:
Bug
-
Status: Closed (View Workflow)
-
Priority:
Major
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:None
-
Story Points:3
-
Epic Link:
-
Sprint:Fall 3 2022 Sep 26, Fall 4 2022 Oct 10
Description
Situation: When using the Search bar in BioViz Connect, the returned list of files do not have the View in IGB button.
Task: Fix BioViz Connect so that when the Search bar is used any resulting files that can be viewed in IGB have the View in IGB button.
Attachments
Activity
Merged
New Pull request has been created. https://bitbucket.org/nfreese/bioviz-connect/pull-requests/276
Drop the username from parse_individual_search_item since we won't need it and then squash the commits and should be good.
Changes have been done. See commit
It seems like we could avoid some additional logic by looking at the fields:permission instead of the source:userPermissions. The fields:permission should have the correct user permission status for the user.
Changes have been made for the 'Make public' modal issue with search results when View in IGB button is clicked. See commit
Search results in community folder for file that is not owned by nowlanf:
{
"total": 2,
"max_score": null,
"hits": [
{
"matched_queries": null,
"_id": "05330864-eaf8-11e9-8c7d-90e2ba675364",
"_explanation": null,
"_routing": "",
"highlight": null,
"fields": {
"permission": [
"read"
]
},
"_type": "file",
"_parent": "",
"_score": null,
"_index": "data-1-2017.03.13",
"inner_hits": null,
"_version": null,
"_source": {
"userPermissions": [
{
"user": "aloraine#iplant",
"permission": "own"
},
{
"user": "anonymous#iplant",
"permission": "read"
},
{
"user": "kraveend91#iplant",
"permission": "write"
},
{
"user": "nfreese#iplant",
"permission": "own"
},
{
"user": "nowlanf#iplant",
"permission": "read"
},
{
"user": "public#iplant",
"permission": "read"
},
{
"user": "rodsadmin#iplant",
"permission": "own"
}
],
"path": "/iplant/home/shared/BioViz/rnaseq/A_thaliana_Jun_2009/SRP220157/reads/SRR10060893.bam",
"fileSize": 1730631085,
"creator": "nfreese#iplant",
"fileType": "generic",
"dateModified": 1570665474000,
"label": "SRR10060893.bam",
"id": "05330864-eaf8-11e9-8c7d-90e2ba675364",
"dateCreated": 1570665474000,
"metadata": [
{
"value": "bam",
"unit": "ipc-info-typer",
"attribute": "ipc-filetype"
}
]
},
"_uid": "",
"sort": [
1570665474000
]
},
{
"matched_queries": null,
"_id": "323ab5d4-eae2-11e9-8c7b-90e2ba675364",
"_explanation": null,
"_routing": "",
"highlight": null,
"fields": {
"permission": [
"read"
]
},
"_type": "file",
"_parent": "",
"_score": null,
"_index": "data-1-2017.03.13",
"inner_hits": null,
"_version": null,
"_source": {
"userPermissions": [
{
"user": "aloraine#iplant",
"permission": "own"
},
{
"user": "anonymous#iplant",
"permission": "read"
},
{
"user": "kraveend91#iplant",
"permission": "write"
},
{
"user": "nfreese#iplant",
"permission": "own"
},
{
"user": "nowlanf#iplant",
"permission": "read"
},
{
"user": "public#iplant",
"permission": "read"
},
{
"user": "rodsadmin#iplant",
"permission": "own"
}
],
"path": "/iplant/home/shared/BioViz/rnaseq/A_thaliana_Jun_2009/SRP220157/reads/SRR10060893.bam.bai",
"fileSize": 243280,
"creator": "nfreese#iplant",
"fileType": "generic",
"dateModified": 1570659282000,
"label": "SRR10060893.bam.bai",
"id": "323ab5d4-eae2-11e9-8c7b-90e2ba675364",
"dateCreated": 1570659282000,
"metadata": [
{
"value": "unknown",
"unit": "ipc-info-typer",
"attribute": "ipc-filetype"
}
]
},
"_uid": "",
"sort": [
1570659282000
]
}
]
}
Looks like the issue is similar to the isViewable problem we tackled above. In ResponseParser.py we never set the value for "permission" in parse_individual_search_item whereas we do set permission in parse_file_system_info. I'm guessing the default value for permission is "not own".
There is a parse_each_Search method in ResponseParser.py but it looks like it is only called when CheckFilePermission.py is being used.
Here are the search results for a specific file in the nowlanf home directory. The file is NOT public.
Note that the results show the file permissions are set to "own".
{
"total": 1,
"max_score": null,
"hits": [
{
"matched_queries": null,
"_id": "63bb44e8-3dd7-11ed-9bfe-90e2ba675364",
"_explanation": null,
"_routing": "",
"highlight": null,
"fields": {
"permission": [
"own"
]
},
"_type": "file",
"_parent": "",
"_score": null,
"_index": "data-1-2017.03.13",
"inner_hits": null,
"_version": null,
"_source": {
"userPermissions": [
{
"permission": "own",
"user": "nowlanf#iplant"
},
{
"permission": "own",
"user": "rodsadmin#iplant"
}
],
"path": "/iplant/home/nowlanf/CyVerse_Webinar_doNotAlter/reads/2022-9-26_bigwig_893_scaled_test1.bigwig",
"fileSize": 63082526,
"creator": "nowlanf#iplant",
"fileType": "generic",
"dateModified": 1664223081000,
"label": "2022-9-26_bigwig_893_scaled_test1.bigwig",
"id": "63bb44e8-3dd7-11ed-9bfe-90e2ba675364",
"dateCreated": 1664223081000,
"metadata": [
{
"value": "bigwig",
"unit": "ipc-info-typer",
"attribute": "ipc-filetype"
},
{
"value": "3adaa6b0-39c5-11ed-8bdd-008cfa5ae621",
"unit": "UUID",
"attribute": "ipc-analysis-id"
},
{
"value": "c7dd9ad7-01d4-4508-8d3e-f9f1def9904e",
"unit": "UUID",
"attribute": "ipc-execution-id"
}
]
},
"_uid": "",
"sort": [
1664223081000
]
}
]
}
Testing on bvctest3.bioviz.org
View in IGB appears in the search results for the appropriate files. Clicking on View in IGB for a public file successfully opens the file in IGB.
However, when a file in the search results is not public and the user clicks View in IGB, a modal is supposed to appear that informs the user the file is not public and there is the option to make the file public. Instead what is happening is that the toast is appearing telling the user to contact the owner of the file to make the file public. This would imply that when BioViz Connect checks if the search file is owned/not owned/public it is coming back as not owned, even though the file I was testing was owned by the user.
This is probably due to the differences in how Search results are parsed and should be addressed as part of this ticket.
Moving back to to-do.
AL: to re-run playbooks for test sites
Merged.
Will need to push latest changes to one of the bvctest sites and test once CyVerse is available after today's maintenance.
My only complaint is that the variable is named fileBool instead of isFile.
Code looks good. I vote for doing one more commit and changing fileBool to isFile and then squashing all of the commits before doing pull request.
Changes have been pushed. See commit
Note: parse_individual_search_folder has been renamed to parse_individual_search_item which makes more sense
After discussion with Karthik we found that the searchForFile.py is calling:
ResponseParser.parse_searchResponse -> parse_filesearchResponse -> parse_individual_search_folder
At no point in the parsing of the search responses is isViewable being called to check if the files are viewable by IGB. I think we just need to make a call at some point, maybe within the parse_individual_search_folder method. May be worth investigating why both files and folders are being parsed by parse_individual_search_folder.
Here's my interpretation of the current BioViz Connect code:
In cyverse.js we populate tables of files/folders retrieved (populateTable) and there is a call to check if tableData[data].isViewable is true.
In ResponseParser.py isViewable compares the file extension against the array of igbfileormats which contains all of the file formats accepted by IGB (can be seen in ManageAppDetails.py).
So it seems like isViewable is performing the same function as the previous validateFileExtension. But why is isViewable being set to false for files found during search? Is there something in the search data that's not being parsed correctly?
Formats that are validated in the function mentioned above
2bit,axml,bam,bar,bb,bdg,bed,bedgraph,bgn,bgr,bigbed,bigWig,bigwig,bnib,bp1,bps,broadPeak,broadpeak,
brpt,brs,bsnp,bw,chp,cn_segments,cnchp,cnt,cyt,das,das2xml,dasxml,ead,egr,egr.txt,fa,fas,fasta,
fna,fsa,fsh,gb,gen,gff,gff3,gr,gtf,link.psl,loh_segments,lohchp,map,mpfa,narrowPeak,narrowpeak,
psl,psl3,pslx,sam,sgr,sin,tally,useq,var,vcf,wig
See Commit
For "View in IGB" button to be added against a file, early in development we used "validateFileExtension" function to check the file extensions in file names. Later, we changed it to use the "isViewable" attribute of a file to check if the file is viewable in IGB and it worked for the most part. However, when the search function is used "isViewable" attribute is set as false for all files. I am not sure if this is new development in the Cyverse API response for files and folders but it seems like it.
To work around this issue, for search function alone, the "validateFileExtension" function from IGBF-1621 was revived which requires rigorous testing as I am not sure why its use was discontinued.
Tested on bvctest3.bioviz.org using the nowlanf account on macOS 12.6 with FireFox browser 105.0.3.
When I click View in IGB in Search on a file that is not public the modal appears and I am able to make the file public and it is then viewed in IGB.
Closing ticket.