Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-1647

DE uploaded files are broken by Agave public

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Situation: If a user uploads a file to the Discovery Environment, then uses the Agave api to make it public, it makes the file public and removes that user's permissions for that file. This is a blocker.

      Task: Contact TACC or John Fonner.

        Attachments

          Activity

          Hide
          nfreese Nowlan Freese added a comment -

          Hi Dr. Fonner, I've got another question, and I wanted to run it by you to make sure I'm not doing something wrong. Previously, when we would make a file public through Agave it would add the user public and give it read access. However, on Friday we noticed that our same API calls as before had a different behavior. Now when we make a file public, it removes our user permissions for the file, and creates two public users with access. Any call to change user permissions for that file return an error. Oddly, this only affected new files. Any files we had previously uploaded to CyVerse DE behaved correctly, and we could add/remove public access as normal. Tested on three different accounts, and confirmed the behavior again this morning. (edited)
          callsAndResponses.txt

          curl -X GET \
          https://agave.iplantc.org/files/v2/pems/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt \
          -H 'Authorization: Bearer access_token' \
          -H 'cache-control: no-cache'
          Click to expand inline (337 lines)

          John Fonner [10:28 AM]
          That's not at all the normal behavior.
          Sometimes when listing permissions, Agave doesn't give a full list, but the owner of a file can't lose their permissions on it. Can you give yourself permission on the file again?

          Nowlan Freese [10:30 AM]
          No, once we make it public, we are no longer allowed to remove public access.

          { "status": "error", "message": "User does not have permission to manage permissions on this file for directory.", "version": "2.2.23-r7987f7e" }

          John Fonner [10:32 AM]
          I show nowlanf has full permission on that file
          can you check who that token is for? The endpoint is: https://agave.iplantc.org/profiles/v2/me?pretty=true

          Nowlan Freese [10:34 AM]
          {
          "status": "success",
          "message": "User details retrieved successfully.",
          "version": "2.0.0-SNAPSHOT-rc3fad",
          "result":

          { "first_name": "Nowlan", "last_name": "Freese", "full_name": "Nowlan Freese", "email": "nfreese@uncc.edu", "phone": "", "mobile_phone": "", "status": "", "uid": 71509, "username": "nowlanf" }

          }
          Here's what I see when I check to see if public:
          {
          "username": "public",
          "internalUsername": null,
          "permission":

          { "read": true, "write": false, "execute": false }

          ,
          "recursive": false,
          "_links": {
          "self":

          { "href": "https://agave.iplantc.org/files/v2/pems/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt?username.eq=public" }

          ,
          "file":

          { "href": "https://agave.iplantc.org/files/v2/media/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt" }

          ,
          "profile":

          { "href": "https://agave.iplantc.org/profiles/v2/public" }

          }
          },
          {
          "username": "public",
          "internalUsername": null,
          "permission":

          { "read": true, "write": false, "execute": false }

          ,
          "recursive": false,
          "_links": {
          "self":

          { "href": "https://agave.iplantc.org/files/v2/pems/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt?username.eq=public" }

          ,
          "file":

          { "href": "https://agave.iplantc.org/files/v2/media/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt" }

          ,
          "profile":

          { "href": "https://agave.iplantc.org/profiles/v2/public" }

          }
          },

          John Fonner [10:36 AM]
          hmmm... nowlanf doesn't have "execute" permission on the test directory. I can't tell if that is required, though.
          I'm guessing it is okay. Yeah, it looks like that is how other folders are.
          Sorry, I'm trying to rule out anything sneaky before declaring the API has changed somehow.

          Nowlan Freese [10:39 AM]
          I completely understand, one of my students first identified this and he had to show me twice before I thought it was really happening.
          We save all of our calls in Postman, so we are pretty confident those haven't changed.

          John Fonner [10:42 AM]
          let me try to reproduce it. So you 1) create a file, 2) give read access to "public" 3) try to give "NONE" access to "public". That simple?

          Nowlan Freese [10:43 AM]
          Yep, and we are uploading the file through the CyVerse DE. Importantly, we think the file also has to have a new unique name (i.e. if you previously had a file named "test" in the same directory, and upload a new file named "test", the behavior may not be consistent).

          John Fonner [10:45 AM]
          Do you think uploading it through the DE matters?
          or that is just what you are doing?

          Nowlan Freese [10:46 AM]
          Not sure, that's how we've been doing it for all of our testing. However, after our conversation about CyVerse DE permissions being somewhat separated from Agave, it's possible.

          John Fonner [10:47 AM]
          it is possible, but for permissions Agave gets its permissions from iRODS and also mirrors permissions changes back to iRODS. So it isn't like the public link thing where different API endpoints are being called.
          have you tested sharing with a different user (not public)?

          Nowlan Freese [10:49 AM]
          Not yet, I can try right now.

          John Fonner [10:50 AM]
          I have a colleague with me trying to reproduce the permissions error too. I'll let you know if we can.

          Nowlan Freese [10:51 AM]
          Not sure what I did, but I just lost all access to my test directory in CyVerse.

          John Fonner [10:54 AM]
          well, we reproduced your permissions issue. Exactly like you describe. I'm going to pull in the API developers to take a look.

          Nowlan Freese [10:56 AM]
          Sounds good. Let me know if there is anything else you would like me to try.

          John Fonner [10:56 AM]
          ok

          Nowlan Freese [11:32 AM]
          Sharing with another user worked correctly (my user nowlanf retained permission and the new user nfreese was given read access).

          Nowlan Freese [10:18 AM]
          Hi Dr. Fonner, I was wondering if there was any update on the public access issue. This is a blocker for us, so any kind of eta would be great.

          John Fonner [10:49 AM]
          Not yet. We know that the issue only happens when the file is created in the DE. If you upload a file via Agave and share with public, permissions are okay.

          Nowlan Freese [11:20 AM]
          That's good to know, we will upload files via Agave to finish testing our website.

          Show
          nfreese Nowlan Freese added a comment - Hi Dr. Fonner, I've got another question, and I wanted to run it by you to make sure I'm not doing something wrong. Previously, when we would make a file public through Agave it would add the user public and give it read access. However, on Friday we noticed that our same API calls as before had a different behavior. Now when we make a file public, it removes our user permissions for the file, and creates two public users with access. Any call to change user permissions for that file return an error. Oddly, this only affected new files. Any files we had previously uploaded to CyVerse DE behaved correctly, and we could add/remove public access as normal. Tested on three different accounts, and confirmed the behavior again this morning. (edited) callsAndResponses.txt curl -X GET \ https://agave.iplantc.org/files/v2/pems/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt \ -H 'Authorization: Bearer access_token' \ -H 'cache-control: no-cache' Click to expand inline (337 lines) John Fonner [10:28 AM] That's not at all the normal behavior. Sometimes when listing permissions, Agave doesn't give a full list, but the owner of a file can't lose their permissions on it. Can you give yourself permission on the file again? Nowlan Freese [10:30 AM] No, once we make it public, we are no longer allowed to remove public access. { "status": "error", "message": "User does not have permission to manage permissions on this file for directory.", "version": "2.2.23-r7987f7e" } John Fonner [10:32 AM] I show nowlanf has full permission on that file can you check who that token is for? The endpoint is: https://agave.iplantc.org/profiles/v2/me?pretty=true Nowlan Freese [10:34 AM] { "status": "success", "message": "User details retrieved successfully.", "version": "2.0.0-SNAPSHOT-rc3fad", "result": { "first_name": "Nowlan", "last_name": "Freese", "full_name": "Nowlan Freese", "email": "nfreese@uncc.edu", "phone": "", "mobile_phone": "", "status": "", "uid": 71509, "username": "nowlanf" } } Here's what I see when I check to see if public: { "username": "public", "internalUsername": null, "permission": { "read": true, "write": false, "execute": false } , "recursive": false, "_links": { "self": { "href": "https://agave.iplantc.org/files/v2/pems/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt?username.eq=public" } , "file": { "href": "https://agave.iplantc.org/files/v2/media/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt" } , "profile": { "href": "https://agave.iplantc.org/profiles/v2/public" } } }, { "username": "public", "internalUsername": null, "permission": { "read": true, "write": false, "execute": false } , "recursive": false, "_links": { "self": { "href": "https://agave.iplantc.org/files/v2/pems/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt?username.eq=public" } , "file": { "href": "https://agave.iplantc.org/files/v2/media/system/data.iplantcollaborative.org/nowlanf/test/brandNewFile.txt" } , "profile": { "href": "https://agave.iplantc.org/profiles/v2/public" } } }, John Fonner [10:36 AM] hmmm... nowlanf doesn't have "execute" permission on the test directory. I can't tell if that is required, though. I'm guessing it is okay. Yeah, it looks like that is how other folders are. Sorry, I'm trying to rule out anything sneaky before declaring the API has changed somehow. Nowlan Freese [10:39 AM] I completely understand, one of my students first identified this and he had to show me twice before I thought it was really happening. We save all of our calls in Postman, so we are pretty confident those haven't changed. John Fonner [10:42 AM] let me try to reproduce it. So you 1) create a file, 2) give read access to "public" 3) try to give "NONE" access to "public". That simple? Nowlan Freese [10:43 AM] Yep, and we are uploading the file through the CyVerse DE. Importantly, we think the file also has to have a new unique name (i.e. if you previously had a file named "test" in the same directory, and upload a new file named "test", the behavior may not be consistent). John Fonner [10:45 AM] Do you think uploading it through the DE matters? or that is just what you are doing? Nowlan Freese [10:46 AM] Not sure, that's how we've been doing it for all of our testing. However, after our conversation about CyVerse DE permissions being somewhat separated from Agave, it's possible. John Fonner [10:47 AM] it is possible, but for permissions Agave gets its permissions from iRODS and also mirrors permissions changes back to iRODS. So it isn't like the public link thing where different API endpoints are being called. have you tested sharing with a different user (not public)? Nowlan Freese [10:49 AM] Not yet, I can try right now. John Fonner [10:50 AM] I have a colleague with me trying to reproduce the permissions error too. I'll let you know if we can. Nowlan Freese [10:51 AM] Not sure what I did, but I just lost all access to my test directory in CyVerse. John Fonner [10:54 AM] well, we reproduced your permissions issue. Exactly like you describe. I'm going to pull in the API developers to take a look. Nowlan Freese [10:56 AM] Sounds good. Let me know if there is anything else you would like me to try. John Fonner [10:56 AM] ok Nowlan Freese [11:32 AM] Sharing with another user worked correctly (my user nowlanf retained permission and the new user nfreese was given read access). Nowlan Freese [10:18 AM] Hi Dr. Fonner, I was wondering if there was any update on the public access issue. This is a blocker for us, so any kind of eta would be great. John Fonner [10:49 AM] Not yet. We know that the issue only happens when the file is created in the DE. If you upload a file via Agave and share with public, permissions are okay. Nowlan Freese [11:20 AM] That's good to know, we will upload files via Agave to finish testing our website.
          Hide
          nfreese Nowlan Freese added a comment -

          Closing this issue. The workaround in IGBF-1646 makes this issue obsolete.

          Show
          nfreese Nowlan Freese added a comment - Closing this issue. The workaround in IGBF-1646 makes this issue obsolete.

            People

            • Assignee:
              nfreese Nowlan Freese
              Reporter:
              nfreese Nowlan Freese
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: