Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2846

Investigate: Create Add Data Source (Quickload) endpoint

    Details

    • Story Points:
      2
    • Sprint:
      Spring 4 2021 May 3 - May 14, Spring 6 2021 May 31 - June 11, Summer 1 2021 Jun 14 - Jun 25, Fall 1 2021 Aug 16 - Aug 27, Summer 1 2022 May 23, Summer 2 2022 June 6, Summer 3 2022 June 21

      Description

      Situation: QuickLoads can be manually added to IGB through the Data Sources tab in the Preferences window. It would be more user-friendly if there was a way for users to click on a link/button on a web page that would add the QuickLoad to IGB.

      Task: Examine the code in IGB and investigate strategies for creating an endpoint in IGB that accepts the various QuickLoad information and adds the QuickLoad to IGB. See the igbDefaultPrefs.json file for various QuickLoad parameters.

      Examine the App Store endpoint. Ideally, the endpoint would be able to answer queries as to whether the user's requested Quickload URL has already been added, much as the App Store endpoint does.

      We need a workflow that makes it possible for the javascript running a Web browser page to provide information for a user, similar to what we see in App pages in App Store.

        Attachments

          Issue Links

            Activity

            Hide
            nfreese Nowlan Freese added a comment -

            Important files:
            Core - Quickload (data source) -> org.lorainelab.igb.quickload -> QuickloadFactory.java, QuickloadDataProvider.java
            Core - IGB -> com.affymetrix.igb.general -> DataProviderManager.java
            Core - IGB -> com.affymetrix.igb.prefs -> AddDataProvider.java
            Core - Genometry (Data Models) -> com.affymetrix.genometry.data -> DataProviderFactoryManager.java

            General notes:
            AddDataProvider.java calls the createDataProvider method from QuickloadFactory.java and includes the Quickload url and name. The createDataProvider method creates a new QuickloadDataProvider with the Quickload url and name and passes it back to AddDataProvider.java where it is then added to the DataProviderManager.

            Show
            nfreese Nowlan Freese added a comment - Important files: Core - Quickload (data source) -> org.lorainelab.igb.quickload -> QuickloadFactory.java, QuickloadDataProvider.java Core - IGB -> com.affymetrix.igb.general -> DataProviderManager.java Core - IGB -> com.affymetrix.igb.prefs -> AddDataProvider.java Core - Genometry (Data Models) -> com.affymetrix.genometry.data -> DataProviderFactoryManager.java General notes: AddDataProvider.java calls the createDataProvider method from QuickloadFactory.java and includes the Quickload url and name. The createDataProvider method creates a new QuickloadDataProvider with the Quickload url and name and passes it back to AddDataProvider.java where it is then added to the DataProviderManager.
            Hide
            karthik Karthik Raveendran added a comment - - edited

            The workflow when a request is made to hit the bookmarks endpoint for quickload is the following:

            BookmarkHttpRequestHandler (class) -> processRequest() -> case: SERVLET_NAME ->parseAndGoToBookmark() -> (BookmarkUnibrowControlServlet.java) goToBookmark()

            Here an IllegalArgumentExceptionError is thrown in runInBackground() at the line:

            String preferredVersionName = LOOKUP.getPreferredName(version) 
            Error: synonyms can not be null
            

            However, for Quickloads, we can simply bypass this line as the majority of runInBackground() is not required because we are supposed to add a DataProvider to the DataProviderManager for a Quickload request.

            I bypassed most of runInBackground() by calling a new function, addDataProviderToManager(). However, at this point, IGB is refusing to run after maven clean install. Tracing back my steps to find what went wrong.

            Show
            karthik Karthik Raveendran added a comment - - edited The workflow when a request is made to hit the bookmarks endpoint for quickload is the following: BookmarkHttpRequestHandler (class) -> processRequest() -> case: SERVLET_NAME ->parseAndGoToBookmark() -> (BookmarkUnibrowControlServlet.java) goToBookmark() Here an IllegalArgumentExceptionError is thrown in runInBackground() at the line: String preferredVersionName = LOOKUP.getPreferredName(version) Error: synonyms can not be null However, for Quickloads, we can simply bypass this line as the majority of runInBackground() is not required because we are supposed to add a DataProvider to the DataProviderManager for a Quickload request. I bypassed most of runInBackground() by calling a new function, addDataProviderToManager(). However, at this point, IGB is refusing to run after maven clean install. Tracing back my steps to find what went wrong.
            Hide
            nfreese Nowlan Freese added a comment - - edited

            I have created a mockup in Google Drive of the flow for how a user would interact with the trackhub converter and trackhub public web pages here.

            The file is located in Google Drive > IGB Project Documentation and Plans > Track Hub > workflow

            Show
            nfreese Nowlan Freese added a comment - - edited I have created a mockup in Google Drive of the flow for how a user would interact with the trackhub converter and trackhub public web pages here . The file is located in Google Drive > IGB Project Documentation and Plans > Track Hub > workflow
            Hide
            karthik Karthik Raveendran added a comment -

            The storyboard of the potential backup solution for this issue is added here

            Show
            karthik Karthik Raveendran added a comment - The storyboard of the potential backup solution for this issue is added here
            Hide
            ann.loraine Ann Loraine added a comment -

            Nowlan to review storyboard for next level review.

            Show
            ann.loraine Ann Loraine added a comment - Nowlan to review storyboard for next level review.
            Hide
            nfreese Nowlan Freese added a comment -

            Notes on backup storyboard:

            I'm unsure if opening a new tab each time the user clicks on the Q (to add a Quickload) will work well. It may jarring for the user to both open a new webpage and bring IGB to front. However, we do need a way to show/inform the user of the steps they need to take to add the Quickload:

            1. User clicks Q (Data Sources tab of Preferences window is brought to front).
            2. User clicks Add...
            3. User enters name of Quickload.
            4. User pastes the URL of the Quickload.
            • User navigates back to IGB and selects correct genome.

            Potential ideas:

            • Instead of opening a new page, when the user clicks the Q we could show a modal or a toast that says something along the lines of "opening IGB so you can add the quickload" along with a link to the page that describes how to add quickloads. The modal/toast is present for a couple of seconds before the Data Sources tab and IGB are brought to front. So the user would get immediate feedback on their button press, but would also have a moment to read the instructions or click to get more information. The con would be balancing the amount of time that the modal/toast be available to the user before bringing IGB to the front.
            • Instead of opening a new page, we could have a set of instructions/image towards the top of the webpage that would describe how to use the webpage and add the Quickload to IGB. Then when the user clicks Q we bring IGB to front and no page is opened and we probably wouldn't need a modal/toast (except when IGB is not running).

            In general, I think the flow for the backup solution is more difficult, as it relies on the user adding the Quickload. Thus we need to make sure the user knows how to add the Quickload without making it overly obtrusive.

            Show
            nfreese Nowlan Freese added a comment - Notes on backup storyboard: I'm unsure if opening a new tab each time the user clicks on the Q (to add a Quickload) will work well. It may jarring for the user to both open a new webpage and bring IGB to front. However, we do need a way to show/inform the user of the steps they need to take to add the Quickload: User clicks Q (Data Sources tab of Preferences window is brought to front). User clicks Add... User enters name of Quickload. User pastes the URL of the Quickload. User navigates back to IGB and selects correct genome. Potential ideas: Instead of opening a new page, when the user clicks the Q we could show a modal or a toast that says something along the lines of "opening IGB so you can add the quickload" along with a link to the page that describes how to add quickloads. The modal/toast is present for a couple of seconds before the Data Sources tab and IGB are brought to front. So the user would get immediate feedback on their button press, but would also have a moment to read the instructions or click to get more information. The con would be balancing the amount of time that the modal/toast be available to the user before bringing IGB to the front. Instead of opening a new page, we could have a set of instructions/image towards the top of the webpage that would describe how to use the webpage and add the Quickload to IGB. Then when the user clicks Q we bring IGB to front and no page is opened and we probably wouldn't need a modal/toast (except when IGB is not running). In general, I think the flow for the backup solution is more difficult, as it relies on the user adding the Quickload. Thus we need to make sure the user knows how to add the Quickload without making it overly obtrusive.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Discussion during scrum:

            • Not a good idea to open a new tab because this will force the user to notice there are two new things: a new page in the web browser and something different about IGB requiring the user to do something
            • Loraine suggestion: Use javascript widget on the page to send the HTTP request to localhost when a user selects is, intercept response coming back from localhost (IGB), and then change appearance to show that the action succeeded (or did not)
            • Freese observation: data source tab is very confusing and difficult; buttons on the tab are a bit weird looking (e.g., "Add..." was hard to understand for a test user)
            • Opening up the Preferences is barely an improvement because that window is incredibly difficult to understand

            We realized that changing the table interface to make it the best it can be under the current technical constraints of IGB (e.g., limited functionality of localhost endpoints) would require a lot of work.
            We decided it would be less wasteful of our work to instead correct the technical constraint of lacking the precise localhost endpoint(s) that we think we need, e.g., adding a new QL data source.
            That is, let's say we do both in parallel - i.e., improve the table interface to use the current IGB with its current technical constraints while at the same time, adding new endpoint functionality to IGB. If the second task succeeds, which we know it will because we are awesome engineers, then we still would have to re-do the interface to take advantage of the new endpoint.

            Show
            ann.loraine Ann Loraine added a comment - - edited Discussion during scrum: Not a good idea to open a new tab because this will force the user to notice there are two new things: a new page in the web browser and something different about IGB requiring the user to do something Loraine suggestion: Use javascript widget on the page to send the HTTP request to localhost when a user selects is, intercept response coming back from localhost (IGB), and then change appearance to show that the action succeeded (or did not) Freese observation: data source tab is very confusing and difficult; buttons on the tab are a bit weird looking (e.g., "Add..." was hard to understand for a test user) Opening up the Preferences is barely an improvement because that window is incredibly difficult to understand We realized that changing the table interface to make it the best it can be under the current technical constraints of IGB (e.g., limited functionality of localhost endpoints) would require a lot of work. We decided it would be less wasteful of our work to instead correct the technical constraint of lacking the precise localhost endpoint(s) that we think we need, e.g., adding a new QL data source. That is, let's say we do both in parallel - i.e., improve the table interface to use the current IGB with its current technical constraints while at the same time, adding new endpoint functionality to IGB. If the second task succeeds, which we know it will because we are awesome engineers, then we still would have to re-do the interface to take advantage of the new endpoint.
            Hide
            ann.loraine Ann Loraine added a comment -

            AL: requests attaching the two storyboard documents to the ticket for safe-keeping.

            Show
            ann.loraine Ann Loraine added a comment - AL: requests attaching the two storyboard documents to the ticket for safe-keeping.
            Hide
            ann.loraine Ann Loraine added a comment -

            Further comment: This feature would have many uses. For example, we could have a page of "public Quickloads" user could find out about from us and also submit for others. For example, we could create a "Quickload Store" similar to App Store, where you would add or remove Quickloads to IGB.

            Show
            ann.loraine Ann Loraine added a comment - Further comment: This feature would have many uses. For example, we could have a page of "public Quickloads" user could find out about from us and also submit for others. For example, we could create a "Quickload Store" similar to App Store, where you would add or remove Quickloads to IGB.

              People

              • Assignee:
                karthik Karthik Raveendran
                Reporter:
                nfreese Nowlan Freese
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: