Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2455

Investigate requirements for IRB approval (if any) for testing user interfaces with people

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Story Points:
      5
    • Sprint:
      Summer 3: 6 Jul - 17 Jul, Summer 4: 14 Jul - 28 Jul, Summer 5: 3 Aug - 14 Aug

      Description

      Testing the effects of medications or other kinds of health related treatments requires prior approval of the study protocol by an institutional review board (IRB) at your institution.

      The IRB is supposed to review the study protocol and make sure that the potential benefits of the research outweigh the risks, if any.

      However, whether or not a review of this type needs to be done for studies that don't involve medications or any kind of physical intervention is unclear.

      For example, is IRB review required for a study that evaluates a user interface design?

      To find out if the design is working properly, we need to observe people using it. The risk of someone being harmed is low, but a person who helps us test-run something that turns out to be a bad design might experience some discomfort.

      We need to investigate this before we move too far ahead in testing new designs for IGB and our software. So for this task, please do some research to find out what other groups at UNC Charlotte are doing with respect to study designs and making sure their design testing protocols are sound.

        Attachments

          Issue Links

            Activity

            Hide
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited

            Below is a copy of the content in an email sent to a UNCC IRB "Research and Economic Development" staff member:

            The Loraine Genome Visualization Lab would like to start some usability research for product improvement with the potential for using our results in publishable research in the future.

            We are trying to understand how these goals would work with IRB approval requirements. It seems to be that if we ever want to analyze and publish any data we collected from users interacting with our software, we would need to submit for IRB approval up-front?

            Is this accurate? Would we have to submit a complete protocol with materials for IRB approval for usability studies if we think there is a potential to generate generalizable, publishable research?

            Here is the response I received from "Research and Economic Development" at UNCC:

            If you will conduct Research with Human Subjects, you do need IRB approval up-front.

            Generally, with user studies, the initial IRB protocol defines all of the procedures for the planned user study. This includes all of the study materials. After this user study is conducted, you then submit a Modification to the protocol if/when you design another user study. Just like an initial approval, a Modification must be approved before you conduct any of the Research.

            So, I think it depends on what you mean by "complete protocol." You do need to submit a complete protocol and all study materials for the specific user study you want to conduct. But you don't need to have all of the details for a series of user studies if each subsequent user study is developed and designed based on the results from the previous user study. This is why the IRB process allows for Modifications. You can submit a Modification for each subsequent user study.

            If the intent is to conduct Research, IRB approval is needed. Generally, the goal is to conduct a study that will result in generalizable knowledge. The findings may not result in this but when designing a study, this is the goal.

            Here is the Research definition.
            Research means a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge.

            Show
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited Below is a copy of the content in an email sent to a UNCC IRB "Research and Economic Development" staff member: The Loraine Genome Visualization Lab would like to start some usability research for product improvement with the potential for using our results in publishable research in the future. We are trying to understand how these goals would work with IRB approval requirements. It seems to be that if we ever want to analyze and publish any data we collected from users interacting with our software, we would need to submit for IRB approval up-front? Is this accurate? Would we have to submit a complete protocol with materials for IRB approval for usability studies if we think there is a potential to generate generalizable, publishable research? Here is the response I received from "Research and Economic Development" at UNCC: If you will conduct Research with Human Subjects, you do need IRB approval up-front. Generally, with user studies, the initial IRB protocol defines all of the procedures for the planned user study. This includes all of the study materials. After this user study is conducted, you then submit a Modification to the protocol if/when you design another user study. Just like an initial approval, a Modification must be approved before you conduct any of the Research. So, I think it depends on what you mean by "complete protocol." You do need to submit a complete protocol and all study materials for the specific user study you want to conduct. But you don't need to have all of the details for a series of user studies if each subsequent user study is developed and designed based on the results from the previous user study. This is why the IRB process allows for Modifications. You can submit a Modification for each subsequent user study. If the intent is to conduct Research, IRB approval is needed. Generally, the goal is to conduct a study that will result in generalizable knowledge. The findings may not result in this but when designing a study, this is the goal. Here is the Research definition. Research means a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge.
            Hide
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited

            Here is a copy of content from email correspondence with a CCI faculty member. I sent the following:

            We are preparing to conduct some software usability studies.

            I know that usability research for pure product improvement doesn't require IRB approval, but the group would like to set ourselves up to use the data we collect in these studies as part of publishable research in the future. It seems like if we would ever want to publish using that data, we would have to prepare all of our materials and submit for IRB approval up-front anyway, even if all we end up doing is a regular, no-IRB-submission-needed study.

            Do you have any advice or know of any resources in this area I can look into?

            I am also trying to connect with others in CCI who have experience in this area. Do you know anyone else in the college or department who may be able to offer us some guidance?

            Below is the response I received:

            Many faculty are involved in user studies, and I know that the HCI faculty (naturally) and the Center for Education Innovation related faculty would be involved in user studies and may be able to provide guidance.

            In order to do IRB studies, those involved need to undergo CITI training and have a current certification. Which training to do can be confusing, but if it is the software interaction that you are testing, it is likely that you need the human behavioral track. If you have not done so, that would be a good place to start, as it provides an overview of the main issues and points of concern that would need documentation in and IRB application.

            As a place to start, I would suggest the university resource on human subjects: in particular, the menu items on "guidelines for research", "things you need to know", "training information", and "IRB application procedeure". Starting an application for IRB will give you an idea of the questions you will need to address.
            https://research.uncc.edu/departments/office-research-protections-and-integrity-orpi/human-subjects

            Show
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited Here is a copy of content from email correspondence with a CCI faculty member. I sent the following: We are preparing to conduct some software usability studies. I know that usability research for pure product improvement doesn't require IRB approval, but the group would like to set ourselves up to use the data we collect in these studies as part of publishable research in the future. It seems like if we would ever want to publish using that data, we would have to prepare all of our materials and submit for IRB approval up-front anyway, even if all we end up doing is a regular, no-IRB-submission-needed study. Do you have any advice or know of any resources in this area I can look into? I am also trying to connect with others in CCI who have experience in this area. Do you know anyone else in the college or department who may be able to offer us some guidance? Below is the response I received: Many faculty are involved in user studies, and I know that the HCI faculty (naturally) and the Center for Education Innovation related faculty would be involved in user studies and may be able to provide guidance. In order to do IRB studies, those involved need to undergo CITI training and have a current certification. Which training to do can be confusing, but if it is the software interaction that you are testing, it is likely that you need the human behavioral track. If you have not done so, that would be a good place to start, as it provides an overview of the main issues and points of concern that would need documentation in and IRB application. As a place to start, I would suggest the university resource on human subjects: in particular, the menu items on "guidelines for research", "things you need to know", "training information", and "IRB application procedeure". Starting an application for IRB will give you an idea of the questions you will need to address. https://research.uncc.edu/departments/office-research-protections-and-integrity-orpi/human-subjects
            Hide
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited

            Per UNCC human subjects research training information site: https://research.uncc.edu/departments/office-research-protections-and-integrity-orpi/human-subjects/training-information

            All research team members (Investigators, Co-investigators, Responsible Faculty, Research Associates, and Research Assistants) are required to complete human subjects training.

            The training course "Social & Behavioral Research (IRB) - Basic/Refresher" can be found at https://about.citiprogram.org/en/homepage/ after creating an account, adding an affiliated university "University of North Carolina at Charlotte", selecting to add a course, and selecting the curriculum for Social and Behavioral Research Investigators.

            the course will now be found in the courses tab as Courses Ready to Begin.

            Show
            rweidenh Logan Weidenhammer (Inactive) added a comment - - edited Per UNCC human subjects research training information site: https://research.uncc.edu/departments/office-research-protections-and-integrity-orpi/human-subjects/training-information All research team members (Investigators, Co-investigators, Responsible Faculty, Research Associates, and Research Assistants) are required to complete human subjects training. The training course "Social & Behavioral Research (IRB) - Basic/Refresher" can be found at https://about.citiprogram.org/en/homepage/ after creating an account, adding an affiliated university "University of North Carolina at Charlotte", selecting to add a course, and selecting the curriculum for Social and Behavioral Research Investigators. the course will now be found in the courses tab as Courses Ready to Begin.
            Hide
            rweidenh Logan Weidenhammer (Inactive) added a comment -

            Continuing work on CITI Training "Social & Behavioral Research (IRB) - Basic/Refresher" course. There are 14 modules. So far they seem to take about ~45 minutes each to complete the reading and the quiz.

            Show
            rweidenh Logan Weidenhammer (Inactive) added a comment - Continuing work on CITI Training "Social & Behavioral Research (IRB) - Basic/Refresher" course. There are 14 modules. So far they seem to take about ~45 minutes each to complete the reading and the quiz.
            Hide
            rweidenh Logan Weidenhammer (Inactive) added a comment -

            Course is completed. The completion credentials of all involved in research will need to be entered on any future IRB study applications.

            Show
            rweidenh Logan Weidenhammer (Inactive) added a comment - Course is completed. The completion credentials of all involved in research will need to be entered on any future IRB study applications.

              People

              • Assignee:
                rweidenh Logan Weidenhammer (Inactive)
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: