Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2288

Investigation of the existing python packages compatibility with python 3.8

    Details

    • Type: Story
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Several python packages have newer versions. Initial analysis of (a) has been attached.

      Below were directly observed issues.
      a. Mysqlclient throws an error during installation on python 3.8 version
      b. Pillow version 6.0.0 is not compatible with python3.8 https://pillow.readthedocs.io/en/latest/installation.html

        Attachments

          Activity

          Hide
          ann.loraine Ann Loraine added a comment -

          Chester Dias - can you let me know if this ticket is still valid or do we still need to investigate?

          Show
          ann.loraine Ann Loraine added a comment - Chester Dias - can you let me know if this ticket is still valid or do we still need to investigate?
          Hide
          ann.loraine Ann Loraine added a comment -

          Chester Dias - please take a look at this today and see if you can close it!

          Show
          ann.loraine Ann Loraine added a comment - Chester Dias - please take a look at this today and see if you can close it!
          Hide
          cdias1 Chester Dias (Inactive) added a comment - - edited

          Upon running installer for python3... apt installs python3.6 for now.....

          sudo apt install python3
          

          For the installation of python3.8 we need to explicitly mention python3.8 as a command below

          sudo apt install python3.8
          

          Current used Pillow version is 6.0 incompatible with python3.8 on higher
          The suggested immediate change for the Pillow version to 7.2 which will remain compatible with python3.6 as well as python3.8
          Investigating mysql issue on build.

          Show
          cdias1 Chester Dias (Inactive) added a comment - - edited Upon running installer for python3... apt installs python3.6 for now..... sudo apt install python3 For the installation of python3.8 we need to explicitly mention python3.8 as a command below sudo apt install python3.8 Current used Pillow version is 6.0 incompatible with python3.8 on higher The suggested immediate change for the Pillow version to 7.2 which will remain compatible with python3.6 as well as python3.8 Investigating mysql issue on build.
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          RCA for mysqlclient installation
          The mysqlclient version in the python_packages.txt installs properly on python3.6 but throws an error on python3.8 due to missing python3.8-dev package that can be installed using

          sudo apt install python3.8-dev

          The latest version available is mysqlclient-2.0.1 while the one mentioned is 1.4.2.post1 both have been tested to install correctly on python3.8...

          Summary of the issue
          It seems that when we install python3 by default what is getting installed in v3.6. To install 3.8 we need to explicitly install v3.8. In the future if these defaults changes then we will end up with incompatible packages....
          Suggestion:
          Change pillow version to 7.2 on some local machine test out the appstore codebase. Ensure to test this on python3.8 and 3.6
          Add for apt installation of python3-dev and-or python3.8-dev packages for immediate fix in the process of infrastructure deployment setup.

          Show
          cdias1 Chester Dias (Inactive) added a comment - RCA for mysqlclient installation The mysqlclient version in the python_packages.txt installs properly on python3.6 but throws an error on python3.8 due to missing python3.8-dev package that can be installed using sudo apt install python3.8-dev The latest version available is mysqlclient-2.0.1 while the one mentioned is 1.4.2.post1 both have been tested to install correctly on python3.8... Summary of the issue It seems that when we install python3 by default what is getting installed in v3.6. To install 3.8 we need to explicitly install v3.8. In the future if these defaults changes then we will end up with incompatible packages.... Suggestion: Change pillow version to 7.2 on some local machine test out the appstore codebase. Ensure to test this on python3.8 and 3.6 Add for apt installation of python3-dev and-or python3.8-dev packages for immediate fix in the process of infrastructure deployment setup.
          Hide
          ann.loraine Ann Loraine added a comment -

          For the next steps, please specify which code changes need to be made on:

          Which files need to be modified?

          Show
          ann.loraine Ann Loraine added a comment - For the next steps, please specify which code changes need to be made on: https://bitbucket.org/lorainelab/a/ppstoresrc/master/ (appstore codebase) https://bitbucket.org/lorainelab/appstore-playbooks/src/master/ (appstore playbooks) Which files need to be modified?
          Hide
          cdias1 Chester Dias (Inactive) added a comment - - edited

          For the existing setup
          https://bitbucket.org/lorainelab/appstore-playbooks/src/master/roles/install/tasks/main.yml
          Change python-dev to python3-dev

          https://bitbucket.org/lorainelab/appstore/src/master/python_packages.txt
          Change pillow version to 7.2.0
          No change needed for MySQL client but need to install python3-dev.

          For setup of python3.8 we will need to add logic for correct setup and installation and additional modifications(ansible interpreter used) in playbooks are needed.

          Show
          cdias1 Chester Dias (Inactive) added a comment - - edited For the existing setup https://bitbucket.org/lorainelab/appstore-playbooks/src/master/roles/install/tasks/main.yml Change python-dev to python3-dev https://bitbucket.org/lorainelab/appstore/src/master/python_packages.txt Change pillow version to 7.2.0 No change needed for MySQL client but need to install python3-dev. For setup of python3.8 we will need to add logic for correct setup and installation and additional modifications(ansible interpreter used) in playbooks are needed.
          Hide
          ann.loraine Ann Loraine added a comment -

          Please clarify and edit above comment if required:

          • If we uncomment mysqlclient in python_packages.txt, then the for-development setup on local will probably fail for most people because they will not have the required dependencies on their local. During deployment with ansible, we actually uncomment this line. Please review playbooks again to check that this is happening. Do your configuration changes require that we uncomment mysqlclient in python_packages.txt?
          • Please clarify what we need to do for:

          For setup of python3.8 we will need to add logic for correct setup and installation and additional modifications(ansible interpreter used) in playbooks are needed.

          Show
          ann.loraine Ann Loraine added a comment - Please clarify and edit above comment if required: If we uncomment mysqlclient in python_packages.txt, then the for-development setup on local will probably fail for most people because they will not have the required dependencies on their local. During deployment with ansible, we actually uncomment this line. Please review playbooks again to check that this is happening. Do your configuration changes require that we uncomment mysqlclient in python_packages.txt? Please clarify what we need to do for: For setup of python3.8 we will need to add logic for correct setup and installation and additional modifications(ansible interpreter used) in playbooks are needed.
          Hide
          cdias1 Chester Dias (Inactive) added a comment -

          The venv currently uses python3->python3.6
          In order to use python3.8, we need to do the following.
          1. Install(edit install role task) packages python3.8, python3.8-dev, python3.8-venv
          2. Add the code to Change the softlink of python3 from pointing to python3.6 to python 3.8
          2. Change venv role task to use python3.8 to build the virtual environment.

          Show
          cdias1 Chester Dias (Inactive) added a comment - The venv currently uses python3->python3.6 In order to use python3.8, we need to do the following. 1. Install(edit install role task) packages python3.8, python3.8-dev, python3.8-venv 2. Add the code to Change the softlink of python3 from pointing to python3.6 to python 3.8 2. Change venv role task to use python3.8 to build the virtual environment.
          Hide
          ann.loraine Ann Loraine added a comment -

          For the next steps, let's make the recommended changes.
          I think we have time to do this during the current sprint, but we should wait until next week as full testing of appstore playbooks current iteration is not done.

          Show
          ann.loraine Ann Loraine added a comment - For the next steps, let's make the recommended changes. I think we have time to do this during the current sprint, but we should wait until next week as full testing of appstore playbooks current iteration is not done.

            People

            • Assignee:
              cdias1 Chester Dias (Inactive)
              Reporter:
              cdias1 Chester Dias (Inactive)
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: