Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-1568

Write script that updates database using info from OBR XML - repository.xml

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:

      Description

      Write script (python) that reads relevant App information from OBR XML file "repository.xml" and imports data about Apps into App Store database.

      Specification:

      • Read XML, compare contents to database
      • Identify Apps via defining "coordinates" - bundle (App) version and symbolic name
      • If an App in XML also exists in database, check that the information contained in the XML matches what is in the database. If there is a difference, update it. Only compare relevant fields.

      Note: The database will contain info about Apps that is not in the XML, such as number of downloads, details about the authors, etc. Leave that stuff intact.

      • If an App in XML does not exist in the database, create new records for it in the database.

      Note that this script can update information about Apps and can also add entirely new Apps to the database.

      Implementation: Write this in python. Run/test on EC2. Can assume database is MySQL.

        Attachments

          Issue Links

            Activity

            Hide
            svallapu Sai Charan Reddy Vallapureddy (Inactive) added a comment -

            Branch : https://bitbucket.org/svallapu/appstore-charan/branch/IGBF-1568

            parseRepositoryXML.py file does the following things:

            1. takes input from the user (host address, username, password, and database name)
            2. Reads the XML file.
            3. Copy content from XML file to string to replace "&" with "&". It is because XML parser gave an exception when parsing ampersand(&).
            4. Gets the required content from the XML file using the elementTree parser and stores them in variables.
            5. Modify the variables to remove exception in the app store. It creates a row when there is no data. It updates the row when there is data.
            6. Connects to Mysql RDS database and stores the data.

            Database details:
            database : testdjango
            table name : apps_app

            base64 Parser should be used to decode the details field in the database
            Example:
            import base64
            details = base64.b64decode(details)

            Show
            svallapu Sai Charan Reddy Vallapureddy (Inactive) added a comment - Branch : https://bitbucket.org/svallapu/appstore-charan/branch/IGBF-1568 parseRepositoryXML.py file does the following things: 1. takes input from the user (host address, username, password, and database name) 2. Reads the XML file. 3. Copy content from XML file to string to replace "&" with "&". It is because XML parser gave an exception when parsing ampersand(&). 4. Gets the required content from the XML file using the elementTree parser and stores them in variables. 5. Modify the variables to remove exception in the app store. It creates a row when there is no data. It updates the row when there is data. 6. Connects to Mysql RDS database and stores the data. Database details: database : testdjango table name : apps_app base64 Parser should be used to decode the details field in the database Example: import base64 details = base64.b64decode(details)
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Functional testing:

            • Clone repo, checkout branch, run script with test "repository.xml" files
            • Check that database was modified according to specification

            Code review:

            • Check that a user can't accidentally corrupt database
            Show
            ann.loraine Ann Loraine added a comment - - edited Functional testing: Clone repo, checkout branch, run script with test "repository.xml" files Check that database was modified according to specification Code review: Check that a user can't accidentally corrupt database
            Hide
            ptambvek Pranav Sanjay Tambvekar (Inactive) added a comment -

            1. Need to add the fields which are mentioned as 'Not Null' in the document in Google Drive.
            Not sure how it worked on RDS, but on local database, it throws error like- Error while connecting to MySQL 1364 (HY000): Field 'stars' doesn't have a default value
            This can be avoided by having default values to the fields.
            2. Please include the dependencies you used (mysql-connector-python) in the installDependencies.sh

            Show
            ptambvek Pranav Sanjay Tambvekar (Inactive) added a comment - 1. Need to add the fields which are mentioned as 'Not Null' in the document in Google Drive. Not sure how it worked on RDS, but on local database, it throws error like- Error while connecting to MySQL 1364 (HY000): Field 'stars' doesn't have a default value This can be avoided by having default values to the fields. 2. Please include the dependencies you used (mysql-connector-python) in the installDependencies.sh
            Hide
            ptambvek Pranav Sanjay Tambvekar (Inactive) added a comment -

            Because of the way database and Appstore is is designed, It will be beneficial to write django-code for reading and populating the database

            Show
            ptambvek Pranav Sanjay Tambvekar (Inactive) added a comment - Because of the way database and Appstore is is designed, It will be beneficial to write django-code for reading and populating the database
            Hide
            svallapu Sai Charan Reddy Vallapureddy (Inactive) added a comment -

            The suggested way is to use Django code instead of python code. Closed the ticket.

            Show
            svallapu Sai Charan Reddy Vallapureddy (Inactive) added a comment - The suggested way is to use Django code instead of python code. Closed the ticket.
            Hide
            ann.loraine Ann Loraine added a comment - - edited

            Please move repository.xml & parseRepositoryXML.py to appstore/submit_app and submit pull request.
            (Make a new branch or squash commits first.)

            Show
            ann.loraine Ann Loraine added a comment - - edited Please move repository.xml & parseRepositoryXML.py to appstore/submit_app and submit pull request. (Make a new branch or squash commits first.)
            Hide
            svallapu Sai Charan Reddy Vallapureddy (Inactive) added a comment -

            Moved repository.xml & parseRepositoryXML.py to appstore/submit_app folder.

            Branch: https://bitbucket.org/svallapu/appstore-charan/branch/IGBF-1568

            Show
            svallapu Sai Charan Reddy Vallapureddy (Inactive) added a comment - Moved repository.xml & parseRepositoryXML.py to appstore/submit_app folder. Branch: https://bitbucket.org/svallapu/appstore-charan/branch/IGBF-1568

              People

              • Assignee:
                svallapu Sai Charan Reddy Vallapureddy (Inactive)
                Reporter:
                ann.loraine Ann Loraine
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: