Make sure to update python_packages.txt with any new modules required. See commits for how to use pip freeze to make sure all dependencies are listed.

Issue:
Implement data cleaning mechanism to remove stale entries from AppPending table and S3 bucket

Solution:
To delete the stale entries from the AppPending table and the jar from the S3 bucket implemented a scheduler which will run every midnight and delete the entries from the AppPending table for the entries with submitter_approval="False".

Use below website to get the cronstring of the job if required to be changed it in future:
https://crontab.guru/#0_0_*_*_*

Files changed:
Code changes are present in branch https://bitbucket.org/rpatil14/appstore_riddhi/branch/IGBF-2087#diff
python_packages.txt
submit_app/app_pending_cleanup.py
submit_app/apps.py

Testing steps:
Scenario 1:
1) Click on submit your app
2) Select an app to submit from "Choose File" option or Enter URL
3) Go back to the home page or click browser back button without submitting the app
4) Entry will be created in the AppPending table with submitter_approved value as "False" and the jar file will be uploaded to the s3 bucket
Expected output: Once the scheduler run at midnight this entry will be deleted from the AppPending table and the jar file will be deleted from the s3 bucket

Scenario 2:
1) Click on submit your app
2) Select an app to submit from "Choose File" option or Enter URL
3) Go back to the home page or click browser back button without submitting the app
4) Repeat step 1-2 for the same app with same version and this time submit the app by clicking "It's all good. Submit my app" button
5) Two entries should be created in the AppPending table with one entry submitter_approved value as "False" and other submitter_approved value as "True". Only one jar should be uploaded to the s3 bucket with the latest submission timestamp
Expected output: Once the scheduler run at midnight entry submitter_approved value as "False" will be deleted from the AppPending table and the jar file will not be deleted from the s3 bucket

Testing results:
Code changes are deployed to https://dev-appstore-6.bioviz.org. Admin username and password for devappstore6 is same as devappstore2

Note:
Run command "pip install -r python_packages.txt" to install new package "apscheduler" required for this task

cc.[~aloraine]

A request:

Can you modify the scheduler so that it only deletes "stale" apps that are more than 5 hours old?

Delete entries from app table which are 5 hours old.
https://bitbucket.org/rpatil14/appstore_riddhi/branch/IGBF-2087#diff

cc. [~aloraine]

pull request submitted:
https://bitbucket.org/lorainelab/appstore/pull-requests/153/igbf-2087/diff

To change the time span for deleting a stale app, edit: app_pending_cleanup.py
Will make this configuration vai settings.py and settings.ini.template

Merged into the master branch and deployed to https://dev-appstore-2.bioviz.org for testing.

Shamika Gajanan Kulkarni This has been sitting in the testing queue assigned to yourself for about a week. Please indicate the status - does it work and can you now close it?

Prof [~aloraine] I have successfully tested the first scenario mentioned above. Today I started on the second and will update the status tomorrow as the scheduler will run at midnight. I should be able to close it then.

This ticket has been tested and I had observed that the jar for the app without submitter approval got deleted when the scheduler runs. I just want to make sure that it was done by the scheduler itself and not someone else who was working on Testing the Appstore tickets. Just to be sure I am testing this once again today. Hope it's okay.

Successfully tested the ticket. Observed the following:
1) For the settings made recently, the scheduler runs every 2 hours [odd hours of the day like 13:00, 15:00, etc.] and the offset value is set to 1 hour.
2) The entry for app with submitter_approved value as false in AppPending table is deleted if the app was uploaded before one hour time span. The jar is also deleted from S3 pending_releases. [Get Average Score was used to test this]
3) When I tried to submit an app and returned to home page without clicking on Submit, an entry for the app was created in AppPending table with submitter_approved value as false. When I try to resubmit the same app and follow the entire process, there is only one entry in AppPending Table with submitter_approved value as True and the jar is replaced with the new timestamp in S3 pending_releases. When the scheduler runs, it does not delete this app entry or the jar from S3. [Merge Annotation Operator was used to test this]
4) If an app is uploaded within the one hour time span, and the submitter_approved value is False, it does not get deleted by the scheduler right now as this app entry is not treated as stale entry yet. [Super Simple IGB app was used to test this.]

Working as expected. Kindly refer the screenshots attached. Moving to Closed. Thank you.

Thank you Shamika Gajanan Kulkarni for the very clear & thorough description.